Skip to main content

RBE Practicum Presentation: Soumya Balijepally | Image Captioning and Text Generation using Deep Reinforcement Learning.

robotics_banner.jpg

Various images of robots at Robotics Engineering WPI alt
WPI Robotics Engineering
Wednesday, May 12, 2021
2:30 pm to 3:30 pm

Robotics Engineering Practicum Presentation

 

Soumya Balijepally

Image Captioning and Text Generation using Deep Reinforcement Learning

 

Wednesday, May 12, 2021

2:30 PM - 3:30 PM

Virtual | Zoom: https://zoom.us/j/92710816108?pwd=Nnc3UDJzUXZvSjdzSGs1TkNwWUpjQT09#success

 

Abstract: Captioning an image in natural language is the task of explaining the contents of the image. As a result, an algorithm is required not just to model the relationships between visual and textual objects but also to produce both syntactically and semantically correct sets of sentences. Although Deep Learning architectures are capable of handling image captioning, they do have their own set of shortcomings. This work involves combining Deep Learning and Reinforcement Learning based architectures to tackle the complexities of image captioning like single caption generation, uniqueness, and fidelity of the generated captions using a general dataset like the COCO Dataset and also more specific datasets. Further, the work presents the comparative analysis of these algorithms through their performance on the Text generation aspect of image captioning.

Practicum Advisors:

Professor Loris Fichera, RBE, WPI

Professor Berk Calli, RBE, WPI

DEPARTMENT(S):