Enhancing Video Segment Discovery in Educational Content: A Conversational Approach with Retrieval-Augmented Generation

Abstract

Accessing information in educational video collections is not only a question of finding the right video, but also of locating the particular moment in which the relevant information is discussed. This becomes especially important for long-form recordings, where a user’s information need may correspond to only a small part of the overall content. This thesis investigates how retrieval-augmented generation (RAG) can support this form of segment-level access. It develops a prototype that takes natural-language questions as input, retrieves semantically relevant passages from video transcripts, and uses the retrieved material to generate grounded answers. The system additionally links the response to the corresponding timestamps, allowing the user to move directly from a question to the parts of the source videos that are relevant to that specific information need. The approach is studied on long-form educational videos from the Digital Humanism initiative at TU Wien. A manually constructed gold-standard dataset is used to evaluate how effectively the retrieval component identifies relevant video segments, while the generated responses are assessed in relation to the retrieved source material.