Enhancement: Multimodal Content Support via SRT File Generation

[mysticaltech]

• Current Scope: "aifs" excels in local semantic search for text-based content.
• Proposed Enhancement: Extend capabilities to include audio and video content.
• Implementation:
  • Generate .srt Files: Utilize tools like Whisper (runs great locally on most computers) for transcribing audio and video into subtitle files.
  • Search Integration: Incorporate these .srt files into the search functionality.
• Outcome:
  • Search Utility: Enable searches to return specific segments from multimedia content.
  • Timestamps Feature: Provide timestamps alongside search results for precise referencing.
• Impact: This enhancement would transform "aifs" into a comprehensive tool for searching across various content formats, greatly expanding its usability.

[mysticaltech]

A good ressource to assist with this is: https://github.com/sindresorhus/awesome-whisper

[KillianLucas]

oh I love this @mysticaltech. will make a ROADMAP.md and put this there. thanks!

[mysticaltech]

@KillianLucas Just stumbled on that high level whisper library, it seems ideal for the job. I checked in the code, the real API is more extensive than what is presented in the readme. https://github.com/kadirnar/whisper-plus

[mysticaltech]

This can definitely help a lot with this. https://github.com/video-db/Director