glowing-telegram

A tool for managing stream recordings.

This is a tool for managing stream recordings, ingesting them into a database, providing a web interface for searching, analyzing, and passing them to a video processing pipeline.

I'm developing this tool live on Twitch. Why not come check it out sometime? I'm developing this tool to practice my Rust, as it's a bit rusty, and to automate some of the video processing tasks that I do manually by spending way more time programming than I would have spent doing the tasks manually.

Features

1. Track locally recorded clips from a stream
2. Generate a set of "episodes" from the stream based on when the speaker is speaking
3. Episode transcription
4. Review interface for the transcriptions
5. Automatic summaries of the episode via text summarization provided by GPT-4
6. Capture chat messages with author and timestamp metadata from the stream
7. Flag areas of the video that are interesting
8. Generate a set of "highlights" from the stream based on the flagged areas
9. Generate chapter markers for the episode based on the flagged areas
10. Archive the stream videos to a cloud storage provider
11. Generate an OTIO file for for the stream video for use in a video editing tool

Architecture

The tool is broken down into several repositories:

1. glowing-telegram - The backend for the tool (this repository)
2. glowing-telegram-frontend - The frontend for the tool
3. glowing-telegram-video-editor - A React component for reviewing stream videos and generating episodes

This repository contains these directories:

1. ai_chat_lambda - A lambda function for that wraps the OpenAI API for chat completion
2. audio_transcriber - An executable for transcribing audio files with OpenAI's Whisper Python library
3. cdk - An AWS CDK project for deploying the backend to AWS
4. crud_api - A lambda function for managing the CRUD operations for the DynamoDB tables
5. docs - Documentation for the project
6. gt_ffmpeg - A library for interacting with FFmpeg
7. scripts - Scripts for managing the project, migrating data from the old database, and other tasks
8. summarize_transcription - A lambda function for summarizing the transcriptions of the episodes using OpenAI's API
9. twitch_bot - An unfinished Twitch bot for interacting with Twitch chat and storing messages in the database, implemented in Elixir
10. twitch_lambda - A lambda function for ingesting authenticating with Twitch
11. types - Shared types for the project generated from the JSON schemas in the docs directory, also used by the frontend
12. video_ingestor - An executable for analyzing video files for silence detection, storing the speech audio track and keyframes of the video to S3, and storing the metadata in the database

Development

I should probably write some instructions here, but I haven't yet. If you're interested in contributing, please reach out to me on Twitch or Twitter.