Serving Files

[gr8rehanna]

Better approach to serve files
Well thanks for the update..,.

Its now better and works fine but i noticed 1 major flaw.

Problem
Now as i used this as an S3 alternative on mastodon.
Its a small local development server.
When we upload files they are now getting uploaded now even we can upload very larger files.
When a user opens the timeline and suppose timeline will show 100 images.
So then S3 has to fetch 100 images in no times.
Our bot does not handle file as streaming or serving directly from telegram.
Instead it uses tempfile to download the files on server then send to user/mastodon.

Actual Problem.
Here is the thing, if we got 10K users and at same time 5K or even 1K users will open timeline.
So each users timeline showing 100 images = 10K or 50K images to served to users.
So our s3 server has to download 50K files from telegram and store on server in tempfile and serve.

Worse thing.
Telegram rate limit is imposed on bots for too many concurrent request

Solution
If possible can we directly stream files from telegram instead of downloading files on server eveerytime.

Additional context
Well its my personal experience maybe i am wrong but i
But as per below code when we call get_file it downloads file and sends to users.

As i guess bots can not stream files directly from telegram.
How about users? Session string of premium account can also improve uploading speed.
And most important feature is serving files to millions without download in server and also maintaining speed and performance.
We can use multi session strings/bots to handle uploads .
So which ever bot is available will handle the uploading (I have implemented this)

At last serving files remains an issue.
Sorry for creating new issues instead of opening existing one.
Also got a job so not often on local projects.
And its a hobby project if you are busy or not interested just ignore.

Thanks

[luiscib3r]

Thank you for your valuable feedback! While I'm not actively developing it, I'm really glad to see someone else using this project. I recently made an update that should address the file serving concerns.

A pro tip: you can use a personal Telegram account instead of a bot, which doesn't have the same rate limitations. I personally use my Telegram account when working with this project.
To set this up:

Run make setup_account_storage
Enter your Telegram phone number (instead of a bot token)
Input the authentication code you'll receive
Use the generated session string to run the service

This will create a persistent session, so you won't need to authenticate again in future runs.
Let me know if you have any questions.

[luiscib3r]

Just want to add an important note about scalability:
For high-traffic projects, Telegram's limitations might eventually become a bottleneck that can't be circumvented. In such cases, I'd recommend using actual S3 or another storage solution specifically designed for high-concurrency file serving.
This project works great for personal/small deployments, but it's important to acknowledge its limitations for production environments with high demand.

[gr8rehanna]

Thanks. Well i do use R2 from cloudflare that works fine for us.
I was search google and found your repo it good alternative for low traffic but for high volume/request it does not work as S3 / R2.

Thanks.
It helped me alot.