Implement media file redirect

[dnnspaul]

Hey,

a friend that uses beeper.com noticed that he can't send me pictures to my dendrite instance or better said the picture doesn't come up in my chat. A look into the log showed:

time="2025-01-16T13:19:51.528563500Z" level=error msg="r.fetchRemoteFileAndStoreMetadata: failed to fetch remote file" MediaID=username_ULTRASECRETMEDIAID Origin=local.beeper.com error="Location header is not yet supported" req.id=dPbahDej54O1 req.method=GET req.path=/_matrix/media/r0/download/local.beeper.com/username_ULTRASECRETMEDIAID

Key Changes

1. Implemented redirect handling in parseMultipartResponse when encountering a Location header
2. Added proper response body management to prevent "read on closed body" errors
3. Added security measures:
   • Maximum of 10 redirects to prevent infinite loops
   • Support for relative URLs in redirects

Technical Details

• The implementation avoids using the federation client for redirects since they often point to external storage services
• Response bodies are carefully managed to prevent premature closing while still ensuring cleanup
• Uses io.NopCloser to properly wrap response bodies that will be passed back to the caller

Testing Notes

This can be tested by attempting to download media that results in a multipart response containing a Location header, particularly when media is stored on external services like Cloudflare R2. Booper for example.

• I have added Go unit tests or Complement integration tests for this PR or I have justified why this PR doesn't need tests
• Pull request includes a sign off below or I have already signed off privately

Signed-off-by: Dennis Paul <[email protected]>

[S7evinK]

We should definitely not use the default http client here. We just merged an update (e9cc37a) to fix a SSRF vuln.

So yea, prefer using the federation client here.

[dnnspaul]

Can totally understand that. I never worked with the federation client before, but do I understand correctly that it's mostly used for getting in contact with other matrix servers?

Because I had a look into the federation client interface and didn't find a function for a simple GET request. The best candidates I found were https://github.com/matrix-org/gomatrixserverlib/blob/main/fclient/federationclient.go#L67 and https://github.com/matrix-org/gomatrixserverlib/blob/main/fclient/federationclient.go#L22 but both implementations had a URL manipulator logic that would make it impossible with current state to access a non-matrix server with it, like I mentioned in the initial message for example an s3 URL like in beeper.com's case.

EDIT: I found the DoHTTPRequest() function in the *fclient.Client, but it always makes GET requests to the 8448 port and I can't find a possibility to overwrite it. I'm passing it an URL without any port suggestion.

time="2025-01-16T21:58:46.889093655Z" level=error msg="r.fetchRemoteFileAndStoreMetadata: failed to fetch remote file" MediaID=xxx_xxx Origin=local.beeper.com error="failed to send http request to redirect: Get \"https://s3.eu-central-1.wasabisys.com/hungryserv-media-eu-central-1.beeper.com/xxx/media/xxx?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=xxx%2F20250116%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20250116T215836Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&response-content-disposition=attachment&X-Amz-Signature=xxx\": dial tcp 130.117.252.101:8448: i/o timeout" req.id=c6fSn19V3pwC req.method=OPTIONS req.path=/_matrix/media/v3/download/local.beeper.com/xxx_xxx

[CLAassistant]

All committers have signed the CLA.