[10.7] Transcoding to HEVC on unsupported iPad when fMP4-HLS is enabled

[TheNamelessWonderer]

Describe the bug

Jellyfin is transcoding to HEVC if fMP4-HLS is enabled on all iPads regardless of if it can play HEVC or not. This causes my iPad Air 2 to not be able to play the file if HEVC encoding and fMP4-HLS are enabled.

System (please complete the following information):

• OS: macOS
• Virtualization: Native
• Clients: Jellyfin iOS app
• Browser: Based on Safari
• Jellyfin Version: 10.7-rc2
• Playback: Transcode
• Installed Plugins: Kodi Sync Queue, Playback Reporting, TV Maze,
• Reverse Proxy: Nginx
• Base URL: None
• Networking: Host
• Storage: NFS

To Reproduce

1. Enable HEVC encoding and fMP4-HLS on an unsupported HEVC iPad.
2. Play an unsupported video such as H264 high to get it to transcode.
3. See error (play icon with line through it).

Expected behavior

Jellyfin should detect the device doesn’t support HEVC and transcode to H264 instead.

Additional Information
Disabling HEVC encoding on my iPad Air 2 allows fMP4-HLS to work.

[TheNamelessWonderer]

Just tested 10.7-RC3 and it is still an issue on this build.

[nyanmisaka]

Safari is always a lier on video support. It reports that it supports HVC1 on your iPad Air 2 but cannot play actually.
Now we don't have a proper method to determine the iPad model through browser UA.
This problem should be solved in the native client in the future.

[TheNamelessWonderer]

Thanks should I leave this open then or close it?

[TheNamelessWonderer]

Oddly enough 480p HEVC content plays fine but not 1080p HEVC. Also if the content is HEVC then it tries to direct play it and fails.

[knackebrot]

Can you paste here what this page displays on your iPad?
https://shaka-player-demo.appspot.com/support.html

[TheNamelessWonderer]

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15
v3.0.10

{
  "manifest": {
    "application/dash+xml": true,
    "video/vnd.mpeg.dash.mpd": true,
    "application/x-mpegurl": true,
    "application/vnd.apple.mpegurl": true,
    "application/x-offline-manifest": true,
    "mpd": true,
    "m3u8": true,
    "application/vnd.ms-sstr+xml": false,
    "ism": false
  },
  "media": {
    "video/mp4; codecs=\"avc1.42E01E\"": true,
    "video/mp4": true,
    "video/mp4; codecs=\"avc3.42E01E\"": true,
    "video/mp4; codecs=\"hev1.1.6.L93.90\"": true,
    "video/mp4; codecs=\"hvc1.1.6.L93.90\"": true,
    "video/mp4; codecs=\"hev1.2.4.L153.B0\"; eotf=\"smpte2084\"": true,
    "video/mp4; codecs=\"hvc1.2.4.L153.B0\"; eotf=\"smpte2084\"": true,
    "video/mp4; codecs=\"vp9\"": false,
    "video/mp4; codecs=\"vp09.00.10.08\"": false,
    "video/mp4; codecs=\"av01.0.01M.08\"": false,
    "audio/mp4; codecs=\"mp4a.40.2\"": true,
    "audio/mp4": true,
    "audio/mp4; codecs=\"ac-3\"": true,
    "audio/mp4; codecs=\"ec-3\"": true,
    "audio/mp4; codecs=\"opus\"": true,
    "audio/mp4; codecs=\"flac\"": true,
    "video/webm; codecs=\"vp8\"": false,
    "video/webm": false,
    "video/webm; codecs=\"vp9\"": false,
    "video/webm; codecs=\"vp09.00.10.08\"": false,
    "audio/webm; codecs=\"vorbis\"": false,
    "audio/webm": false,
    "audio/webm; codecs=\"opus\"": false,
    "video/mp2t; codecs=\"avc1.42E01E\"": false,
    "video/mp2t": false,
    "video/mp2t; codecs=\"avc3.42E01E\"": false,
    "video/mp2t; codecs=\"hvc1.1.6.L93.90\"": false,
    "video/mp2t; codecs=\"mp4a.40.2\"": false,
    "video/mp2t; codecs=\"ac-3\"": false,
    "video/mp2t; codecs=\"ec-3\"": false,
    "text/vtt": true,
    "application/mp4; codecs=\"wvtt\"": true,
    "application/mp4": true,
    "application/ttml+xml": true,
    "application/mp4; codecs=\"stpp\"": true
  },
  "drm": {
    "org.w3.clearkey": null,
    "com.widevine.alpha": null,
    "com.microsoft.playready": null,
    "com.adobe.primetime": null,
    "com.apple.fps.3_0": {
      "persistentState": false
    },
    "com.apple.fps.2_0": {
      "persistentState": false
    },
    "com.apple.fps.1_0": {
      "persistentState": false
    },
    "com.apple.fps": {
      "persistentState": false
    }
  },
  "offline": true
}

[knackebrot]

I did some digging and it seems like HEVC is hardware-accelerated only since Apple A9. Older devices that run iOS 11+ do technically support it, but aren't fast enough to decode it real-time at higher resolutions. So, we need a way to detect the SoC of the device. It's been a while since anything useful was in the user agent and since iOS 12.2 the GPU in WebGL only shows up as "Apple GPU". Luckily, people have figured it out. We can pull either this function or this library into jellyfin-web and add a condition not to enable HEVC on /A[7-8]X?/.
@nyanmisaka What do you think?

[TheNamelessWonderer]

The second one doesn’t seem to work. I tried the demo and it tells me:

{
  "device": "apple ipad pro (11-inch)",
  "fps": 116,
  "gpu": "apple a12x gpu",
  "isMobile": true,
  "tier": 3,
  "type": "BENCHMARK"
}

Which is completely wrong as I’m on an iPad Air 2 which an Apple A8X GPU. Also VLC can play 1080p HEVC just fine so it seems a limitation from Apple or Safari rather than the hardware being to weak. Although VLC is most likely using software decoding to play the video.

[knackebrot]

what can you see on this page? does it also say a12x, or Unknown?

https://codepen.io/knackebrot/pen/GRrpEyO

[TheNamelessWonderer]

It says Apple A8X GPU which is correct.

[cvium]

Since this is a client capability issue I'm moving it to the iOS repo.