Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aspect ratio issues investigations #258

Open
courville opened this issue Dec 10, 2019 · 34 comments
Open

aspect ratio issues investigations #258

courville opened this issue Dec 10, 2019 · 34 comments
Assignees
Labels
Important Bug severity
Milestone

Comments

@courville
Copy link
Contributor

courville commented Dec 10, 2019

On Philips TV (mtk) it has been reported that AR is wrong https://www.reddit.com/r/NovaVideoPlayer/comments/dz4wh5/reqissue_android_tv_9_pie_aspect_ratio_totally/
On amlogic when selecting full screen AR is wrong with 21:9 and 4:3 videos that are stretched instead of applying a zoom.
It appears when surface has been scaled and exceeds screen dimension i.e. codec does a scale instead of crop.
@phhusson hinted towards https://github.com/xbmc/xbmc/blob/master/tools/android/packaging/xbmc/src/XBMCVideoView.java.in#L117
@phhusson also suggested that a echo 6 > /sys/class/video/screen_mode could help on amlogic cf. https://gist.github.com/phhusson/b36755226e06f313caca41ec573b8e93 i.e.:

#define  SCREEN_MODE_NORMAL           0
#define  SCREEN_MODE_FULL_STRETCH     1
#define  SCREEN_MODE_4_3              2
#define  SCREEN_MODE_16_9             3
#define  SCREEN_MODE_NONLINEAR        4
#define  SCREEN_MODE_NORMAL_NOSCALEUP 5
#define  SCREEN_MODE_CROP_FULL        6
#define  SCREEN_MODE_CROP             7
@courville
Copy link
Contributor Author

@courville
Copy link
Contributor Author

In Ndk domain, AMEDIAFORMAT_KEY_DISPLAY_CROP = "crop" exists but has been introduced in 28 (frameworks/av/media/ndk/NdkMediaFormat.cpp and android-ndk/sysroot/usr/include/media/NdkMediaFormat.h).

@courville
Copy link
Contributor Author

Relevant part in avos is native/avos/external/android/libsfdec/sfdec_ndkmediacodec.cpp:sfdec_init

@phhusson
Copy link
Contributor

Java's setVideoScalingMode calls:

        native_window_set_scaling_mode(mSurfaceTextureClient.get(), mode);
        msg->setInt32("android._video-scaling", mode);
        (void)mCodec->setParameters(msg);

This means that the information is transmitted to both hwcomposer and codec.
We can't safely access native_window_set_scaling_mode but we can send a parameter to the codec.

So we can try to use AMediaCodec_setParameters to set "android._video-scaling"'s value to the mode we want

@courville
Copy link
Contributor Author

courville commented Dec 10, 2019

Regarding more strange AR issues as reported here #257 one need to check that codec report is correct (should be enforced by CTS) and this can be done monitoring INFO_FORMAT_CHANGED from avos:

diff --git a/external/android/libsfdec/sfdec_ndkmediacodec.cpp b/external/android/libsfdec/sfdec_ndkmediacodec.cpp
index 57f6c7e..fe6d488 100644
--- a/external/android/libsfdec/sfdec_ndkmediacodec.cpp
+++ b/external/android/libsfdec/sfdec_ndkmediacodec.cpp
@@ -274,7 +274,7 @@ static int sfdec_read(sfdec_priv_t *sfdec, int64_t seek, sfdec_read_out_t *read_
             read_out->size.width = sfdec->width;
             read_out->size.height = sfdec->height;
             read_out->size.interlaced = 0;
-            DBG LOG("INFO_FORMAT_CHANGED: %dx%d", sfdec->width, sfdec->height);
+            LOG("INFO_FORMAT_CHANGED: %dx%d", sfdec->width, sfdec->height);
             return 0;
         } else if (index == AMEDIACODEC_INFO_OUTPUT_BUFFERS_CHANGED) {
             DBG LOG("INFO_OUTPUT_BUFFERS_CHANGED");

@courville
Copy link
Contributor Author

On another track, Philips TVs (MTK based) are showing also AR issues after upgrading to Android 9 but this is not happening on Sony TVs (MTK based too).
Check Kodi references https://forum.kodi.tv/showthread.php?tid=341491 and notice that somepeople are recommending to update their Philips TV firmware since Philips is supposed to have fixed the issue in an update. In the thread they prompt the user to check https://toengel.net/philipsblog/firmware-download/

@courville
Copy link
Contributor Author

/sys/class/video/screen_mode is not accessible on miproj or mibox.

@courville
Copy link
Contributor Author

Cf. https://gist.github.com/courville/f64cb4cd6e6f644c52042de2ca5f1b6a
Asking the codec to do the scale to fit with crop seems to make amlogic mibox s working.
Need to check on miproj.
Issue with the above approach is that it requires API26 because AMediaCodec_setParameters is not available on former versions.

@phhusson
Copy link
Contributor

phhusson commented Dec 12, 2019 via email

@courville
Copy link
Contributor Author

courville commented Dec 12, 2019

We need someone to test on Philips/MTK and the weird rk3328 aspect-ratio issue too Le jeu. 12 déc.

Already done: https://www.reddit.com/r/NovaVideoPlayer/comments/dz4wh5/reqissue_android_tv_9_pie_aspect_ratio_totally/ test results soon for Philips.
Regarding RK apk shared as well here: #257

@courville
Copy link
Contributor Author

Feedback from testers: no effect on miproj, philips tv, rk box.
New approach to be tested: AR would work if we trust container resolution and not the codec format output change. This could result in new bugs if container information is wrong.
But this could fix firestick 4k ko, philips tv with latest firmware KO and RK KO.

@courville
Copy link
Contributor Author

New approach: https://gist.github.com/courville/2fe4056f2846319f69368f80fc63c2be and test apk shared.

@courville
Copy link
Contributor Author

OK I found a firestick4k and aspect ratio is corrected with the proposed fix regarding the video provided here #259
What happens is that the codec reports 1920x1088 resolution and real video as reported is 1280x544. Applying what proposes the codec results in stretched fullscreen video.
It is worth noting that normally codec should report correct resolution and this is enforced on AndroidTV by proper CTS tests. But hey... amazon is almighty.

12-13 23:35:04.457   275  1056 D MtkOmxVdecEx: [0xa818d000] MtkOmxVdec::FreeBuffer all output buffers have been freed!!! signal mOutPortFreeDoneSem(1)
12-13 23:35:04.458   275  1056 D MtkOmxMVAMgr: [0xa851b5c8] Remove ion MVA Map, remain buffer count=0
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] MtkOmxVdec::SendCommand cmd=OMX_CommandPortEnable
12-13 23:35:04.459   275  6099 D MtkOmxVdecEx: [0xa818d000] # Got general command (OMX_CommandPortEnable)
12-13 23:35:04.459   275  6099 D MtkOmxVdecEx: [0xa818d000] Wait on mOutPortAllocDoneSem(0)
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] [MtkOmxVdec] Get decoder property, VDEC_DRV_QUERY_TYPE_UFO_SUPPORT: 1, 4489
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] mOutputPortDef eColorFormat(7f000001), eColorFormat(7f000001), meDecodeType(0), mForceOutputBufferCount(0), mIsUsingNativeBuffers(1)
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] [MtkOmxVdec] Get decoder property, VDEC_DRV_QUERY_TYPE_UFO_SUPPORT: 1, 4489
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] 32x32 Aligned! mOutputPortDef.nBufferSize(3133440), nStride(1920), nSliceHeight(1088) nBufferCountActual(13)
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] [MtkOmxVdec] Get decoder property, VDEC_DRV_QUERY_TYPE_UFO_SUPPORT: 1, 4489
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] mOutputPortDef eColorFormat(7f000001), eColorFormat(7f000001), meDecodeType(0), mForceOutputBufferCount(0), mIsUsingNativeBuffers(1)
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] [MtkOmxVdec] Get decoder property, VDEC_DRV_QUERY_TYPE_UFO_SUPPORT: 1, 4489
12-13 23:35:04.459   275  1056 D MtkOmxVdecEx: [0xa818d000] 32x32 Aligned! mOutputPortDef.nBufferSize(3133440), nStride(1920), nSliceHeight(1088) nBufferCountActual(13)
12-13 23:35:04.460  5974  6097 D SurfaceUtils: set up nativeWindow 0x8c26c808 for 1920x1088, color 0x7f000001, rotation 0, usage 0x2933
12-13 23:35:04.460   275  1056 E OMXNodeInstance: setParameter(1130047:MTK.DECODER.AVC, ParamPortDefinition(0x2000001)) ERROR: BadParameter(0x80001005)
12-13 23:35:04.460  5974  6097 W ACodec  : [OMX.MTK.VIDEO.DECODER.AVC] setting nBufferCountActual to 14 failed: -2147483648
12-13 23:35:04.461   275  1056 D MtkOmxVdecEx: [0xa818d000] UseBuffer:: clear mFrameBuf
12-13 23:35:04.461   275  1056 D MtkOmxMVAMgr: [0xa851b5c8] [WARNING] Can not find MVA from VA: 0xa82fc540
12-13 23:35:04.463   275  1056 D MtkOmxVdecEx: [0xa818d000] signal mOutPortAllocDoneSem (1)
12-13 23:35:04.463   275  1056 D MtkOmxVdecEx: [0xa818d000] output port populated
12-13 23:35:04.464   275  6099 D MtkOmxVdecEx: [0xa818d000] mPortReconfigInProgress as FALSE
12-13 23:35:04.495  5974  6096 I MediaCodecLogger: updateFormatChanged width = 1280 height = 544
12-13 23:35:04.499  5974  6096 I MediaCodecLogger: updatePTSTime [HW.video.avc] First PTS after Flush or reset = 14556000
12-13 23:35:04.499  5974  6096 I MediaCodecLogger: [ASAP] HW.video.avc Got First Frame Ready 434448
12-13 23:35:04.501  5974  6096 I MediaCodecLogger: [ASAP] HW.video.avc Got First Frame Render 434450
12-13 23:35:04.501  5974  6096 I MediaCodecLogger: App uses Timestamps for AVSync
12-13 23:35:04.502  5974  5995 D avos_player: init_renderer: width NOT changed: 1280 -> 1920
12-13 23:35:04.502  5974  5995 D avos_player: init_renderer: height NOT changed: 544 -> 1088
12-13 23:35:04.502  5974  5995 D avos_player: sfdec_read: INFO_FORMAT_CHANGED: 1280x544

@courville
Copy link
Contributor Author

Some grep command to parse the logcat: grep -i '(avos\|gralloc\|surface\|codec\|\omx\|mali)'

@courville
Copy link
Contributor Author

On Philips TV nothing strange is seen:

D avos_player: VIDEO:  [H264] [H.264] 1920x784   0fps  1964kbit/s  dec [(none)]
D avos_player: sfdec_read: INFO_FORMAT_CHANGED: 1920x784
D avos_player: sfdec_read: INFO_FORMAT_CHANGED: 1920x784
D SurfaceUtils: set up nativeWindow 0xe0019008 for 1920x784, color 0x7f000101, rotation 0, usage 0x30002900
D SurfaceController: updateSurface: lcd plugged dw=1920, dh=1080
D SurfaceController: updateSurface: lcd plugged dw=1920, dh=1080
D SurfaceController: updateSurface: vw=1920, vh=784
D SurfaceController: getVideoFormat: return mVideoFormat
D SurfaceController: updateSurface: sar=2.4489795918367347, ar=2.4489795918367347, dar=1.7777777777777777
D SurfaceController: updateSurface: VideoFormat.ORIGINAL dar<ar dh=784
D SurfaceController: updateSurface: setFixedSize(1920,784)
D SurfaceController: updateSurface: setLayoutParams(1920,784)
D SurfaceController: updateSurface: 1920x784 -> 1920x784 / formatCrop: 1.0x1.0 / mEffectMode: 0

@courville
Copy link
Contributor Author

courville commented Dec 14, 2019

For Philips TV user reports that when plex is used or casted from PC or native Android TV player AR is ok.
When using nova it is stretched with a height smaller than normal.
Capture d’écran 2019-12-14 à 10 14 38
This happens with all videos 1920x800 21:9.

@courville
Copy link
Contributor Author

Full philips TV logs.
novaAspectratio.txt.zip

@courville
Copy link
Contributor Author

courville added a commit to nova-video-player/aos-Video that referenced this issue Dec 20, 2019
…the display.

One can think that this is an horridious option that should never have been introduced because it breaks the golden rule of preserving the aspect ratio.
However this is the only way to get a correct aspect ratio on recent Android 9+ Philips Android TV.
Philips TV seems to do something fundamentally wrong using MediaCodec which is to add black horizontal bars resulting in compressed vertically video image.
This stretched mode should restore a good aspect ratio for 1920x800 21:9 movies.

See nova-video-player/aos-AVP#258
@courville
Copy link
Contributor Author

I will add a stretched option in the next release since it solves the wrong aspect ratio on Philips 65PUS7304 (2019 model), Philips has really made a bad implementation.
I will explain, nova for a 1920x800 video displays it in a 1920x800 surface to preserve the aspect ratio.
Sony does add top and bottom black bars to this video in the same surface so that it looks compressed vertically: it ends up with this video height of 592=800*800/1080.
When asking the Philips TV to stretch the video to a 1920x1080 with what one could expect a really bad aspect ratio, you end up with a 1920x800 video image and a 1920x1080 surface including the black bars.
I am really against messing up with the aspect ratio and thus proposing the stretched option (that some are requesting) but hey if this is the only way to get the Sony TV to behave, I will add this one.
Bad implementations leads to bad compromises.
Sony if you read this: boohoo!

@primozcerar
Copy link

I figured out what the issue was that was plaguing many apps on newer Philips TVs.
It should solve the problem without having to set stretch mode on every video with AR greater than the display.

Philips Android 9 MediaCodec implementation requires that on MediaFormat used to configure the codec you set not only the KEY_WIDTH and KEY_HEIGHT but also the KEY_MAX_WIDTH and KEY_MAX_HEIGHT parameters.

This has already been tested on a Kodi build and works as expected.

@courville
Copy link
Contributor Author

@primozcerar thanks for the tip. According to other references it could also fix the issue on amlogic not getting fullscreen for 4:3 video. I will investigate and it might help me remove the stretch mode.

@courville
Copy link
Contributor Author

See xbmc/xbmc#17353

courville added a commit to nova-video-player/aos-avos that referenced this issue Feb 12, 2020
Possibly fixes the inability to tune aspect ratio on latest amlogic Android 9 versions.
Thanks @primozcerar for the tip.
See nova-video-player/aos-AVP#258
@courville
Copy link
Contributor Author

courville commented Jun 12, 2020

I noticed in xbmc that the change for KEY_MAX_WIDTH has been reverted. Are you aware of any reason for this or another way to fix the AR for philips TVs?

See xbmc/xbmc#17557 referencing xbmc/xbmc#17534

And the severe issues and regressions on sony and philips are described there xbmc/xbmc#17559

courville added a commit to nova-video-player/aos-avos that referenced this issue Jun 12, 2020
This reverts commit 72156d4.

Apparently it causes many regressions on sony and philips tvs...
See nova-video-player/aos-AVP#258
courville added a commit to nova-video-player/aos-Video that referenced this issue Jun 12, 2020
… to the size of the display.""

This reverts commit a3edbc5.

afterall this is needed see nova-video-player/aos-AVP#258
courville added a commit to nova-video-player/aos-Video that referenced this issue Jun 12, 2020
@primozcerar
Copy link

primozcerar commented Jun 13, 2020 via email

@courville
Copy link
Contributor Author

Closing due to inactivity and possibly prior solving. Please reopen an issue if needed.

@andrei-tatar
Copy link

@courville I have an issue on Google TV. When I try to set the Format to Fullscreen, it shows as stretched (H264 video file from SMB share).

  • Original and Optimized show the same way (video contained within the TV view)
  • Fullscreen and Stretched show the same way (video stretched on the TV)

If I switch to the software decoder, it all works ok. Should I stay on the software decoder?

@phhusson
Copy link
Contributor

@andrei-tatar What's your device?

@andrei-tatar
Copy link

@phhusson I'm using a Google TV or Google Chromecast TV. Not sure exactly what it's called. It's the one with a small remote.
image

@phhusson
Copy link
Contributor

k so FYI the official name of this device is "Chromecast with Google TV" (I prefer calling it "sabrina" since the name is a non-sense).

@courville will test this on his own

@courville
Copy link
Contributor Author

@andrei-tatar ok I confirm the issue and it seems to be a regression. Will have a look at it.

@courville courville reopened this Jan 22, 2022
@courville courville self-assigned this Jan 22, 2022
@courville courville added this to the nova v6 milestone Jan 22, 2022
@courville courville added the Important Bug severity label Jan 22, 2022
@courville
Copy link
Contributor Author

@andrei-tatar quick question for you: can you please provide the video resolution of your sample and check that the video itself does not contain black top and bottom bars that would prevent the video scaling since the video aspect ratio would match the one of the screen of the TV? In this case this would not be a bug...

@andrei-tatar
Copy link

Sure, I can check in a few hours. But I already confirmed the software decoder works as expected. It seems to be an issue with the hardware decoder.

@courville
Copy link
Contributor Author

courville commented Jan 23, 2022

@andrei-tatar you are right. Issue not reproduced on nvidia shield and emulator but present on all amlogic based hardware (miprod + googletv=sabrina) and only present when hwdec enabled. @phhusson do you reproduce on freePlayerPop?

EDIT: works on pop. Thus probably amlogic SDK bug.
Subliminal message to Google: this should really be verified by dedicated CTS test for devices certification (sabrina does not comply).

@Pentaphon
Copy link

Pentaphon commented Jan 30, 2022

present on all amlogic based hardware

Indeed, I have a brand new Onn 4K box which has an Amlogic S905Y2 (it's an amazing value since the major bugs got fixed, btw) and I see the exact same thing as @andrei-tatar on this device when playing https://www.youtube.com/watch?v=UmBAhFSROLo which I downloaded as 1920x800 using yt-dlp. When I switch to software decoding, fullscreen and stretched look different, while original and optimized look the same.

This is interesting since the Onn 4K is just a re-brand of the ADT-3 developer box that came out in 2020. so along with the Sabrina device, that's 2 devices that Google devs can be contacted about to see if they can help with getting Amlogic to provide a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Important Bug severity
Projects
None yet
Development

No branches or pull requests

5 participants