diff --git a/RELEASENOTES.md b/RELEASENOTES.md index 522d2f457eb..f5353a24e1c 100644 --- a/RELEASENOTES.md +++ b/RELEASENOTES.md @@ -1,5 +1,277 @@ # Release notes +### 2.13.0 (2021-02-04) + +* Core library: + * Verify correct thread usage in `SimpleExoPlayer` by default. Opt-out is + still possible until the next major release using + `setThrowsWhenUsingWrongThread(false)` + ([#4463](https://github.com/google/ExoPlayer/issues/4463)). + * Add `Player.getCurrentStaticMetadata` and + `EventListener.onStaticMetadataChanged` to expose static metadata + belonging to the currently playing stream + ([#7266](https://github.com/google/ExoPlayer/issues/7266)). + * Add `PlayerMessage.setLooper` and deprecate `PlayerMessage.setHandler`. + * Add option to `MergingMediaSource` to clip the durations of all sources + to have the same length + ([#8422](https://github.com/google/ExoPlayer/issues/8422)). + * Remove `Player.setVideoDecoderOutputBufferRenderer` from Player API. Use + `setVideoSurfaceView` and `clearVideoSurfaceView` instead. + * Default `SingleSampleMediaSource.treatLoadErrorsAsEndOfStream` to `true` + so that errors loading external subtitle files do not cause playback + to fail ([#8430](https://github.com/google/ExoPlayer/issues/8430)). A + warning will be logged by `SingleSampleMediaPeriod` whenever a load + error is treated as though the end of the stream has been reached. + * Time out on release to prevent ANRs if an underlying platform call is + stuck ([#4352](https://github.com/google/ExoPlayer/issues/4352)). + * Time out when detaching a surface to prevent ANRs if the underlying + platform call is stuck + ([#5887](https://github.com/google/ExoPlayer/issues/5887)). + * Fix bug where `AnalyticsListener` callbacks could arrive in the wrong + order ([#8048](https://github.com/google/ExoPlayer/issues/8048)). +* Media transformation: + * Add a new `transformer` module for converting media streams. The + initially available transformations are changing the container format, + removing tracks, and slow motion flattening. +* Low latency live streaming: + * Support low-latency DASH (also known as ULL-CMAF) and Apple's + low-latency HLS extension. + * Add `LiveConfiguration` to `MediaItem` to define live offset and + playback speed adjustment parameters. The same parameters can be set on + `DefaultMediaSourceFactory` to apply for all `MediaItems`. + * Add `LivePlaybackSpeedControl` to control playback speed adjustments + during live playbacks. Such adjustments allow the player to stay close + to the live offset. `DefaultLivePlaybackSpeedControl` is provided as a + default implementation. + * Add `targetLiveOffsetUs` parameter to `LoadControl.shouldStartPlayback`. +* Extractors: + * Populate codecs string for H.264/AVC in MP4, Matroska and FLV streams to + allow decoder capability checks based on codec profile and level + ([#8393](https://github.com/google/ExoPlayer/issues/8393)). + * Populate codecs string for H.265/HEVC in MP4, Matroska and MPEG-TS + streams to allow decoder capability checks based on codec profile and + level ([#8393](https://github.com/google/ExoPlayer/issues/8393)). + * Add support for playing JPEG motion photos + ([#5405](https://github.com/google/ExoPlayer/issues/5405)). + * Handle sample size mismatches between raw audio `stsd` information and + `stsz` fixed sample size in MP4 extractors. + * Fix Vorbis private codec data parsing in the Matroska extractor + ([#8496](https://github.com/google/ExoPlayer/issues/8496)). +* Track selection: + * Move `Player.getTrackSelector` to the `ExoPlayer` interface. + * Move the mutable parts of `TrackSelection` into an `ExoTrackSelection` + subclass. + * Allow parallel adaptation of video and audio + ([#5111](https://github.com/google/ExoPlayer/issues/5111)). + * Simplify enabling tunneling with `DefaultTrackSelector`. + `ParametersBuilder.setTunnelingAudioSessionId` has been replaced with + `ParametersBuilder.setTunnelingEnabled`. The player's audio session ID + will be used, and so a tunneling specific ID is no longer needed. + * Add additional configuration parameters to `DefaultTrackSelector`. + `DefaultTrackSelector.ParametersBuilder` now includes: + * `setPreferredVideoMimeType`, `setPreferredVideoMimeTypes`, + `setPreferredAudioMimeType` and `setPreferredAudioMimeTypes` for + specifying preferred video and audio MIME type(s) + ([#8320](https://github.com/google/ExoPlayer/issues/8320)). + * `setPreferredAudioLanguages` and `setPreferredTextLanguages` for + specifying multiple preferred audio and text languages. + * `setPreferredAudioRoleFlags` for specifying preferred audio role + flags. + * Forward `Timeline` and `MediaPeriodId` to `TrackSelection.Factory`. +* DASH: + * Support low-latency DASH playback (`availabilityTimeOffset` and + `ServiceDescription` tags) + ([#4904](https://github.com/google/ExoPlayer/issues/4904)). + * Improve logic for determining whether to refresh the manifest when a + chunk load error occurs in a live streams that contains EMSG data + ([#8408](https://github.com/google/ExoPlayer/issues/8408)). +* HLS: + * Support playlist delta updates, blocking playlist reloads and rendition + reports. + * Support low-latency HLS playback (`EXT-X-PART` and preload hints) + ([#5011](https://github.com/google/ExoPlayer/issues/5011)). +* UI: + * Improve `StyledPlayerControlView` button animations. + * Miscellaneous fixes for `StyledPlayerControlView` in minimal mode. +* DRM: + * Fix playback failure when switching from PlayReady protected content to + Widevine or Clearkey protected content in a playlist. + * Add `ExoMediaDrm.KeyRequest.getRequestType` + ([#7847](https://github.com/google/ExoPlayer/issues/7847)). + * Drop key and provision responses if `DefaultDrmSession` is released + while waiting for the response. This prevents harmless log messages of + the form: + `IllegalStateException: sending message to a Handler on a dead thread` + ([#8328](https://github.com/google/ExoPlayer/issues/8328)). + * Allow apps to fully customize DRM behaviour for each `MediaItem` by + passing a `DrmSessionManagerProvider` to `MediaSourceFactory` + ([#8466](https://github.com/google/ExoPlayer/issues/8466)). +* Analytics: + * Add an `onEvents` callback to `Player.EventListener` and + `AnalyticsListener`. When one or more player states change + simultaneously, `onEvents` is called once after all of the callbacks + associated with the individual state changes. + * Pass a `DecoderReuseEvaluation` to `AnalyticsListener`'s + `onVideoInputFormatChanged` and `onAudioInputFormatChanged` methods. The + `DecoderReuseEvaluation` indicates whether it was possible to re-use an + existing decoder instance for the new format, and if not then the + reasons why. +* Video: + * Fall back to AVC/HEVC decoders for Dolby Vision streams with level 10 + to 13 ([#8530](https://github.com/google/ExoPlayer/issues/8530)). + * Fix VP9 format capability checks on API level 23 and earlier. The + platform does not correctly report the VP9 level supported by the + decoder in this case, so we estimate it based on the decoder's maximum + supported bitrate. +* Audio: + * Fix handling of audio session IDs + ([#8190](https://github.com/google/ExoPlayer/issues/8190)). + `SimpleExoPlayer` now generates an audio session ID on construction, + which can be immediately queried by calling + `SimpleExoPlayer.getAudioSessionId`. The audio session ID will only + change if application code calls `SimpleExoPlayer.setAudioSessionId`. + * Replace `onAudioSessionId` with `onAudioSessionIdChanged` in + `AudioListener` and `AnalyticsListener`. Note that + `onAudioSessionIdChanged` is called in fewer cases than + `onAudioSessionId` was called, due to the improved handling of audio + session IDs as described above. + * Retry playback after some types of `AudioTrack` error. + * Create E-AC3 JOC passthrough `AudioTrack` instances using the maximum + supported channel count (instead of assuming 6 channels) from API 29. +* Text: + * Add support for the SSA `primaryColour` style attribute + ([#8435](https://github.com/google/ExoPlayer/issues/8435)). + * Fix CEA-708 sequence number discontinuity handling + ([#1807](https://github.com/google/ExoPlayer/issues/1807)). + * Fix CEA-708 handling of unexpectedly small packets + ([#1807](https://github.com/google/ExoPlayer/issues/1807)). +* Data sources: + * For `HttpDataSource` implementations, default to using the user agent of + the underlying network stack. + * Deprecate `HttpDataSource.Factory.getDefaultRequestProperties`. + `HttpDataSource.Factory.setDefaultRequestProperties` instead. + * Add `DefaultHttpDataSource.Factory` and deprecate + `DefaultHttpDataSourceFactory`. +* Metadata retriever: + * Parse Google Photos HEIC and JPEG motion photo metadata. +* IMA extension: + * Add support for playback of ads in playlists + ([#3750](https://github.com/google/ExoPlayer/issues/3750)). + * Add `ImaAdsLoader.Builder.setEnableContinuousPlayback` for setting + whether to request ads for continuous playback. + * Upgrade IMA SDK dependency to 3.22.0. This fixes leaking of the ad view + group ([#7344](https://github.com/google/ExoPlayer/issues/7344), + [#8339](https://github.com/google/ExoPlayer/issues/8339)). + * Fix a bug that could cause the next content position played after a seek + to snap back to the cue point of the preceding ad, rather than the + requested content position. + * Fix a regression that caused an ad group to be skipped after an initial + seek to a non-zero position. Unsupported VPAID ads will still be + skipped, but only after the preload timeout rather than instantly + ([#8428](https://github.com/google/ExoPlayer/issues/8428), + [#7832](https://github.com/google/ExoPlayer/issues/7832)). + * Fix a regression that caused a short ad followed by another ad to be + skipped due to playback being stuck buffering waiting for the second ad + to load ([#8492](https://github.com/google/ExoPlayer/issues/8492)). +* FFmpeg extension: + * Link the FFmpeg library statically, saving 350KB in binary size on + average. +* OkHttp extension: + * Add `OkHttpDataSource.Factory` and deprecate `OkHttpDataSourceFactory`. +* Cronet extension: + * Add `CronetDataSource.Factory` and deprecate `CronetDataSourceFactory`. + * Support setting the user agent on `CronetDataSource.Factory` and + `CronetEngineWrapper`. +* MediaSession extension: + * Support `setPlaybackSpeed(float)` and disable it by default. Use + `MediaSessionConnector.setEnabledPlaybackActions(long)` to enable + ([#8229](https://github.com/google/ExoPlayer/issues/8229)). +* Remove deprecated symbols: + * `AdaptiveMediaSourceEventListener`. Use `MediaSourceEventListener` + instead. + * `DashMediaSource.Factory.setMinLoadableRetryCount(int)`. Use + `DashMediaSource.Factory.setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)` + instead. + * `DefaultAnalyticsListener`. Use `AnalyticsListener` instead. + * `DefaultLoadControl` constructors. Use `DefaultLoadControl.Builder` + instead. + * `DrmInitData.get(UUID)`. Use `DrmInitData.get(int)` and + `DrmInitData.SchemeData.matches(UUID)` instead. + * `ExtractorsMediaSource.Factory.setMinLoadableRetryCount(int)`. Use + `ExtractorsMediaSource.Factory.setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)` + instead. + * `FixedTrackSelection.Factory`. If you need to disable adaptive + selection in `DefaultTrackSelector`, enable the + `DefaultTrackSelector.Parameters.forceHighestSupportedBitrate` flag. + * `HlsMediaSource.Factory.setMinLoadableRetryCount(int)`. Use + `HlsMediaSource.Factory.setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)` + instead. + * `MappedTrackInfo.getTrackFormatSupport(int, int, int)`. Use + `MappedTrackInfo.getTrackSupport(int, int, int)` instead. + * `MappedTrackInfo.getTrackTypeRendererSupport(int)`. Use + `MappedTrackInfo.getTypeSupport(int)` instead. + * `MappedTrackInfo.getUnassociatedTrackGroups()`. Use + `MappedTrackInfo.getUnmappedTrackGroups()` instead. + * `MappedTrackInfo.length`. Use `MappedTrackInfo.getRendererCount()` + instead. + * `Player.DefaultEventListener.onTimelineChanged(Timeline, Object)`. + Use `Player.EventListener.onTimelineChanged(Timeline, int)` instead. + * `Player.setAudioAttributes(AudioAttributes)`. Use + `Player.AudioComponent.setAudioAttributes(AudioAttributes, boolean)` + instead. + * `PlayerView.setDefaultArtwork(Bitmap)`. Use + `PlayerView.setDefaultArtwork(Drawable)` instead. + * `PlayerView.setShowBuffering(boolean)`. Use + `PlayerView.setShowBuffering(int)` instead. + * `SimpleExoPlayer.clearMetadataOutput(MetadataOutput)`. Use + `SimpleExoPlayer.removeMetadataOutput(MetadataOutput)` instead. + * `SimpleExoPlayer.clearTextOutput(TextOutput)`. Use + `SimpleExoPlayer.removeTextOutput(TextOutput)` instead. + * `SimpleExoPlayer.clearVideoListener()`. Use + `SimpleExoPlayer.removeVideoListener(VideoListener)` instead. + * `SimpleExoPlayer.getAudioStreamType()`. Use + `SimpleExoPlayer.getAudioAttributes()` instead. + * `SimpleExoPlayer.setAudioDebugListener(AudioRendererEventListener)`. + Use `SimpleExoPlayer.addAnalyticsListener(AnalyticsListener)` instead. + * `SimpleExoPlayer.setAudioStreamType(int)`. Use + `SimpleExoPlayer.setAudioAttributes(AudioAttributes)` instead. + * `SimpleExoPlayer.setMetadataOutput(MetadataOutput)`. Use + `SimpleExoPlayer.addMetadataOutput(MetadataOutput)` instead. If your + application is calling `SimpleExoPlayer.setMetadataOutput(null)`, make + sure to replace this call with a call to + `SimpleExoPlayer.removeMetadataOutput(MetadataOutput)`. + * `SimpleExoPlayer.setPlaybackParams(PlaybackParams)`. Use + `SimpleExoPlayer.setPlaybackParameters(PlaybackParameters)` instead. + * `SimpleExoPlayer.setTextOutput(TextOutput)`. Use + `SimpleExoPlayer.addTextOutput(TextOutput)` instead. If your + application is calling `SimpleExoPlayer.setTextOutput(null)`, make sure + to replace this call with a call to + `SimpleExoPlayer.removeTextOutput(TextOutput)`. + * `SimpleExoPlayer.setVideoDebugListener(VideoRendererEventListener)`. + Use `SimpleExoPlayer.addAnalyticsListener(AnalyticsListener)` instead. + * `SimpleExoPlayer.setVideoListener(VideoListener)`. Use + `SimpleExoPlayer.addVideoListener(VideoListener)` instead. If your + application is calling `SimpleExoPlayer.setVideoListener(null)`, make + sure to replace this call with a call to + `SimpleExoPlayer.removeVideoListener(VideoListener)`. + * `SimpleExoPlayer.VideoListener`. Use + `com.google.android.exoplayer2.video.VideoListener` instead. + * `SingleSampleMediaSource.EventListener` and constructors. Use + `MediaSourceEventListener` and `SingleSampleMediaSource.Factory` + instead. + * `SimpleExoPlayer.addVideoDebugListener`, + `SimpleExoPlayer.removeVideoDebugListener`, + `SimpleExoPlayer.addAudioDebugListener` and + `SimpleExoPlayer.removeAudioDebugListener`. Use + `SimpleExoPlayer.addAnalyticsListener` and + `SimpleExoPlayer.removeAnalyticsListener` instead. + * `SingleSampleMediaSource.Factory.setMinLoadableRetryCount(int)`. Use + `SingleSampleMediaSource.Factory.setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)` + instead. + * `SsMediaSource.Factory.setMinLoadableRetryCount(int)`. Use + `SsMediaSource.Factory.setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)` + instead. + ### 2.12.3 (2021-01-13) ### * Core library: diff --git a/build.gradle b/build.gradle index 8c044f2fdaa..8c2186522ae 100644 --- a/build.gradle +++ b/build.gradle @@ -26,7 +26,6 @@ allprojects { repositories { google() jcenter() - maven { url "https://oss.sonatype.org/content/repositories/snapshots" } } project.ext { exoplayerPublishEnabled = false diff --git a/constants.gradle b/constants.gradle index 50ce661b9a7..bb775e70509 100644 --- a/constants.gradle +++ b/constants.gradle @@ -13,18 +13,18 @@ // limitations under the License. project.ext { // ExoPlayer version and version code. - releaseVersion = '2.12.3' - releaseVersionCode = 2012003 + releaseVersion = '2.13.0' + releaseVersionCode = 2013000 minSdkVersion = 16 appTargetSdkVersion = 29 targetSdkVersion = 28 // TODO: Bump once b/143232359 is resolved. Also fix TODOs in UtilTest. - compileSdkVersion = 29 + compileSdkVersion = 30 dexmakerVersion = '2.21.0' junitVersion = '4.13-rc-2' guavaVersion = '27.1-android' mockitoVersion = '2.28.2' mockWebServerVersion = '3.12.0' - robolectricVersion = '4.4' + robolectricVersion = '4.5' checkerframeworkVersion = '3.3.0' checkerframeworkCompatVersion = '2.5.0' jsr305Version = '3.0.2' @@ -33,14 +33,15 @@ project.ext { androidxAppCompatVersion = '1.1.0' androidxCollectionVersion = '1.1.0' androidxFuturesVersion = '1.1.0' - androidxMediaVersion = '1.0.1' + androidxMediaVersion = '1.2.1' androidxMedia2Version = '1.1.0' androidxMultidexVersion = '2.0.0' androidxRecyclerViewVersion = '1.1.0' - androidxTestCoreVersion = '1.2.0' + androidxTestCoreVersion = '1.3.0' androidxTestJUnitVersion = '1.1.1' - androidxTestRunnerVersion = '1.2.0' - androidxTestRulesVersion = '1.2.0' + androidxTestRunnerVersion = '1.3.0' + androidxTestRulesVersion = '1.3.0' + androidxTestServicesStorageVersion = '1.3.0' truthVersion = '1.0' modulePrefix = ':' if (gradle.ext.has('exoplayerModulePrefix')) { diff --git a/core_settings.gradle b/core_settings.gradle index bd217a37e56..241b94a19ba 100644 --- a/core_settings.gradle +++ b/core_settings.gradle @@ -28,6 +28,7 @@ include modulePrefix + 'library-dash' include modulePrefix + 'library-extractor' include modulePrefix + 'library-hls' include modulePrefix + 'library-smoothstreaming' +include modulePrefix + 'library-transformer' include modulePrefix + 'library-ui' include modulePrefix + 'robolectricutils' include modulePrefix + 'testutils' @@ -56,6 +57,7 @@ project(modulePrefix + 'library-dash').projectDir = new File(rootDir, 'library/d project(modulePrefix + 'library-extractor').projectDir = new File(rootDir, 'library/extractor') project(modulePrefix + 'library-hls').projectDir = new File(rootDir, 'library/hls') project(modulePrefix + 'library-smoothstreaming').projectDir = new File(rootDir, 'library/smoothstreaming') +project(modulePrefix + 'library-transformer').projectDir = new File(rootDir, 'library/transformer') project(modulePrefix + 'library-ui').projectDir = new File(rootDir, 'library/ui') project(modulePrefix + 'robolectricutils').projectDir = new File(rootDir, 'robolectricutils') project(modulePrefix + 'testutils').projectDir = new File(rootDir, 'testutils') diff --git a/demos/cast/src/main/java/com/google/android/exoplayer2/castdemo/PlayerManager.java b/demos/cast/src/main/java/com/google/android/exoplayer2/castdemo/PlayerManager.java index 9dc82e0b236..8c367b8734d 100644 --- a/demos/cast/src/main/java/com/google/android/exoplayer2/castdemo/PlayerManager.java +++ b/demos/cast/src/main/java/com/google/android/exoplayer2/castdemo/PlayerManager.java @@ -312,7 +312,8 @@ private void setCurrentPlayer(Player currentPlayer) { windowIndex = currentItemIndex; } } - previousPlayer.stop(true); + previousPlayer.stop(); + previousPlayer.clearMediaItems(); } this.currentPlayer = currentPlayer; diff --git a/demos/gl/src/main/java/com/google/android/exoplayer2/gldemo/MainActivity.java b/demos/gl/src/main/java/com/google/android/exoplayer2/gldemo/MainActivity.java index dc0a8b990ac..191602dfb8c 100644 --- a/demos/gl/src/main/java/com/google/android/exoplayer2/gldemo/MainActivity.java +++ b/demos/gl/src/main/java/com/google/android/exoplayer2/gldemo/MainActivity.java @@ -152,7 +152,7 @@ private void initializePlayer() { .setUuidAndExoMediaDrmProvider(drmSchemeUuid, FrameworkMediaDrm.DEFAULT_PROVIDER) .build(drmCallback); } else { - drmSessionManager = DrmSessionManager.getDummyDrmSessionManager(); + drmSessionManager = DrmSessionManager.DRM_UNSUPPORTED; } DataSource.Factory dataSourceFactory = new DefaultDataSourceFactory(this); diff --git a/demos/main/src/main/assets/media.exolist.json b/demos/main/src/main/assets/media.exolist.json index 4d9e1df23a9..b515eca98a2 100644 --- a/demos/main/src/main/assets/media.exolist.json +++ b/demos/main/src/main/assets/media.exolist.json @@ -354,6 +354,23 @@ "name": "VMAP midroll ad pod at 5 s with 10 skippable ads", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp4/frame-counter-one-hour.mp4", "ad_tag_uri": "https://vastsynthesizer.appspot.com/midroll-10-skippable-ads" + }, + { + "name": "Playlist with three ad tags", + "playlist": [ + { + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp4/android-screens-10s.mp4", + "ad_tag_uri": "https://pubads.g.doubleclick.net/gampad/ads?sz=640x480&iu=/124319096/external/single_ad_samples&ciu_szs=300x250&impl=s&gdfp_req=1&env=vp&output=vast&unviewed_position_start=1&cust_params=deployment%3Ddevsite%26sample_ct%3Dlinear&correlator=" + }, + { + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp4/android-screens-25s.mp4", + "ad_tag_uri": "https://pubads.g.doubleclick.net/gampad/ads?sz=640x480&iu=/124319096/external/ad_rule_samples&ciu_szs=300x250&ad_rule=1&impl=s&gdfp_req=1&env=vp&output=vmap&unviewed_position_start=1&cust_params=deployment%3Ddevsite%26sample_ar%3Dpremidpost&cmsid=496&vid=short_onecue&correlator=" + }, + { + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp4/frame-counter-one-hour.mp4", + "ad_tag_uri": "https://pubads.g.doubleclick.net/gampad/ads?sz=640x480&iu=/124319096/external/ad_rule_samples&ciu_szs=300x250&ad_rule=1&impl=s&gdfp_req=1&env=vp&output=vmap&unviewed_position_start=1&cust_params=deployment%3Ddevsite%26sample_ar%3Dpremidpostlongpod&cmsid=496&vid=short_tencue&correlator=" + } + ] } ] }, diff --git a/demos/main/src/main/java/com/google/android/exoplayer2/demo/DemoUtil.java b/demos/main/src/main/java/com/google/android/exoplayer2/demo/DemoUtil.java index 2d15dfcbb48..080387db7e1 100644 --- a/demos/main/src/main/java/com/google/android/exoplayer2/demo/DemoUtil.java +++ b/demos/main/src/main/java/com/google/android/exoplayer2/demo/DemoUtil.java @@ -16,11 +16,13 @@ package com.google.android.exoplayer2.demo; import android.content.Context; +import android.os.Build; import com.google.android.exoplayer2.DefaultRenderersFactory; +import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.RenderersFactory; import com.google.android.exoplayer2.database.DatabaseProvider; import com.google.android.exoplayer2.database.ExoDatabaseProvider; -import com.google.android.exoplayer2.ext.cronet.CronetDataSourceFactory; +import com.google.android.exoplayer2.ext.cronet.CronetDataSource; import com.google.android.exoplayer2.ext.cronet.CronetEngineWrapper; import com.google.android.exoplayer2.offline.ActionFileUpgradeUtil; import com.google.android.exoplayer2.offline.DefaultDownloadIndex; @@ -28,6 +30,7 @@ import com.google.android.exoplayer2.ui.DownloadNotificationHelper; import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DefaultDataSourceFactory; +import com.google.android.exoplayer2.upstream.DefaultHttpDataSource; import com.google.android.exoplayer2.upstream.HttpDataSource; import com.google.android.exoplayer2.upstream.cache.Cache; import com.google.android.exoplayer2.upstream.cache.CacheDataSource; @@ -36,6 +39,9 @@ import com.google.android.exoplayer2.util.Log; import java.io.File; import java.io.IOException; +import java.net.CookieHandler; +import java.net.CookieManager; +import java.net.CookiePolicy; import java.util.concurrent.Executors; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @@ -44,6 +50,22 @@ public final class DemoUtil { public static final String DOWNLOAD_NOTIFICATION_CHANNEL_ID = "download_channel"; + /** + * Whether the demo application uses Cronet for networking. Note that Cronet does not provide + * automatic support for cookies (https://github.com/google/ExoPlayer/issues/5975). + * + *

If set to false, the platform's default network stack is used with a {@link CookieManager} + * configured in {@link #getHttpDataSourceFactory}. + */ + private static final boolean USE_CRONET_FOR_NETWORKING = true; + + private static final String USER_AGENT = + "ExoPlayerDemo/" + + ExoPlayerLibraryInfo.VERSION + + " (Linux; Android " + + Build.VERSION.RELEASE + + ") " + + ExoPlayerLibraryInfo.VERSION_SLASHY; private static final String TAG = "DemoUtil"; private static final String DOWNLOAD_ACTION_FILE = "actions"; private static final String DOWNLOAD_TRACKER_ACTION_FILE = "tracked_actions"; @@ -78,10 +100,18 @@ public static RenderersFactory buildRenderersFactory( public static synchronized HttpDataSource.Factory getHttpDataSourceFactory(Context context) { if (httpDataSourceFactory == null) { - context = context.getApplicationContext(); - CronetEngineWrapper cronetEngineWrapper = new CronetEngineWrapper(context); - httpDataSourceFactory = - new CronetDataSourceFactory(cronetEngineWrapper, Executors.newSingleThreadExecutor()); + if (USE_CRONET_FOR_NETWORKING) { + context = context.getApplicationContext(); + CronetEngineWrapper cronetEngineWrapper = + new CronetEngineWrapper(context, USER_AGENT, /* preferGMSCoreCronet= */ false); + httpDataSourceFactory = + new CronetDataSource.Factory(cronetEngineWrapper, Executors.newSingleThreadExecutor()); + } else { + CookieManager cookieManager = new CookieManager(); + cookieManager.setCookiePolicy(CookiePolicy.ACCEPT_ORIGINAL_SERVER); + CookieHandler.setDefault(cookieManager); + httpDataSourceFactory = new DefaultHttpDataSource.Factory().setUserAgent(USER_AGENT); + } } return httpDataSourceFactory; } diff --git a/demos/main/src/main/java/com/google/android/exoplayer2/demo/DownloadTracker.java b/demos/main/src/main/java/com/google/android/exoplayer2/demo/DownloadTracker.java index 80c64bc9bd9..2cf2671abab 100644 --- a/demos/main/src/main/java/com/google/android/exoplayer2/demo/DownloadTracker.java +++ b/demos/main/src/main/java/com/google/android/exoplayer2/demo/DownloadTracker.java @@ -332,7 +332,7 @@ private void onDownloadPrepared(DownloadHelper helper) { /* titleId= */ R.string.exo_download_description, mappedTrackInfo, trackSelectorParameters, - /* allowAdaptiveSelections =*/ false, + /* allowAdaptiveSelections= */ false, /* allowMultipleOverrides= */ true, /* onClickListener= */ this, /* onDismissListener= */ this); diff --git a/demos/main/src/main/java/com/google/android/exoplayer2/demo/IntentUtil.java b/demos/main/src/main/java/com/google/android/exoplayer2/demo/IntentUtil.java index d2d962c568e..d1cb0357187 100644 --- a/demos/main/src/main/java/com/google/android/exoplayer2/demo/IntentUtil.java +++ b/demos/main/src/main/java/com/google/android/exoplayer2/demo/IntentUtil.java @@ -173,7 +173,9 @@ private static void addPlaybackPropertiesToIntent( .putExtra(MIME_TYPE_EXTRA + extrasKeySuffix, playbackProperties.mimeType) .putExtra( AD_TAG_URI_EXTRA + extrasKeySuffix, - playbackProperties.adTagUri != null ? playbackProperties.adTagUri.toString() : null); + playbackProperties.adsConfiguration != null + ? playbackProperties.adsConfiguration.adTagUri.toString() + : null); if (playbackProperties.drmConfiguration != null) { addDrmConfigurationToIntent(playbackProperties.drmConfiguration, intent, extrasKeySuffix); } diff --git a/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java b/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java index e1a698acdca..5fb342be432 100644 --- a/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java +++ b/demos/main/src/main/java/com/google/android/exoplayer2/demo/PlayerActivity.java @@ -19,7 +19,6 @@ import android.content.Intent; import android.content.pm.PackageManager; -import android.net.Uri; import android.os.Bundle; import android.util.Pair; import android.view.KeyEvent; @@ -59,9 +58,6 @@ import com.google.android.exoplayer2.util.ErrorMessageProvider; import com.google.android.exoplayer2.util.EventLogger; import com.google.android.exoplayer2.util.Util; -import java.net.CookieHandler; -import java.net.CookieManager; -import java.net.CookiePolicy; import java.util.ArrayList; import java.util.Collections; import java.util.List; @@ -78,13 +74,6 @@ public class PlayerActivity extends AppCompatActivity private static final String KEY_POSITION = "position"; private static final String KEY_AUTO_PLAY = "auto_play"; - private static final CookieManager DEFAULT_COOKIE_MANAGER; - - static { - DEFAULT_COOKIE_MANAGER = new CookieManager(); - DEFAULT_COOKIE_MANAGER.setCookiePolicy(CookiePolicy.ACCEPT_ORIGINAL_SERVER); - } - protected StyledPlayerView playerView; protected LinearLayout debugRootView; protected TextView debugTextView; @@ -102,20 +91,16 @@ public class PlayerActivity extends AppCompatActivity private int startWindow; private long startPosition; - // Fields used only for ad playback. + // For ad playback only. private AdsLoader adsLoader; - private Uri loadedAdTagUri; - // Activity lifecycle + // Activity lifecycle. @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); dataSourceFactory = DemoUtil.getDataSourceFactory(/* context= */ this); - if (CookieHandler.getDefault() != DEFAULT_COOKIE_MANAGER) { - CookieHandler.setDefault(DEFAULT_COOKIE_MANAGER); - } setContentView(); debugRootView = findViewById(R.id.controls_root); @@ -348,7 +333,7 @@ private List createMediaItems(Intent intent) { return Collections.emptyList(); } } - hasAds |= mediaItem.playbackProperties.adTagUri != null; + hasAds |= mediaItem.playbackProperties.adsConfiguration != null; } if (!hasAds) { releaseAdsLoader(); @@ -356,16 +341,7 @@ private List createMediaItems(Intent intent) { return mediaItems; } - private AdsLoader getAdsLoader(Uri adTagUri) { - if (mediaItems.size() > 1) { - showToast(R.string.unsupported_ads_in_playlist); - releaseAdsLoader(); - return null; - } - if (!adTagUri.equals(loadedAdTagUri)) { - releaseAdsLoader(); - loadedAdTagUri = adTagUri; - } + private AdsLoader getAdsLoader(MediaItem.AdsConfiguration adsConfiguration) { // The ads loader is reused for multiple playbacks, so that ad playback can resume. if (adsLoader == null) { adsLoader = new ImaAdsLoader.Builder(/* context= */ this).build(); @@ -394,7 +370,6 @@ private void releaseAdsLoader() { if (adsLoader != null) { adsLoader.release(); adsLoader = null; - loadedAdTagUri = null; playerView.getOverlayFrameLayout().removeAllViews(); } } diff --git a/demos/main/src/main/java/com/google/android/exoplayer2/demo/SampleChooserActivity.java b/demos/main/src/main/java/com/google/android/exoplayer2/demo/SampleChooserActivity.java index ea5b38ce8e8..a66a1e03014 100644 --- a/demos/main/src/main/java/com/google/android/exoplayer2/demo/SampleChooserActivity.java +++ b/demos/main/src/main/java/com/google/android/exoplayer2/demo/SampleChooserActivity.java @@ -252,7 +252,7 @@ private int getDownloadUnsupportedStringId(PlaylistHolder playlistHolder) { } MediaItem.PlaybackProperties playbackProperties = checkNotNull(playlistHolder.mediaItems.get(0).playbackProperties); - if (playbackProperties.adTagUri != null) { + if (playbackProperties.adsConfiguration != null) { return R.string.download_ads_unsupported; } String scheme = playbackProperties.uri.getScheme(); diff --git a/demos/main/src/main/java/com/google/android/exoplayer2/demo/TrackSelectionDialog.java b/demos/main/src/main/java/com/google/android/exoplayer2/demo/TrackSelectionDialog.java index d3f9b3880da..5b50298df0d 100644 --- a/demos/main/src/main/java/com/google/android/exoplayer2/demo/TrackSelectionDialog.java +++ b/demos/main/src/main/java/com/google/android/exoplayer2/demo/TrackSelectionDialog.java @@ -94,7 +94,7 @@ public static TrackSelectionDialog createForTrackSelector( /* titleId= */ R.string.track_selection_title, mappedTrackInfo, /* initialParameters = */ parameters, - /* allowAdaptiveSelections =*/ true, + /* allowAdaptiveSelections= */ true, /* allowMultipleOverrides= */ false, /* onClickListener= */ (dialog, which) -> { DefaultTrackSelector.ParametersBuilder builder = parameters.buildUpon(); diff --git a/demos/main/src/main/res/values/strings.xml b/demos/main/src/main/res/values/strings.xml index 9fe882b1629..49441ef7dac 100644 --- a/demos/main/src/main/res/values/strings.xml +++ b/demos/main/src/main/res/values/strings.xml @@ -45,8 +45,6 @@ One or more sample lists failed to load - Playing without ads, as ads are not supported in playlists - Failed to start download Failed to obtain offline license diff --git a/demos/surface/src/main/java/com/google/android/exoplayer2/surfacedemo/MainActivity.java b/demos/surface/src/main/java/com/google/android/exoplayer2/surfacedemo/MainActivity.java index eb669ecf946..a31cd7efe01 100644 --- a/demos/surface/src/main/java/com/google/android/exoplayer2/surfacedemo/MainActivity.java +++ b/demos/surface/src/main/java/com/google/android/exoplayer2/surfacedemo/MainActivity.java @@ -197,7 +197,7 @@ private void initializePlayer() { .setUuidAndExoMediaDrmProvider(drmSchemeUuid, FrameworkMediaDrm.DEFAULT_PROVIDER) .build(drmCallback); } else { - drmSessionManager = DrmSessionManager.getDummyDrmSessionManager(); + drmSessionManager = DrmSessionManager.DRM_UNSUPPORTED; } DataSource.Factory dataSourceFactory = new DefaultDataSourceFactory(this); diff --git a/extensions/av1/proguard-rules.txt b/extensions/av1/proguard-rules.txt index 9d73f7e2b58..c4ef2286f22 100644 --- a/extensions/av1/proguard-rules.txt +++ b/extensions/av1/proguard-rules.txt @@ -5,3 +5,7 @@ native ; } +# Some members of this class are being accessed from native methods. Keep them unobfuscated. +-keep class com.google.android.exoplayer2.video.VideoDecoderOutputBuffer { + *; +} diff --git a/extensions/av1/src/main/java/com/google/android/exoplayer2/ext/av1/Gav1Decoder.java b/extensions/av1/src/main/java/com/google/android/exoplayer2/ext/av1/Gav1Decoder.java index ad8c8a682cc..08d48e9699e 100644 --- a/extensions/av1/src/main/java/com/google/android/exoplayer2/ext/av1/Gav1Decoder.java +++ b/extensions/av1/src/main/java/com/google/android/exoplayer2/ext/av1/Gav1Decoder.java @@ -15,10 +15,12 @@ */ package com.google.android.exoplayer2.ext.av1; +import static androidx.annotation.VisibleForTesting.PACKAGE_PRIVATE; import static java.lang.Runtime.getRuntime; import android.view.Surface; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; import com.google.android.exoplayer2.decoder.SimpleDecoder; @@ -28,7 +30,8 @@ import java.nio.ByteBuffer; /** Gav1 decoder. */ -/* package */ final class Gav1Decoder +@VisibleForTesting(otherwise = PACKAGE_PRIVATE) +public final class Gav1Decoder extends SimpleDecoder { // LINT.IfChange diff --git a/extensions/av1/src/main/java/com/google/android/exoplayer2/ext/av1/Libgav1VideoRenderer.java b/extensions/av1/src/main/java/com/google/android/exoplayer2/ext/av1/Libgav1VideoRenderer.java index 7c558d24b2d..a083cd8414d 100644 --- a/extensions/av1/src/main/java/com/google/android/exoplayer2/ext/av1/Libgav1VideoRenderer.java +++ b/extensions/av1/src/main/java/com/google/android/exoplayer2/ext/av1/Libgav1VideoRenderer.java @@ -15,12 +15,15 @@ */ package com.google.android.exoplayer2.ext.av1; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITHOUT_RECONFIGURATION; + import android.os.Handler; import android.view.Surface; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.RendererCapabilities; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.drm.ExoMediaCrypto; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.TraceUtil; @@ -42,7 +45,10 @@ public class Libgav1VideoRenderer extends DecoderVideoRenderer { private static final String TAG = "Libgav1VideoRenderer"; private static final int DEFAULT_NUM_OF_INPUT_BUFFERS = 4; private static final int DEFAULT_NUM_OF_OUTPUT_BUFFERS = 4; - /* Default size based on 720p resolution video compressed by a factor of two. */ + /** + * Default input buffer size in bytes, based on 720p resolution video compressed by a factor of + * two. + */ private static final int DEFAULT_INPUT_BUFFER_SIZE = Util.ceilDivide(1280, 64) * Util.ceilDivide(720, 64) * (64 * 64 * 3 / 2) / 2; @@ -124,12 +130,13 @@ public String getName() { public final int supportsFormat(Format format) { if (!MimeTypes.VIDEO_AV1.equalsIgnoreCase(format.sampleMimeType) || !Gav1Library.isAvailable()) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } if (format.exoMediaCryptoType != null) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_DRM); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_DRM); } - return RendererCapabilities.create(FORMAT_HANDLED, ADAPTIVE_SEAMLESS, TUNNELING_NOT_SUPPORTED); + return RendererCapabilities.create( + C.FORMAT_HANDLED, ADAPTIVE_SEAMLESS, TUNNELING_NOT_SUPPORTED); } @Override @@ -164,7 +171,13 @@ protected void setDecoderOutputMode(@C.VideoOutputMode int outputMode) { } @Override - protected boolean canKeepCodec(Format oldFormat, Format newFormat) { - return true; + protected DecoderReuseEvaluation canReuseDecoder( + String decoderName, Format oldFormat, Format newFormat) { + return new DecoderReuseEvaluation( + decoderName, + oldFormat, + newFormat, + REUSE_RESULT_YES_WITHOUT_RECONFIGURATION, + /* discardReasons= */ 0); } } diff --git a/extensions/cast/build.gradle b/extensions/cast/build.gradle index 4c8f648e344..d0cc501fcb4 100644 --- a/extensions/cast/build.gradle +++ b/extensions/cast/build.gradle @@ -16,7 +16,7 @@ apply from: "$gradle.ext.exoplayerSettingsDir/common_library_config.gradle" dependencies { api 'com.google.android.gms:play-services-cast-framework:18.1.0' implementation 'androidx.annotation:annotation:' + androidxAnnotationVersion - implementation project(modulePrefix + 'library-core') + implementation project(modulePrefix + 'library-common') implementation project(modulePrefix + 'library-ui') compileOnly 'org.checkerframework:checker-qual:' + checkerframeworkVersion compileOnly 'org.checkerframework:checker-compat-qual:' + checkerframeworkCompatVersion diff --git a/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastPlayer.java b/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastPlayer.java index eeda98d2d9e..d20b84cbc3e 100644 --- a/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastPlayer.java +++ b/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastPlayer.java @@ -27,13 +27,14 @@ import com.google.android.exoplayer2.PlaybackParameters; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.TrackGroupArray; -import com.google.android.exoplayer2.trackselection.FixedTrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; -import com.google.android.exoplayer2.trackselection.TrackSelector; import com.google.android.exoplayer2.util.Assertions; +import com.google.android.exoplayer2.util.Clock; +import com.google.android.exoplayer2.util.ListenerSet; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.gms.cast.CastStatusCodes; @@ -49,11 +50,8 @@ import com.google.android.gms.cast.framework.media.RemoteMediaClient.MediaChannelResult; import com.google.android.gms.common.api.PendingResult; import com.google.android.gms.common.api.ResultCallback; -import java.util.ArrayDeque; -import java.util.ArrayList; -import java.util.Iterator; +import java.util.Collections; import java.util.List; -import java.util.concurrent.CopyOnWriteArrayList; import org.checkerframework.checker.nullness.qual.RequiresNonNull; /** @@ -97,9 +95,7 @@ public final class CastPlayer extends BasePlayer { private final SeekResultCallback seekResultCallback; // Listeners and notification. - private final CopyOnWriteArrayList listeners; - private final ArrayList notificationsBatch; - private final ArrayDeque ongoingNotificationsTasks; + private final ListenerSet listeners; @Nullable private SessionAvailabilityListener sessionAvailabilityListener; // Internal state. @@ -138,9 +134,12 @@ public CastPlayer(CastContext castContext, MediaItemConverter mediaItemConverter period = new Timeline.Period(); statusListener = new StatusListener(); seekResultCallback = new SeekResultCallback(); - listeners = new CopyOnWriteArrayList<>(); - notificationsBatch = new ArrayList<>(); - ongoingNotificationsTasks = new ArrayDeque<>(); + listeners = + new ListenerSet<>( + Looper.getMainLooper(), + Clock.DEFAULT, + Player.Events::new, + (listener, eventFlags) -> listener.onEvents(/* player= */ this, eventFlags)); playWhenReady = new StateHolder<>(false); repeatMode = new StateHolder<>(REPEAT_MODE_OFF); @@ -293,18 +292,12 @@ public Looper getApplicationLooper() { @Override public void addListener(EventListener listener) { - Assertions.checkNotNull(listener); - listeners.addIfAbsent(new ListenerHolder(listener)); + listeners.add(listener); } @Override public void removeListener(EventListener listener) { - for (ListenerHolder listenerHolder : listeners) { - if (listenerHolder.listener.equals(listener)) { - listenerHolder.release(); - listeners.remove(listenerHolder); - } - } + listeners.remove(listener); } @Override @@ -416,7 +409,7 @@ public void setPlayWhenReady(boolean playWhenReady) { // the local state will be updated to reflect the state reported by the Cast SDK. setPlayerStateAndNotifyIfChanged( playWhenReady, PLAY_WHEN_READY_CHANGE_REASON_USER_REQUEST, playbackState); - flushNotifications(); + listeners.flushEvents(); PendingResult pendingResult = playWhenReady ? remoteMediaClient.play() : remoteMediaClient.pause(); this.playWhenReady.pendingResultCallback = @@ -425,7 +418,7 @@ public void setPlayWhenReady(boolean playWhenReady) { public void onResult(MediaChannelResult mediaChannelResult) { if (remoteMediaClient != null) { updatePlayerStateAndNotifyIfChanged(this); - flushNotifications(); + listeners.flushEvents(); } } }; @@ -456,13 +449,13 @@ public void seekTo(int windowIndex, long positionMs) { pendingSeekCount++; pendingSeekWindowIndex = windowIndex; pendingSeekPositionMs = positionMs; - notificationsBatch.add( - new ListenerNotificationTask( - listener -> listener.onPositionDiscontinuity(DISCONTINUITY_REASON_SEEK))); + listeners.queueEvent( + Player.EVENT_POSITION_DISCONTINUITY, + listener -> listener.onPositionDiscontinuity(DISCONTINUITY_REASON_SEEK)); } else if (pendingSeekCount == 0) { - notificationsBatch.add(new ListenerNotificationTask(EventListener::onSeekProcessed)); + listeners.queueEvent(/* eventFlag= */ C.INDEX_UNSET, EventListener::onSeekProcessed); } - flushNotifications(); + listeners.flushEvents(); } @Override @@ -511,12 +504,6 @@ public int getRendererType(int index) { } } - @Override - @Nullable - public TrackSelector getTrackSelector() { - return null; - } - @Override public void setRepeatMode(@RepeatMode int repeatMode) { if (remoteMediaClient == null) { @@ -526,7 +513,7 @@ public void setRepeatMode(@RepeatMode int repeatMode) { // operation to be perceived as synchronous by the user. When the operation reports a result, // the local state will be updated to reflect the state reported by the Cast SDK. setRepeatModeAndNotifyIfChanged(repeatMode); - flushNotifications(); + listeners.flushEvents(); PendingResult pendingResult = remoteMediaClient.queueSetRepeatMode(getCastRepeatMode(repeatMode), /* jsonObject= */ null); this.repeatMode.pendingResultCallback = @@ -535,7 +522,7 @@ public void setRepeatMode(@RepeatMode int repeatMode) { public void onResult(MediaChannelResult mediaChannelResult) { if (remoteMediaClient != null) { updateRepeatModeAndNotifyIfChanged(this); - flushNotifications(); + listeners.flushEvents(); } } }; @@ -568,6 +555,12 @@ public TrackGroupArray getCurrentTrackGroups() { return currentTrackGroups; } + @Override + public List getCurrentStaticMetadata() { + // CastPlayer does not currently support metadata. + return Collections.emptyList(); + } + @Override public Timeline getCurrentTimeline() { return currentTimeline; @@ -654,8 +647,8 @@ private void updateInternalStateAndNotifyIfChanged() { updatePlayerStateAndNotifyIfChanged(/* resultCallback= */ null); boolean isPlaying = playbackState == Player.STATE_READY && playWhenReady.value; if (wasPlaying != isPlaying) { - notificationsBatch.add( - new ListenerNotificationTask(listener -> listener.onIsPlayingChanged(isPlaying))); + listeners.queueEvent( + Player.EVENT_IS_PLAYING_CHANGED, listener -> listener.onIsPlayingChanged(isPlaying)); } updateRepeatModeAndNotifyIfChanged(/* resultCallback= */ null); updateTimelineAndNotifyIfChanged(); @@ -671,17 +664,16 @@ private void updateInternalStateAndNotifyIfChanged() { } if (this.currentWindowIndex != currentWindowIndex && pendingSeekCount == 0) { this.currentWindowIndex = currentWindowIndex; - notificationsBatch.add( - new ListenerNotificationTask( - listener -> - listener.onPositionDiscontinuity(DISCONTINUITY_REASON_PERIOD_TRANSITION))); + listeners.queueEvent( + Player.EVENT_POSITION_DISCONTINUITY, + listener -> listener.onPositionDiscontinuity(DISCONTINUITY_REASON_PERIOD_TRANSITION)); } if (updateTracksAndSelectionsAndNotifyIfChanged()) { - notificationsBatch.add( - new ListenerNotificationTask( - listener -> listener.onTracksChanged(currentTrackGroups, currentTrackSelection))); + listeners.queueEvent( + Player.EVENT_TRACKS_CHANGED, + listener -> listener.onTracksChanged(currentTrackGroups, currentTrackSelection)); } - flushNotifications(); + listeners.flushEvents(); } /** @@ -720,11 +712,11 @@ private void updateTimelineAndNotifyIfChanged() { if (updateTimeline()) { // TODO: Differentiate TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED and // TIMELINE_CHANGE_REASON_SOURCE_UPDATE [see internal: b/65152553]. - notificationsBatch.add( - new ListenerNotificationTask( - listener -> - listener.onTimelineChanged( - currentTimeline, Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE))); + listeners.queueEvent( + Player.EVENT_TIMELINE_CHANGED, + listener -> + listener.onTimelineChanged( + currentTimeline, Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE)); } } @@ -773,7 +765,7 @@ private boolean updateTracksAndSelectionsAndNotifyIfChanged() { int rendererIndex = getRendererIndexForTrackType(trackType); if (isTrackActive(id, activeTrackIds) && rendererIndex != C.INDEX_UNSET && trackSelections[rendererIndex] == null) { - trackSelections[rendererIndex] = new FixedTrackSelection(trackGroups[i], 0); + trackSelections[rendererIndex] = new CastTrackSelection(trackGroups[i]); } } TrackGroupArray newTrackGroups = new TrackGroupArray(trackGroups); @@ -843,8 +835,8 @@ private PendingResult removeMediaItemsInternal(int[] uids) { private void setRepeatModeAndNotifyIfChanged(@Player.RepeatMode int repeatMode) { if (this.repeatMode.value != repeatMode) { this.repeatMode.value = repeatMode; - notificationsBatch.add( - new ListenerNotificationTask(listener -> listener.onRepeatModeChanged(repeatMode))); + listeners.queueEvent( + Player.EVENT_REPEAT_MODE_CHANGED, listener -> listener.onRepeatModeChanged(repeatMode)); } } @@ -858,17 +850,19 @@ private void setPlayerStateAndNotifyIfChanged( if (playWhenReadyChanged || playbackStateChanged) { this.playbackState = playbackState; this.playWhenReady.value = playWhenReady; - notificationsBatch.add( - new ListenerNotificationTask( - listener -> { - listener.onPlayerStateChanged(playWhenReady, playbackState); - if (playbackStateChanged) { - listener.onPlaybackStateChanged(playbackState); - } - if (playWhenReadyChanged) { - listener.onPlayWhenReadyChanged(playWhenReady, playWhenReadyChangeReason); - } - })); + listeners.queueEvent( + /* eventFlag= */ C.INDEX_UNSET, + listener -> listener.onPlayerStateChanged(playWhenReady, playbackState)); + if (playbackStateChanged) { + listeners.queueEvent( + Player.EVENT_PLAYBACK_STATE_CHANGED, + listener -> listener.onPlaybackStateChanged(playbackState)); + } + if (playWhenReadyChanged) { + listeners.queueEvent( + Player.EVENT_PLAY_WHEN_READY_CHANGED, + listener -> listener.onPlayWhenReadyChanged(playWhenReady, playWhenReadyChangeReason)); + } } } @@ -976,20 +970,6 @@ private static int getCastRepeatMode(@RepeatMode int repeatMode) { } } - private void flushNotifications() { - boolean recursiveNotification = !ongoingNotificationsTasks.isEmpty(); - ongoingNotificationsTasks.addAll(notificationsBatch); - notificationsBatch.clear(); - if (recursiveNotification) { - // This will be handled once the current notification task is finished. - return; - } - while (!ongoingNotificationsTasks.isEmpty()) { - ongoingNotificationsTasks.peekFirst().execute(); - ongoingNotificationsTasks.removeFirst(); - } - } - private MediaQueueItem[] toMediaQueueItems(List mediaItems) { MediaQueueItem[] mediaQueueItems = new MediaQueueItem[mediaItems.size()]; for (int i = 0; i < mediaItems.size(); i++) { @@ -1100,8 +1080,7 @@ public void onResult(MediaChannelResult result) { if (--pendingSeekCount == 0) { pendingSeekWindowIndex = C.INDEX_UNSET; pendingSeekPositionMs = C.TIME_UNSET; - notificationsBatch.add(new ListenerNotificationTask(EventListener::onSeekProcessed)); - flushNotifications(); + listeners.sendEvent(/* eventFlag= */ C.INDEX_UNSET, EventListener::onSeekProcessed); } } } @@ -1141,21 +1120,4 @@ public boolean acceptsUpdate(@Nullable ResultCallback resultCallback) { return pendingResultCallback == resultCallback; } } - - private final class ListenerNotificationTask { - - private final Iterator listenersSnapshot; - private final ListenerInvocation listenerInvocation; - - private ListenerNotificationTask(ListenerInvocation listenerInvocation) { - this.listenersSnapshot = listeners.iterator(); - this.listenerInvocation = listenerInvocation; - } - - public void execute() { - while (listenersSnapshot.hasNext()) { - listenersSnapshot.next().invoke(listenerInvocation); - } - } - } } diff --git a/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastTimeline.java b/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastTimeline.java index edd2a060d28..1a03b779f60 100644 --- a/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastTimeline.java +++ b/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastTimeline.java @@ -46,8 +46,8 @@ public static final class ItemData { private ItemData() { this( - /* durationUs= */ C.TIME_UNSET, /* defaultPositionUs */ - C.TIME_UNSET, + /* durationUs= */ C.TIME_UNSET, + /* defaultPositionUs= */ C.TIME_UNSET, /* isLive= */ false); } @@ -126,16 +126,18 @@ public int getWindowCount() { public Window getWindow(int windowIndex, Window window, long defaultPositionProjectionUs) { long durationUs = durationsUs[windowIndex]; boolean isDynamic = durationUs == C.TIME_UNSET; + MediaItem mediaItem = + new MediaItem.Builder().setUri(Uri.EMPTY).setTag(ids[windowIndex]).build(); return window.set( /* uid= */ ids[windowIndex], - /* mediaItem= */ new MediaItem.Builder().setUri(Uri.EMPTY).setTag(ids[windowIndex]).build(), + /* mediaItem= */ mediaItem, /* manifest= */ null, /* presentationStartTimeMs= */ C.TIME_UNSET, /* windowStartTimeMs= */ C.TIME_UNSET, /* elapsedRealtimeEpochOffsetMs= */ C.TIME_UNSET, /* isSeekable= */ !isDynamic, isDynamic, - isLive[windowIndex], + isLive[windowIndex] ? mediaItem.liveConfiguration : null, defaultPositionsUs[windowIndex], durationUs, /* firstPeriodIndex= */ windowIndex, diff --git a/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastTrackSelection.java b/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastTrackSelection.java new file mode 100644 index 00000000000..22fe86d9e4e --- /dev/null +++ b/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/CastTrackSelection.java @@ -0,0 +1,91 @@ +/* + * Copyright (C) 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.ext.cast; + +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.source.TrackGroup; +import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.util.Assertions; + +/** + * {@link TrackSelection} that only selects the first track of the provided {@link TrackGroup}. + * + *

This relies on {@link CastPlayer} track groups only having one track. + */ +/* package */ class CastTrackSelection implements TrackSelection { + + private final TrackGroup trackGroup; + + /** @param trackGroup The {@link TrackGroup} from which the first track will only be selected. */ + public CastTrackSelection(TrackGroup trackGroup) { + this.trackGroup = trackGroup; + } + + @Override + public TrackGroup getTrackGroup() { + return trackGroup; + } + + @Override + public int length() { + return 1; + } + + @Override + public Format getFormat(int index) { + Assertions.checkArgument(index == 0); + return trackGroup.getFormat(0); + } + + @Override + public int getIndexInTrackGroup(int index) { + return index == 0 ? 0 : C.INDEX_UNSET; + } + + @Override + @SuppressWarnings("ReferenceEquality") + public int indexOf(Format format) { + return format == trackGroup.getFormat(0) ? 0 : C.INDEX_UNSET; + } + + @Override + public int indexOf(int indexInTrackGroup) { + return indexInTrackGroup == 0 ? 0 : C.INDEX_UNSET; + } + + // Object overrides. + + @Override + public int hashCode() { + return System.identityHashCode(trackGroup); + } + + // Track groups are compared by identity not value, as distinct groups may have the same value. + @Override + @SuppressWarnings({"ReferenceEquality", "EqualsGetClass"}) + public boolean equals(@Nullable Object obj) { + if (this == obj) { + return true; + } + if (obj == null || getClass() != obj.getClass()) { + return false; + } + CastTrackSelection other = (CastTrackSelection) obj; + return trackGroup == other.trackGroup; + } +} diff --git a/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/DefaultMediaItemConverter.java b/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/DefaultMediaItemConverter.java index 705f2c25085..c72a1fb316f 100644 --- a/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/DefaultMediaItemConverter.java +++ b/extensions/cast/src/main/java/com/google/android/exoplayer2/ext/cast/DefaultMediaItemConverter.java @@ -45,7 +45,9 @@ public final class DefaultMediaItemConverter implements MediaItemConverter { @Override public MediaItem toMediaItem(MediaQueueItem item) { // `item` came from `toMediaQueueItem()` so the custom JSON data must be set. - return getMediaItem(Assertions.checkNotNull(item.getMedia().getCustomData())); + MediaInfo mediaInfo = item.getMedia(); + Assertions.checkNotNull(mediaInfo); + return getMediaItem(Assertions.checkNotNull(mediaInfo.getCustomData())); } @Override diff --git a/extensions/cast/src/test/java/com/google/android/exoplayer2/ext/cast/CastTrackSelectionTest.java b/extensions/cast/src/test/java/com/google/android/exoplayer2/ext/cast/CastTrackSelectionTest.java new file mode 100644 index 00000000000..0a30c0c4b82 --- /dev/null +++ b/extensions/cast/src/test/java/com/google/android/exoplayer2/ext/cast/CastTrackSelectionTest.java @@ -0,0 +1,77 @@ +/* + * Copyright (C) 2018 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.ext.cast; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.source.TrackGroup; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Test for {@link CastTrackSelection}. */ +@RunWith(AndroidJUnit4.class) +public class CastTrackSelectionTest { + + private static final TrackGroup TRACK_GROUP = + new TrackGroup(new Format.Builder().build(), new Format.Builder().build()); + + private static final CastTrackSelection SELECTION = new CastTrackSelection(TRACK_GROUP); + + @Test + public void length_isOne() { + assertThat(SELECTION.length()).isEqualTo(1); + } + + @Test + public void getTrackGroup_returnsSameGroup() { + assertThat(SELECTION.getTrackGroup()).isSameInstanceAs(TRACK_GROUP); + } + + @Test + public void getFormatSelectedTrack_isFirstTrack() { + assertThat(SELECTION.getFormat(0)).isSameInstanceAs(TRACK_GROUP.getFormat(0)); + } + + @Test + public void getIndexInTrackGroup_ofSelectedTrack_returnsFirstTrack() { + assertThat(SELECTION.getIndexInTrackGroup(0)).isEqualTo(0); + } + + @Test + public void getIndexInTrackGroup_onePastTheEnd_returnsIndexUnset() { + assertThat(SELECTION.getIndexInTrackGroup(1)).isEqualTo(C.INDEX_UNSET); + } + + @Test + public void indexOf_selectedTrack_returnsFirstTrack() { + assertThat(SELECTION.indexOf(0)).isEqualTo(0); + } + + @Test + public void indexOf_onePastTheEnd_returnsIndexUnset() { + assertThat(SELECTION.indexOf(1)).isEqualTo(C.INDEX_UNSET); + } + + @Test(expected = Exception.class) + public void getFormat_outOfBound_throws() { + CastTrackSelection selection = new CastTrackSelection(TRACK_GROUP); + + selection.getFormat(1); + } +} diff --git a/extensions/cronet/build.gradle b/extensions/cronet/build.gradle index c0f443d5df7..f50304fb94a 100644 --- a/extensions/cronet/build.gradle +++ b/extensions/cronet/build.gradle @@ -13,14 +13,28 @@ // limitations under the License. apply from: "$gradle.ext.exoplayerSettingsDir/common_library_config.gradle" +android { + defaultConfig { + multiDexEnabled true + } +} + dependencies { api "com.google.android.gms:play-services-cronet:17.0.0" - implementation project(modulePrefix + 'library-core') + implementation project(modulePrefix + 'library-common') implementation 'androidx.annotation:annotation:' + androidxAnnotationVersion compileOnly 'org.checkerframework:checker-qual:' + checkerframeworkVersion compileOnly 'org.jetbrains.kotlin:kotlin-annotations-jvm:' + kotlinAnnotationsVersion + androidTestImplementation 'androidx.test:rules:' + androidxTestRulesVersion + androidTestImplementation 'androidx.test:runner:' + androidxTestRunnerVersion + androidTestImplementation 'androidx.multidex:multidex:' + androidxMultidexVersion + // Instrumentation tests assume that an app-packaged version of cronet is + // available. + androidTestImplementation 'org.chromium.net:cronet-embedded:72.3626.96' + androidTestImplementation(project(modulePrefix + 'testutils')) testImplementation project(modulePrefix + 'library') testImplementation project(modulePrefix + 'testutils') + testImplementation 'com.squareup.okhttp3:mockwebserver:' + mockWebServerVersion testImplementation 'org.robolectric:robolectric:' + robolectricVersion } diff --git a/extensions/cronet/src/androidTest/AndroidManifest.xml b/extensions/cronet/src/androidTest/AndroidManifest.xml new file mode 100644 index 00000000000..96e8e54f571 --- /dev/null +++ b/extensions/cronet/src/androidTest/AndroidManifest.xml @@ -0,0 +1,34 @@ + + + + + + + + + + + + + + diff --git a/extensions/cronet/src/androidTest/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceContractTest.java b/extensions/cronet/src/androidTest/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceContractTest.java new file mode 100644 index 00000000000..967c894c397 --- /dev/null +++ b/extensions/cronet/src/androidTest/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceContractTest.java @@ -0,0 +1,92 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.ext.cronet; + +import static com.google.common.truth.Truth.assertThat; + +import android.net.Uri; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.HttpDataSourceTestEnv; +import com.google.android.exoplayer2.upstream.DataSource; +import com.google.android.exoplayer2.upstream.HttpDataSource; +import com.google.common.collect.ImmutableList; +import java.util.Map; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import org.junit.After; +import org.junit.Rule; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link CronetDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class CronetDataSourceContractTest extends DataSourceContractTest { + + @Rule public HttpDataSourceTestEnv httpDataSourceTestEnv = new HttpDataSourceTestEnv(); + private final ExecutorService executorService = Executors.newSingleThreadExecutor(); + + @After + public void tearDown() { + executorService.shutdown(); + } + + @Override + protected DataSource createDataSource() { + CronetEngineWrapper cronetEngineWrapper = + new CronetEngineWrapper( + ApplicationProvider.getApplicationContext(), + /* userAgent= */ "test-agent", + /* preferGMSCoreCronet= */ false); + assertThat(cronetEngineWrapper.getCronetEngineSource()) + .isEqualTo(CronetEngineWrapper.SOURCE_NATIVE); + return new CronetDataSource.Factory(cronetEngineWrapper, executorService) + .setFallbackFactory(new InvalidDataSourceFactory()) + .createDataSource(); + } + + @Override + protected ImmutableList getTestResources() { + return httpDataSourceTestEnv.getServedResources(); + } + + @Override + protected Uri getNotFoundUri() { + return Uri.parse(httpDataSourceTestEnv.getNonexistentUrl()); + } + + /** + * An {@link HttpDataSource.Factory} that throws {@link UnsupportedOperationException} on every + * interaction. + */ + private static class InvalidDataSourceFactory implements HttpDataSource.Factory { + @Override + public HttpDataSource createDataSource() { + throw new UnsupportedOperationException(); + } + + @Override + public HttpDataSource.RequestProperties getDefaultRequestProperties() { + throw new UnsupportedOperationException(); + } + + @Override + public HttpDataSource.Factory setDefaultRequestProperties( + Map defaultRequestProperties) { + throw new UnsupportedOperationException(); + } + } +} diff --git a/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetDataSource.java b/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetDataSource.java index b215b6d763d..2726b00c734 100644 --- a/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetDataSource.java +++ b/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetDataSource.java @@ -17,7 +17,6 @@ import static com.google.android.exoplayer2.util.Util.castNonNull; import static java.lang.Math.max; -import static java.lang.Math.min; import android.net.Uri; import android.text.TextUtils; @@ -25,15 +24,19 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.upstream.BaseDataSource; +import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DataSourceException; import com.google.android.exoplayer2.upstream.DataSpec; +import com.google.android.exoplayer2.upstream.DefaultHttpDataSource; import com.google.android.exoplayer2.upstream.HttpDataSource; +import com.google.android.exoplayer2.upstream.TransferListener; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Clock; import com.google.android.exoplayer2.util.ConditionVariable; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.Util; import com.google.common.base.Predicate; +import com.google.common.primitives.Ints; import java.io.IOException; import java.io.InterruptedIOException; import java.net.SocketTimeoutException; @@ -65,6 +68,204 @@ */ public class CronetDataSource extends BaseDataSource implements HttpDataSource { + static { + ExoPlayerLibraryInfo.registerModule("goog.exo.cronet"); + } + + /** {@link DataSource.Factory} for {@link CronetDataSource} instances. */ + public static final class Factory implements HttpDataSource.Factory { + + private final CronetEngineWrapper cronetEngineWrapper; + private final Executor executor; + private final RequestProperties defaultRequestProperties; + private final DefaultHttpDataSource.Factory internalFallbackFactory; + + @Nullable private HttpDataSource.Factory fallbackFactory; + @Nullable private Predicate contentTypePredicate; + @Nullable private TransferListener transferListener; + @Nullable private String userAgent; + private int connectTimeoutMs; + private int readTimeoutMs; + private boolean resetTimeoutOnRedirects; + private boolean handleSetCookieRequests; + + /** + * Creates an instance. + * + * @param cronetEngineWrapper A {@link CronetEngineWrapper}. + * @param executor The {@link java.util.concurrent.Executor} that will handle responses. This + * may be a direct executor (i.e. executes tasks on the calling thread) in order to avoid a + * thread hop from Cronet's internal network thread to the response handling thread. + * However, to avoid slowing down overall network performance, care must be taken to make + * sure response handling is a fast operation when using a direct executor. + */ + public Factory(CronetEngineWrapper cronetEngineWrapper, Executor executor) { + this.cronetEngineWrapper = cronetEngineWrapper; + this.executor = executor; + defaultRequestProperties = new RequestProperties(); + internalFallbackFactory = new DefaultHttpDataSource.Factory(); + connectTimeoutMs = DEFAULT_CONNECT_TIMEOUT_MILLIS; + readTimeoutMs = DEFAULT_READ_TIMEOUT_MILLIS; + } + + /** @deprecated Use {@link #setDefaultRequestProperties(Map)} instead. */ + @Deprecated + @Override + public final RequestProperties getDefaultRequestProperties() { + return defaultRequestProperties; + } + + @Override + public final Factory setDefaultRequestProperties(Map defaultRequestProperties) { + this.defaultRequestProperties.clearAndSet(defaultRequestProperties); + internalFallbackFactory.setDefaultRequestProperties(defaultRequestProperties); + return this; + } + + /** + * Sets the user agent that will be used. + * + *

The default is {@code null}, which causes the default user agent of the underlying {@link + * CronetEngine} to be used. + * + * @param userAgent The user agent that will be used, or {@code null} to use the default user + * agent of the underlying {@link CronetEngine}. + * @return This factory. + */ + public Factory setUserAgent(@Nullable String userAgent) { + this.userAgent = userAgent; + internalFallbackFactory.setUserAgent(userAgent); + return this; + } + + /** + * Sets the connect timeout, in milliseconds. + * + *

The default is {@link CronetDataSource#DEFAULT_CONNECT_TIMEOUT_MILLIS}. + * + * @param connectTimeoutMs The connect timeout, in milliseconds, that will be used. + * @return This factory. + */ + public Factory setConnectionTimeoutMs(int connectTimeoutMs) { + this.connectTimeoutMs = connectTimeoutMs; + internalFallbackFactory.setConnectTimeoutMs(connectTimeoutMs); + return this; + } + + /** + * Sets whether the connect timeout is reset when a redirect occurs. + * + *

The default is {@code false}. + * + * @param resetTimeoutOnRedirects Whether the connect timeout is reset when a redirect occurs. + * @return This factory. + */ + public Factory setResetTimeoutOnRedirects(boolean resetTimeoutOnRedirects) { + this.resetTimeoutOnRedirects = resetTimeoutOnRedirects; + return this; + } + + /** + * Sets whether "Set-Cookie" requests on redirect should be forwarded to the redirect url in the + * "Cookie" header. + * + *

The default is {@code false}. + * + * @param handleSetCookieRequests Whether "Set-Cookie" requests on redirect should be forwarded + * to the redirect url in the "Cookie" header. + * @return This factory. + */ + public Factory setHandleSetCookieRequests(boolean handleSetCookieRequests) { + this.handleSetCookieRequests = handleSetCookieRequests; + return this; + } + + /** + * Sets the read timeout, in milliseconds. + * + *

The default is {@link CronetDataSource#DEFAULT_READ_TIMEOUT_MILLIS}. + * + * @param readTimeoutMs The connect timeout, in milliseconds, that will be used. + * @return This factory. + */ + public Factory setReadTimeoutMs(int readTimeoutMs) { + this.readTimeoutMs = readTimeoutMs; + internalFallbackFactory.setReadTimeoutMs(readTimeoutMs); + return this; + } + + /** + * Sets a content type {@link Predicate}. If a content type is rejected by the predicate then a + * {@link HttpDataSource.InvalidContentTypeException} is thrown from {@link #open(DataSpec)}. + * + *

The default is {@code null}. + * + * @param contentTypePredicate The content type {@link Predicate}, or {@code null} to clear a + * predicate that was previously set. + * @return This factory. + */ + public Factory setContentTypePredicate(@Nullable Predicate contentTypePredicate) { + this.contentTypePredicate = contentTypePredicate; + internalFallbackFactory.setContentTypePredicate(contentTypePredicate); + return this; + } + + /** + * Sets the {@link TransferListener} that will be used. + * + *

The default is {@code null}. + * + *

See {@link DataSource#addTransferListener(TransferListener)}. + * + * @param transferListener The listener that will be used. + * @return This factory. + */ + public Factory setTransferListener(@Nullable TransferListener transferListener) { + this.transferListener = transferListener; + internalFallbackFactory.setTransferListener(transferListener); + return this; + } + + /** + * Sets the fallback {@link HttpDataSource.Factory} that is used as a fallback if the {@link + * CronetEngineWrapper} fails to provide a {@link CronetEngine}. + * + *

By default a {@link DefaultHttpDataSource} is used as fallback factory. + * + * @param fallbackFactory The fallback factory that will be used. + * @return This factory. + */ + public Factory setFallbackFactory(@Nullable HttpDataSource.Factory fallbackFactory) { + this.fallbackFactory = fallbackFactory; + return this; + } + + @Override + public HttpDataSource createDataSource() { + @Nullable CronetEngine cronetEngine = cronetEngineWrapper.getCronetEngine(); + if (cronetEngine == null) { + return (fallbackFactory != null) + ? fallbackFactory.createDataSource() + : internalFallbackFactory.createDataSource(); + } + CronetDataSource dataSource = + new CronetDataSource( + cronetEngine, + executor, + connectTimeoutMs, + readTimeoutMs, + resetTimeoutOnRedirects, + handleSetCookieRequests, + userAgent, + defaultRequestProperties, + contentTypePredicate); + if (transferListener != null) { + dataSource.addTransferListener(transferListener); + } + return dataSource; + } + } + /** * Thrown when an error is encountered when trying to open a {@link CronetDataSource}. */ @@ -85,20 +286,11 @@ public OpenException(String errorMessage, DataSpec dataSpec, int cronetConnectio super(errorMessage, dataSpec, TYPE_OPEN); this.cronetConnectionStatus = cronetConnectionStatus; } - } - static { - ExoPlayerLibraryInfo.registerModule("goog.exo.cronet"); - } - - /** - * The default connection timeout, in milliseconds. - */ + /** The default connection timeout, in milliseconds. */ public static final int DEFAULT_CONNECT_TIMEOUT_MILLIS = 8 * 1000; - /** - * The default read timeout, in milliseconds. - */ + /** The default read timeout, in milliseconds. */ public static final int DEFAULT_READ_TIMEOUT_MILLIS = 8 * 1000; /* package */ final UrlRequest.Callback urlRequestCallback; @@ -119,6 +311,7 @@ public OpenException(String errorMessage, DataSpec dataSpec, int cronetConnectio private final int readTimeoutMs; private final boolean resetTimeoutOnRedirects; private final boolean handleSetCookieRequests; + @Nullable private final String userAgent; @Nullable private final RequestProperties defaultRequestProperties; private final RequestProperties requestProperties; private final ConditionVariable operation; @@ -149,16 +342,9 @@ public OpenException(String errorMessage, DataSpec dataSpec, int cronetConnectio private volatile long currentConnectTimeoutMs; - /** - * Creates an instance. - * - * @param cronetEngine A CronetEngine. - * @param executor The {@link java.util.concurrent.Executor} that will handle responses. This may - * be a direct executor (i.e. executes tasks on the calling thread) in order to avoid a thread - * hop from Cronet's internal network thread to the response handling thread. However, to - * avoid slowing down overall network performance, care must be taken to make sure response - * handling is a fast operation when using a direct executor. - */ + /** @deprecated Use {@link CronetDataSource.Factory} instead. */ + @SuppressWarnings("deprecation") + @Deprecated public CronetDataSource(CronetEngine cronetEngine, Executor executor) { this( cronetEngine, @@ -169,21 +355,8 @@ public CronetDataSource(CronetEngine cronetEngine, Executor executor) { /* defaultRequestProperties= */ null); } - /** - * Creates an instance. - * - * @param cronetEngine A CronetEngine. - * @param executor The {@link java.util.concurrent.Executor} that will handle responses. This may - * be a direct executor (i.e. executes tasks on the calling thread) in order to avoid a thread - * hop from Cronet's internal network thread to the response handling thread. However, to - * avoid slowing down overall network performance, care must be taken to make sure response - * handling is a fast operation when using a direct executor. - * @param connectTimeoutMs The connection timeout, in milliseconds. - * @param readTimeoutMs The read timeout, in milliseconds. - * @param resetTimeoutOnRedirects Whether the connect timeout is reset when a redirect occurs. - * @param defaultRequestProperties Optional default {@link RequestProperties} to be sent to the - * server as HTTP headers on every request. - */ + /** @deprecated Use {@link CronetDataSource.Factory} instead. */ + @Deprecated public CronetDataSource( CronetEngine cronetEngine, Executor executor, @@ -197,28 +370,14 @@ public CronetDataSource( connectTimeoutMs, readTimeoutMs, resetTimeoutOnRedirects, - Clock.DEFAULT, + /* handleSetCookieRequests= */ false, + /* userAgent= */ null, defaultRequestProperties, - /* handleSetCookieRequests= */ false); + /* contentTypePredicate= */ null); } - /** - * Creates an instance. - * - * @param cronetEngine A CronetEngine. - * @param executor The {@link java.util.concurrent.Executor} that will handle responses. This may - * be a direct executor (i.e. executes tasks on the calling thread) in order to avoid a thread - * hop from Cronet's internal network thread to the response handling thread. However, to - * avoid slowing down overall network performance, care must be taken to make sure response - * handling is a fast operation when using a direct executor. - * @param connectTimeoutMs The connection timeout, in milliseconds. - * @param readTimeoutMs The read timeout, in milliseconds. - * @param resetTimeoutOnRedirects Whether the connect timeout is reset when a redirect occurs. - * @param defaultRequestProperties Optional default {@link RequestProperties} to be sent to the - * server as HTTP headers on every request. - * @param handleSetCookieRequests Whether "Set-Cookie" requests on redirect should be forwarded to - * the redirect url in the "Cookie" header. - */ + /** @deprecated Use {@link CronetDataSource.Factory} instead. */ + @Deprecated public CronetDataSource( CronetEngine cronetEngine, Executor executor, @@ -233,26 +392,13 @@ public CronetDataSource( connectTimeoutMs, readTimeoutMs, resetTimeoutOnRedirects, - Clock.DEFAULT, + handleSetCookieRequests, + /* userAgent= */ null, defaultRequestProperties, - handleSetCookieRequests); + /* contentTypePredicate= */ null); } - /** - * Creates an instance. - * - * @param cronetEngine A CronetEngine. - * @param executor The {@link java.util.concurrent.Executor} that will handle responses. This may - * be a direct executor (i.e. executes tasks on the calling thread) in order to avoid a thread - * hop from Cronet's internal network thread to the response handling thread. However, to - * avoid slowing down overall network performance, care must be taken to make sure response - * handling is a fast operation when using a direct executor. - * @param contentTypePredicate An optional {@link Predicate}. If a content type is rejected by the - * predicate then an {@link InvalidContentTypeException} is thrown from {@link - * #open(DataSpec)}. - * @deprecated Use {@link #CronetDataSource(CronetEngine, Executor)} and {@link - * #setContentTypePredicate(Predicate)}. - */ + /** @deprecated Use {@link CronetDataSource.Factory} instead. */ @SuppressWarnings("deprecation") @Deprecated public CronetDataSource( @@ -269,26 +415,7 @@ public CronetDataSource( /* defaultRequestProperties= */ null); } - /** - * Creates an instance. - * - * @param cronetEngine A CronetEngine. - * @param executor The {@link java.util.concurrent.Executor} that will handle responses. This may - * be a direct executor (i.e. executes tasks on the calling thread) in order to avoid a thread - * hop from Cronet's internal network thread to the response handling thread. However, to - * avoid slowing down overall network performance, care must be taken to make sure response - * handling is a fast operation when using a direct executor. - * @param contentTypePredicate An optional {@link Predicate}. If a content type is rejected by the - * predicate then an {@link InvalidContentTypeException} is thrown from {@link - * #open(DataSpec)}. - * @param connectTimeoutMs The connection timeout, in milliseconds. - * @param readTimeoutMs The read timeout, in milliseconds. - * @param resetTimeoutOnRedirects Whether the connect timeout is reset when a redirect occurs. - * @param defaultRequestProperties Optional default {@link RequestProperties} to be sent to the - * server as HTTP headers on every request. - * @deprecated Use {@link #CronetDataSource(CronetEngine, Executor, int, int, boolean, - * RequestProperties)} and {@link #setContentTypePredicate(Predicate)}. - */ + /** @deprecated Use {@link CronetDataSource.Factory} instead. */ @SuppressWarnings("deprecation") @Deprecated public CronetDataSource( @@ -310,28 +437,7 @@ public CronetDataSource( /* handleSetCookieRequests= */ false); } - /** - * Creates an instance. - * - * @param cronetEngine A CronetEngine. - * @param executor The {@link java.util.concurrent.Executor} that will handle responses. This may - * be a direct executor (i.e. executes tasks on the calling thread) in order to avoid a thread - * hop from Cronet's internal network thread to the response handling thread. However, to - * avoid slowing down overall network performance, care must be taken to make sure response - * handling is a fast operation when using a direct executor. - * @param contentTypePredicate An optional {@link Predicate}. If a content type is rejected by the - * predicate then an {@link InvalidContentTypeException} is thrown from {@link - * #open(DataSpec)}. - * @param connectTimeoutMs The connection timeout, in milliseconds. - * @param readTimeoutMs The read timeout, in milliseconds. - * @param resetTimeoutOnRedirects Whether the connect timeout is reset when a redirect occurs. - * @param defaultRequestProperties Optional default {@link RequestProperties} to be sent to the - * server as HTTP headers on every request. - * @param handleSetCookieRequests Whether "Set-Cookie" requests on redirect should be forwarded to - * the redirect url in the "Cookie" header. - * @deprecated Use {@link #CronetDataSource(CronetEngine, Executor, int, int, boolean, - * RequestProperties, boolean)} and {@link #setContentTypePredicate(Predicate)}. - */ + /** @deprecated Use {@link CronetDataSource.Factory} instead. */ @Deprecated public CronetDataSource( CronetEngine cronetEngine, @@ -348,31 +454,34 @@ public CronetDataSource( connectTimeoutMs, readTimeoutMs, resetTimeoutOnRedirects, - Clock.DEFAULT, + handleSetCookieRequests, + /* userAgent= */ null, defaultRequestProperties, - handleSetCookieRequests); - this.contentTypePredicate = contentTypePredicate; + contentTypePredicate); } - /* package */ CronetDataSource( + private CronetDataSource( CronetEngine cronetEngine, Executor executor, int connectTimeoutMs, int readTimeoutMs, boolean resetTimeoutOnRedirects, - Clock clock, + boolean handleSetCookieRequests, + @Nullable String userAgent, @Nullable RequestProperties defaultRequestProperties, - boolean handleSetCookieRequests) { + @Nullable Predicate contentTypePredicate) { super(/* isNetwork= */ true); - this.urlRequestCallback = new UrlRequestCallback(); this.cronetEngine = Assertions.checkNotNull(cronetEngine); this.executor = Assertions.checkNotNull(executor); this.connectTimeoutMs = connectTimeoutMs; this.readTimeoutMs = readTimeoutMs; this.resetTimeoutOnRedirects = resetTimeoutOnRedirects; - this.clock = Assertions.checkNotNull(clock); - this.defaultRequestProperties = defaultRequestProperties; this.handleSetCookieRequests = handleSetCookieRequests; + this.userAgent = userAgent; + this.defaultRequestProperties = defaultRequestProperties; + this.contentTypePredicate = contentTypePredicate; + clock = Clock.DEFAULT; + urlRequestCallback = new UrlRequestCallback(); requestProperties = new RequestProperties(); operation = new ConditionVariable(); } @@ -384,6 +493,7 @@ public CronetDataSource( * @param contentTypePredicate The content type {@link Predicate}, or {@code null} to clear a * predicate that was previously set. */ + @Deprecated public void setContentTypePredicate(@Nullable Predicate contentTypePredicate) { this.contentTypePredicate = contentTypePredicate; } @@ -464,19 +574,11 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { UrlResponseInfo responseInfo = Assertions.checkNotNull(this.responseInfo); int responseCode = responseInfo.getHttpStatusCode(); if (responseCode < 200 || responseCode > 299) { - byte[] responseBody = Util.EMPTY_BYTE_ARRAY; - ByteBuffer readBuffer = getOrCreateReadBuffer(); - while (!readBuffer.hasRemaining()) { - operation.close(); - readBuffer.clear(); - readInternal(readBuffer); - if (finished) { - break; - } - readBuffer.flip(); - int existingResponseBodyEnd = responseBody.length; - responseBody = Arrays.copyOf(responseBody, responseBody.length + readBuffer.remaining()); - readBuffer.get(responseBody, existingResponseBodyEnd, readBuffer.remaining()); + byte[] responseBody; + try { + responseBody = readResponseBody(); + } catch (HttpDataSourceException e) { + responseBody = Util.EMPTY_BYTE_ARRAY; } InvalidResponseCodeException exception = @@ -553,14 +655,21 @@ public int read(byte[] buffer, int offset, int readLength) throws HttpDataSource readBuffer.flip(); Assertions.checkState(readBuffer.hasRemaining()); if (bytesToSkip > 0) { - int bytesSkipped = (int) min(readBuffer.remaining(), bytesToSkip); + int bytesSkipped = (int) Math.min(readBuffer.remaining(), bytesToSkip); readBuffer.position(readBuffer.position() + bytesSkipped); bytesToSkip -= bytesSkipped; } } } - int bytesRead = min(readBuffer.remaining(), readLength); + // Ensure we read up to bytesRemaining, in case this was a Range request with finite end, but + // the server does not support Range requests and transmitted the entire resource. + int bytesRead = + Ints.min( + bytesRemaining != C.LENGTH_UNSET ? (int) bytesRemaining : Integer.MAX_VALUE, + readBuffer.remaining(), + readLength); + readBuffer.get(buffer, offset, bytesRead); if (bytesRemaining != C.LENGTH_UNSET) { @@ -707,9 +816,7 @@ protected UrlResponseInfo getCurrentUrlResponseInfo() { return responseInfo; } - // Internal methods. - - private UrlRequest.Builder buildRequestBuilder(DataSpec dataSpec) throws IOException { + protected UrlRequest.Builder buildRequestBuilder(DataSpec dataSpec) throws IOException { UrlRequest.Builder requestBuilder = cronetEngine .newUrlRequestBuilder(dataSpec.uri.toString(), urlRequestCallback, executor) @@ -744,6 +851,9 @@ private UrlRequest.Builder buildRequestBuilder(DataSpec dataSpec) throws IOExcep } requestBuilder.addHeader("Range", rangeValue.toString()); } + if (userAgent != null) { + requestBuilder.addHeader("User-Agent", userAgent); + } // TODO: Uncomment when https://bugs.chromium.org/p/chromium/issues/detail?id=711810 is fixed // (adjusting the code as necessary). // Force identity encoding unless gzip is allowed. @@ -759,6 +869,8 @@ private UrlRequest.Builder buildRequestBuilder(DataSpec dataSpec) throws IOExcep return requestBuilder; } + // Internal methods. + private boolean blockUntilConnectTimeout() throws InterruptedException { long now = clock.elapsedRealtime(); boolean opened = false; @@ -773,6 +885,29 @@ private void resetConnectTimeout() { currentConnectTimeoutMs = clock.elapsedRealtime() + connectTimeoutMs; } + /** + * Reads the whole response body. + * + * @return The response body. + * @throws HttpDataSourceException If an error occurs reading from the source. + */ + private byte[] readResponseBody() throws HttpDataSourceException { + byte[] responseBody = Util.EMPTY_BYTE_ARRAY; + ByteBuffer readBuffer = getOrCreateReadBuffer(); + while (!finished) { + operation.close(); + readBuffer.clear(); + readInternal(readBuffer); + readBuffer.flip(); + if (readBuffer.remaining() > 0) { + int existingResponseBodyEnd = responseBody.length; + responseBody = Arrays.copyOf(responseBody, responseBody.length + readBuffer.remaining()); + readBuffer.get(responseBody, existingResponseBodyEnd, readBuffer.remaining()); + } + } + return responseBody; + } + /** * Reads up to {@code buffer.remaining()} bytes of data from {@code currentUrlRequest} and stores * them into {@code buffer}. If there is an error and {@code buffer == readBuffer}, then it resets @@ -911,7 +1046,7 @@ private static boolean isEmpty(@Nullable List list) { // Copy as much as possible from the src buffer into dst buffer. // Returns the number of bytes copied. private static int copyByteBuffer(ByteBuffer src, ByteBuffer dst) { - int remaining = min(src.remaining(), dst.remaining()); + int remaining = Math.min(src.remaining(), dst.remaining()); int limit = src.limit(); src.limit(src.position() + remaining); dst.put(src); diff --git a/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceFactory.java b/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceFactory.java index 85c9d09a791..df3e9549e5a 100644 --- a/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceFactory.java +++ b/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceFactory.java @@ -15,20 +15,17 @@ */ package com.google.android.exoplayer2.ext.cronet; -import static com.google.android.exoplayer2.ExoPlayerLibraryInfo.DEFAULT_USER_AGENT; import androidx.annotation.Nullable; -import com.google.android.exoplayer2.upstream.DefaultHttpDataSourceFactory; +import com.google.android.exoplayer2.upstream.DefaultHttpDataSource; import com.google.android.exoplayer2.upstream.HttpDataSource; import com.google.android.exoplayer2.upstream.HttpDataSource.BaseFactory; -import com.google.android.exoplayer2.upstream.HttpDataSource.Factory; import com.google.android.exoplayer2.upstream.TransferListener; import java.util.concurrent.Executor; import org.chromium.net.CronetEngine; -/** - * A {@link Factory} that produces {@link CronetDataSource}. - */ +/** @deprecated Use {@link CronetDataSource.Factory} instead. */ +@Deprecated public final class CronetDataSourceFactory extends BaseFactory { /** @@ -83,7 +80,7 @@ public CronetDataSourceFactory( * Creates an instance. * *

If the {@link CronetEngineWrapper} fails to provide a {@link CronetEngine}, a {@link - * DefaultHttpDataSourceFactory} will be used instead. + * DefaultHttpDataSource.Factory} will be used instead. * *

Sets {@link CronetDataSource#DEFAULT_CONNECT_TIMEOUT_MILLIS} as the connection timeout, * {@link CronetDataSource#DEFAULT_READ_TIMEOUT_MILLIS} as the read timeout. @@ -92,24 +89,26 @@ public CronetDataSourceFactory( * @param executor The {@link java.util.concurrent.Executor} that will perform the requests. */ public CronetDataSourceFactory(CronetEngineWrapper cronetEngineWrapper, Executor executor) { - this(cronetEngineWrapper, executor, DEFAULT_USER_AGENT); + this(cronetEngineWrapper, executor, /* userAgent= */ (String) null); } /** * Creates an instance. * *

If the {@link CronetEngineWrapper} fails to provide a {@link CronetEngine}, a {@link - * DefaultHttpDataSourceFactory} will be used instead. + * DefaultHttpDataSource.Factory} will be used instead. * *

Sets {@link CronetDataSource#DEFAULT_CONNECT_TIMEOUT_MILLIS} as the connection timeout, * {@link CronetDataSource#DEFAULT_READ_TIMEOUT_MILLIS} as the read timeout. * * @param cronetEngineWrapper A {@link CronetEngineWrapper}. * @param executor The {@link java.util.concurrent.Executor} that will perform the requests. - * @param userAgent A user agent used to create a fallback HttpDataSource if needed. + * @param userAgent The user agent that will be used by the fallback {@link HttpDataSource} if + * needed, or {@code null} for the fallback to use the default user agent of the underlying + * platform. */ public CronetDataSourceFactory( - CronetEngineWrapper cronetEngineWrapper, Executor executor, String userAgent) { + CronetEngineWrapper cronetEngineWrapper, Executor executor, @Nullable String userAgent) { this( cronetEngineWrapper, executor, @@ -117,26 +116,23 @@ public CronetDataSourceFactory( DEFAULT_CONNECT_TIMEOUT_MILLIS, DEFAULT_READ_TIMEOUT_MILLIS, false, - new DefaultHttpDataSourceFactory( - userAgent, - /* listener= */ null, - DEFAULT_CONNECT_TIMEOUT_MILLIS, - DEFAULT_READ_TIMEOUT_MILLIS, - false)); + new DefaultHttpDataSource.Factory().setUserAgent(userAgent)); } /** * Creates an instance. * *

If the {@link CronetEngineWrapper} fails to provide a {@link CronetEngine}, a {@link - * DefaultHttpDataSourceFactory} will be used instead. + * DefaultHttpDataSource.Factory} will be used instead. * * @param cronetEngineWrapper A {@link CronetEngineWrapper}. * @param executor The {@link java.util.concurrent.Executor} that will perform the requests. * @param connectTimeoutMs The connection timeout, in milliseconds. * @param readTimeoutMs The read timeout, in milliseconds. * @param resetTimeoutOnRedirects Whether the connect timeout is reset when a redirect occurs. - * @param userAgent A user agent used to create a fallback HttpDataSource if needed. + * @param userAgent The user agent that will be used by the fallback {@link HttpDataSource} if + * needed, or {@code null} for the fallback to use the default user agent of the underlying + * platform. */ public CronetDataSourceFactory( CronetEngineWrapper cronetEngineWrapper, @@ -144,20 +140,18 @@ public CronetDataSourceFactory( int connectTimeoutMs, int readTimeoutMs, boolean resetTimeoutOnRedirects, - String userAgent) { + @Nullable String userAgent) { this( cronetEngineWrapper, executor, /* transferListener= */ null, - DEFAULT_CONNECT_TIMEOUT_MILLIS, - DEFAULT_READ_TIMEOUT_MILLIS, + connectTimeoutMs, + readTimeoutMs, resetTimeoutOnRedirects, - new DefaultHttpDataSourceFactory( - userAgent, - /* listener= */ null, - connectTimeoutMs, - readTimeoutMs, - resetTimeoutOnRedirects)); + new DefaultHttpDataSource.Factory() + .setUserAgent(userAgent) + .setConnectTimeoutMs(connectTimeoutMs) + .setReadTimeoutMs(readTimeoutMs)); } /** @@ -225,7 +219,7 @@ public CronetDataSourceFactory( * Creates an instance. * *

If the {@link CronetEngineWrapper} fails to provide a {@link CronetEngine}, a {@link - * DefaultHttpDataSourceFactory} will be used instead. + * DefaultHttpDataSource.Factory} will be used instead. * *

Sets {@link CronetDataSource#DEFAULT_CONNECT_TIMEOUT_MILLIS} as the connection timeout, * {@link CronetDataSource#DEFAULT_READ_TIMEOUT_MILLIS} as the read timeout. @@ -238,14 +232,14 @@ public CronetDataSourceFactory( CronetEngineWrapper cronetEngineWrapper, Executor executor, @Nullable TransferListener transferListener) { - this(cronetEngineWrapper, executor, transferListener, DEFAULT_USER_AGENT); + this(cronetEngineWrapper, executor, transferListener, /* userAgent= */ (String) null); } /** * Creates an instance. * *

If the {@link CronetEngineWrapper} fails to provide a {@link CronetEngine}, a {@link - * DefaultHttpDataSourceFactory} will be used instead. + * DefaultHttpDataSource.Factory} will be used instead. * *

Sets {@link CronetDataSource#DEFAULT_CONNECT_TIMEOUT_MILLIS} as the connection timeout, * {@link CronetDataSource#DEFAULT_READ_TIMEOUT_MILLIS} as the read timeout. @@ -253,13 +247,15 @@ public CronetDataSourceFactory( * @param cronetEngineWrapper A {@link CronetEngineWrapper}. * @param executor The {@link java.util.concurrent.Executor} that will perform the requests. * @param transferListener An optional listener. - * @param userAgent A user agent used to create a fallback HttpDataSource if needed. + * @param userAgent The user agent that will be used by the fallback {@link HttpDataSource} if + * needed, or {@code null} for the fallback to use the default user agent of the underlying + * platform. */ public CronetDataSourceFactory( CronetEngineWrapper cronetEngineWrapper, Executor executor, @Nullable TransferListener transferListener, - String userAgent) { + @Nullable String userAgent) { this( cronetEngineWrapper, executor, @@ -267,19 +263,16 @@ public CronetDataSourceFactory( DEFAULT_CONNECT_TIMEOUT_MILLIS, DEFAULT_READ_TIMEOUT_MILLIS, false, - new DefaultHttpDataSourceFactory( - userAgent, - transferListener, - DEFAULT_CONNECT_TIMEOUT_MILLIS, - DEFAULT_READ_TIMEOUT_MILLIS, - false)); + new DefaultHttpDataSource.Factory() + .setUserAgent(userAgent) + .setTransferListener(transferListener)); } /** * Creates an instance. * *

If the {@link CronetEngineWrapper} fails to provide a {@link CronetEngine}, a {@link - * DefaultHttpDataSourceFactory} will be used instead. + * DefaultHttpDataSource.Factory} will be used instead. * * @param cronetEngineWrapper A {@link CronetEngineWrapper}. * @param executor The {@link java.util.concurrent.Executor} that will perform the requests. @@ -287,7 +280,9 @@ public CronetDataSourceFactory( * @param connectTimeoutMs The connection timeout, in milliseconds. * @param readTimeoutMs The read timeout, in milliseconds. * @param resetTimeoutOnRedirects Whether the connect timeout is reset when a redirect occurs. - * @param userAgent A user agent used to create a fallback HttpDataSource if needed. + * @param userAgent The user agent that will be used by the fallback {@link HttpDataSource} if + * needed, or {@code null} for the fallback to use the default user agent of the underlying + * platform. */ public CronetDataSourceFactory( CronetEngineWrapper cronetEngineWrapper, @@ -296,16 +291,19 @@ public CronetDataSourceFactory( int connectTimeoutMs, int readTimeoutMs, boolean resetTimeoutOnRedirects, - String userAgent) { + @Nullable String userAgent) { this( cronetEngineWrapper, executor, transferListener, - DEFAULT_CONNECT_TIMEOUT_MILLIS, - DEFAULT_READ_TIMEOUT_MILLIS, + connectTimeoutMs, + readTimeoutMs, resetTimeoutOnRedirects, - new DefaultHttpDataSourceFactory( - userAgent, transferListener, connectTimeoutMs, readTimeoutMs, resetTimeoutOnRedirects)); + new DefaultHttpDataSource.Factory() + .setUserAgent(userAgent) + .setTransferListener(transferListener) + .setConnectTimeoutMs(connectTimeoutMs) + .setReadTimeoutMs(readTimeoutMs)); } /** @@ -343,7 +341,7 @@ public CronetDataSourceFactory( @Override protected HttpDataSource createDataSourceInternal(HttpDataSource.RequestProperties defaultRequestProperties) { - CronetEngine cronetEngine = cronetEngineWrapper.getCronetEngine(); + @Nullable CronetEngine cronetEngine = cronetEngineWrapper.getCronetEngine(); if (cronetEngine == null) { return fallbackFactory.createDataSource(); } diff --git a/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetEngineWrapper.java b/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetEngineWrapper.java index 9f709b14d03..d9332342e35 100644 --- a/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetEngineWrapper.java +++ b/extensions/cronet/src/main/java/com/google/android/exoplayer2/ext/cronet/CronetEngineWrapper.java @@ -73,25 +73,29 @@ public final class CronetEngineWrapper { public static final int SOURCE_UNAVAILABLE = 4; /** - * Creates a wrapper for a {@link CronetEngine} which automatically selects the most suitable - * {@link CronetProvider}. Sets wrapper to prefer natively bundled Cronet over GMSCore Cronet - * if both are available. + * Creates a wrapper for a {@link CronetEngine} built using the most suitable {@link + * CronetProvider}. When natively bundled Cronet and GMSCore Cronet are both available, the + * natively bundled provider is preferred. * * @param context A context. */ public CronetEngineWrapper(Context context) { - this(context, false); + this(context, /* userAgent= */ null, /* preferGMSCoreCronet= */ false); } /** - * Creates a wrapper for a {@link CronetEngine} which automatically selects the most suitable - * {@link CronetProvider} based on user preference. + * Creates a wrapper for a {@link CronetEngine} built using the most suitable {@link + * CronetProvider}. When natively bundled Cronet and GMSCore Cronet are both available, {@code + * preferGMSCoreCronet} determines which is preferred. * * @param context A context. + * @param userAgent A default user agent, or {@code null} to use a default user agent of the + * {@link CronetEngine}. * @param preferGMSCoreCronet Whether Cronet from GMSCore should be preferred over natively * bundled Cronet if both are available. */ - public CronetEngineWrapper(Context context, boolean preferGMSCoreCronet) { + public CronetEngineWrapper( + Context context, @Nullable String userAgent, boolean preferGMSCoreCronet) { CronetEngine cronetEngine = null; @CronetEngineSource int cronetEngineSource = SOURCE_UNAVAILABLE; List cronetProviders = new ArrayList<>(CronetProvider.getAllProviders(context)); @@ -108,7 +112,11 @@ public CronetEngineWrapper(Context context, boolean preferGMSCoreCronet) { for (int i = 0; i < cronetProviders.size() && cronetEngine == null; i++) { String providerName = cronetProviders.get(i).getName(); try { - cronetEngine = cronetProviders.get(i).createBuilder().build(); + CronetEngine.Builder cronetEngineBuilder = cronetProviders.get(i).createBuilder(); + if (userAgent != null) { + cronetEngineBuilder.setUserAgent(userAgent); + } + cronetEngine = cronetEngineBuilder.build(); if (providerComparator.isNativeProvider(providerName)) { cronetEngineSource = SOURCE_NATIVE; } else if (providerComparator.isGMSCoreProvider(providerName)) { @@ -133,9 +141,9 @@ public CronetEngineWrapper(Context context, boolean preferGMSCoreCronet) { } /** - * Creates a wrapper for an existing CronetEngine. + * Creates a wrapper for an existing {@link CronetEngine}. * - * @param cronetEngine An existing CronetEngine. + * @param cronetEngine The CronetEngine to wrap. */ public CronetEngineWrapper(CronetEngine cronetEngine) { this.cronetEngine = cronetEngine; diff --git a/extensions/cronet/src/test/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceTest.java b/extensions/cronet/src/test/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceTest.java index b0b80a8e128..631e1300d60 100644 --- a/extensions/cronet/src/test/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceTest.java +++ b/extensions/cronet/src/test/java/com/google/android/exoplayer2/ext/cronet/CronetDataSourceTest.java @@ -17,6 +17,7 @@ import static com.google.common.truth.Truth.assertThat; import static java.lang.Math.min; +import static java.util.concurrent.TimeUnit.SECONDS; import static org.junit.Assert.fail; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyString; @@ -32,13 +33,15 @@ import android.net.Uri; import android.os.ConditionVariable; import android.os.SystemClock; +import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.upstream.DataSpec; +import com.google.android.exoplayer2.upstream.DefaultHttpDataSource; import com.google.android.exoplayer2.upstream.HttpDataSource; import com.google.android.exoplayer2.upstream.HttpDataSource.HttpDataSourceException; +import com.google.android.exoplayer2.upstream.HttpDataSource.InvalidResponseCodeException; import com.google.android.exoplayer2.upstream.TransferListener; -import com.google.android.exoplayer2.util.Clock; import com.google.android.exoplayer2.util.Util; import java.io.IOException; import java.io.InterruptedIOException; @@ -53,8 +56,12 @@ import java.util.Map; import java.util.concurrent.CountDownLatch; import java.util.concurrent.Executor; +import java.util.concurrent.Executors; import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicReference; +import okhttp3.Headers; +import okhttp3.mockwebserver.MockResponse; +import okhttp3.mockwebserver.MockWebServer; import org.chromium.net.CronetEngine; import org.chromium.net.NetworkException; import org.chromium.net.UrlRequest; @@ -100,21 +107,18 @@ public final class CronetDataSourceTest { public void setUp() { MockitoAnnotations.initMocks(this); - HttpDataSource.RequestProperties defaultRequestProperties = - new HttpDataSource.RequestProperties(); - defaultRequestProperties.set("defaultHeader1", "defaultValue1"); - defaultRequestProperties.set("defaultHeader2", "defaultValue2"); + Map defaultRequestProperties = new HashMap<>(); + defaultRequestProperties.put("defaultHeader1", "defaultValue1"); + defaultRequestProperties.put("defaultHeader2", "defaultValue2"); dataSourceUnderTest = - new CronetDataSource( - mockCronetEngine, - mockExecutor, - TEST_CONNECT_TIMEOUT_MS, - TEST_READ_TIMEOUT_MS, - /* resetTimeoutOnRedirects= */ true, - Clock.DEFAULT, - defaultRequestProperties, - /* handleSetCookieRequests= */ false); + (CronetDataSource) + new CronetDataSource.Factory(new CronetEngineWrapper(mockCronetEngine), mockExecutor) + .setConnectionTimeoutMs(TEST_CONNECT_TIMEOUT_MS) + .setReadTimeoutMs(TEST_READ_TIMEOUT_MS) + .setResetTimeoutOnRedirects(true) + .setDefaultRequestProperties(defaultRequestProperties) + .createDataSource(); dataSourceUnderTest.addTransferListener(mockTransferListener); when(mockCronetEngine.newUrlRequestBuilder( anyString(), any(UrlRequest.Callback.class), any(Executor.class))) @@ -376,7 +380,8 @@ public void requestOpenFailDueToDnsFailure() { } @Test - public void requestOpenPropagatesFailureResponseBody() throws Exception { + public void requestOpen_withNon2xxResponseCode_throwsInvalidResponseCodeExceptionWithBody() + throws Exception { mockResponseStartSuccess(); // Use a size larger than CronetDataSource.READ_BUFFER_SIZE_BYTES int responseLength = 40 * 1024; @@ -385,8 +390,8 @@ public void requestOpenPropagatesFailureResponseBody() throws Exception { try { dataSourceUnderTest.open(testDataSpec); - fail("HttpDataSource.InvalidResponseCodeException expected"); - } catch (HttpDataSource.InvalidResponseCodeException e) { + fail("InvalidResponseCodeException expected"); + } catch (InvalidResponseCodeException e) { assertThat(e.responseBody).isEqualTo(buildTestDataArray(0, responseLength)); // Check for connection not automatically closed. verify(mockUrlRequest, never()).cancel(); @@ -395,6 +400,26 @@ public void requestOpenPropagatesFailureResponseBody() throws Exception { } } + @Test + public void + requestOpen_withNon2xxResponseCode_andRequestBodyReadFailure_throwsInvalidResponseCodeExceptionWithoutBody() + throws Exception { + mockResponseStartSuccess(); + mockReadFailure(); + testUrlResponseInfo = createUrlResponseInfo(/* statusCode= */ 500); + + try { + dataSourceUnderTest.open(testDataSpec); + fail("InvalidResponseCodeException expected"); + } catch (InvalidResponseCodeException e) { + assertThat(e.responseBody).isEmpty(); + // Check for connection not automatically closed. + verify(mockUrlRequest, never()).cancel(); + verify(mockTransferListener, never()) + .onTransferStart(dataSourceUnderTest, testDataSpec, /* isNetwork= */ true); + } + } + @Test public void requestOpenValidatesContentTypePredicate() { mockResponseStartSuccess(); @@ -1134,15 +1159,13 @@ public void redirectParseAndAttachCookie_dataSourceDoesNotHandleSetCookie_follow testRedirectParseAndAttachCookie_dataSourceHandlesSetCookie_andPreservesOriginalRequestHeaders() throws HttpDataSourceException { dataSourceUnderTest = - new CronetDataSource( - mockCronetEngine, - mockExecutor, - TEST_CONNECT_TIMEOUT_MS, - TEST_READ_TIMEOUT_MS, - true, // resetTimeoutOnRedirects - Clock.DEFAULT, - null, - true); + (CronetDataSource) + new CronetDataSource.Factory(new CronetEngineWrapper(mockCronetEngine), mockExecutor) + .setConnectionTimeoutMs(TEST_CONNECT_TIMEOUT_MS) + .setReadTimeoutMs(TEST_READ_TIMEOUT_MS) + .setResetTimeoutOnRedirects(true) + .setHandleSetCookieRequests(true) + .createDataSource(); dataSourceUnderTest.addTransferListener(mockTransferListener); dataSourceUnderTest.setRequestProperty("Content-Type", TEST_CONTENT_TYPE); @@ -1164,15 +1187,13 @@ public void redirectParseAndAttachCookie_dataSourceDoesNotHandleSetCookie_follow throws HttpDataSourceException { testDataSpec = new DataSpec(Uri.parse(TEST_URL), 1000, 5000); dataSourceUnderTest = - new CronetDataSource( - mockCronetEngine, - mockExecutor, - TEST_CONNECT_TIMEOUT_MS, - TEST_READ_TIMEOUT_MS, - /* resetTimeoutOnRedirects= */ true, - Clock.DEFAULT, - /* defaultRequestProperties= */ null, - /* handleSetCookieRequests= */ true); + (CronetDataSource) + new CronetDataSource.Factory(new CronetEngineWrapper(mockCronetEngine), mockExecutor) + .setConnectionTimeoutMs(TEST_CONNECT_TIMEOUT_MS) + .setReadTimeoutMs(TEST_READ_TIMEOUT_MS) + .setResetTimeoutOnRedirects(true) + .setHandleSetCookieRequests(true) + .createDataSource(); dataSourceUnderTest.addTransferListener(mockTransferListener); dataSourceUnderTest.setRequestProperty("Content-Type", TEST_CONTENT_TYPE); @@ -1202,15 +1223,13 @@ public void redirectNoSetCookieFollowsRedirect() throws HttpDataSourceException public void redirectNoSetCookieFollowsRedirect_dataSourceHandlesSetCookie() throws HttpDataSourceException { dataSourceUnderTest = - new CronetDataSource( - mockCronetEngine, - mockExecutor, - TEST_CONNECT_TIMEOUT_MS, - TEST_READ_TIMEOUT_MS, - /* resetTimeoutOnRedirects= */ true, - Clock.DEFAULT, - /* defaultRequestProperties= */ null, - /* handleSetCookieRequests= */ true); + (CronetDataSource) + new CronetDataSource.Factory(new CronetEngineWrapper(mockCronetEngine), mockExecutor) + .setConnectionTimeoutMs(TEST_CONNECT_TIMEOUT_MS) + .setReadTimeoutMs(TEST_READ_TIMEOUT_MS) + .setResetTimeoutOnRedirects(true) + .setHandleSetCookieRequests(true) + .createDataSource(); dataSourceUnderTest.addTransferListener(mockTransferListener); mockSingleRedirectSuccess(); mockFollowRedirectSuccess(); @@ -1356,6 +1375,97 @@ public void allowDirectExecutor() throws HttpDataSourceException { verify(mockUrlRequestBuilder).allowDirectExecutor(); } + @Test + public void factorySetFallbackHttpDataSourceFactory_cronetNotAvailable_usesFallbackFactory() + throws HttpDataSourceException, InterruptedException { + MockWebServer mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse()); + CronetEngineWrapper cronetEngineWrapper = new CronetEngineWrapper((CronetEngine) null); + DefaultHttpDataSource.Factory fallbackFactory = + new DefaultHttpDataSource.Factory().setUserAgent("customFallbackFactoryUserAgent"); + HttpDataSource dataSourceUnderTest = + new CronetDataSource.Factory(cronetEngineWrapper, Executors.newSingleThreadExecutor()) + .setFallbackFactory(fallbackFactory) + .createDataSource(); + + dataSourceUnderTest.open( + new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build()); + + Headers headers = mockWebServer.takeRequest(10, SECONDS).getHeaders(); + assertThat(headers.get("user-agent")).isEqualTo("customFallbackFactoryUserAgent"); + } + + @Test + public void + factory_noFallbackFactoryCronetNotAvailable_delegateTransferListenerToInternalFallbackFactory() + throws HttpDataSourceException, InterruptedException { + MockWebServer mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse()); + CronetEngineWrapper cronetEngineWrapper = new CronetEngineWrapper((CronetEngine) null); + HttpDataSource dataSourceUnderTest = + new CronetDataSource.Factory(cronetEngineWrapper, Executors.newSingleThreadExecutor()) + .setTransferListener(mockTransferListener) + .createDataSource(); + DataSpec dataSpec = + new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build(); + + dataSourceUnderTest.open(dataSpec); + + verify(mockTransferListener) + .onTransferInitializing(eq(dataSourceUnderTest), eq(dataSpec), /* isNetwork= */ eq(true)); + verify(mockTransferListener) + .onTransferStart(eq(dataSourceUnderTest), eq(dataSpec), /* isNetwork= */ eq(true)); + } + + @Test + public void + factory_noFallbackFactoryCronetNotAvailable_delegateDefaultRequestPropertiesToInternalFallbackFactory() + throws HttpDataSourceException, InterruptedException { + MockWebServer mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse()); + CronetEngineWrapper cronetEngineWrapper = + new CronetEngineWrapper(ApplicationProvider.getApplicationContext()); + Map defaultRequestProperties = new HashMap<>(); + defaultRequestProperties.put("0", "defaultRequestProperty0"); + HttpDataSource dataSourceUnderTest = + new CronetDataSource.Factory(cronetEngineWrapper, Executors.newSingleThreadExecutor()) + .setDefaultRequestProperties(defaultRequestProperties) + .createDataSource(); + + dataSourceUnderTest.open( + new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build()); + + Headers headers = mockWebServer.takeRequest(10, SECONDS).getHeaders(); + assertThat(headers.get("0")).isEqualTo("defaultRequestProperty0"); + assertThat(dataSourceUnderTest).isInstanceOf(DefaultHttpDataSource.class); + } + + @Test + public void + factory_noFallbackFactoryCronetNotAvailable_delegateDefaultRequestPropertiesToInternalFallbackFactoryAfterCreation() + throws HttpDataSourceException, InterruptedException { + MockWebServer mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse()); + CronetEngineWrapper cronetEngineWrapper = new CronetEngineWrapper((CronetEngine) null); + Map defaultRequestProperties = new HashMap<>(); + defaultRequestProperties.put("0", "defaultRequestProperty0"); + CronetDataSource.Factory factory = + new CronetDataSource.Factory(cronetEngineWrapper, Executors.newSingleThreadExecutor()); + HttpDataSource dataSourceUnderTest = + factory.setDefaultRequestProperties(defaultRequestProperties).createDataSource(); + defaultRequestProperties.clear(); + defaultRequestProperties.put("1", "defaultRequestPropertyAfterCreation"); + factory.setDefaultRequestProperties(defaultRequestProperties); + + dataSourceUnderTest.open( + new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build()); + + Headers headers = mockWebServer.takeRequest(10, SECONDS).getHeaders(); + assertThat(headers.get("0")).isNull(); + assertThat(headers.get("1")).isEqualTo("defaultRequestPropertyAfterCreation"); + assertThat(dataSourceUnderTest).isInstanceOf(DefaultHttpDataSource.class); + } + // Helper methods. private void mockStatusResponse() { diff --git a/extensions/ffmpeg/src/main/java/com/google/android/exoplayer2/ext/ffmpeg/FfmpegAudioRenderer.java b/extensions/ffmpeg/src/main/java/com/google/android/exoplayer2/ext/ffmpeg/FfmpegAudioRenderer.java index 0718dc2c5c7..878476d1a1a 100644 --- a/extensions/ffmpeg/src/main/java/com/google/android/exoplayer2/ext/ffmpeg/FfmpegAudioRenderer.java +++ b/extensions/ffmpeg/src/main/java/com/google/android/exoplayer2/ext/ffmpeg/FfmpegAudioRenderer.java @@ -91,19 +91,19 @@ public String getName() { } @Override - @FormatSupport + @C.FormatSupport protected int supportsFormatInternal(Format format) { String mimeType = Assertions.checkNotNull(format.sampleMimeType); if (!FfmpegLibrary.isAvailable() || !MimeTypes.isAudio(mimeType)) { - return FORMAT_UNSUPPORTED_TYPE; + return C.FORMAT_UNSUPPORTED_TYPE; } else if (!FfmpegLibrary.supportsFormat(mimeType) || (!sinkSupportsFormat(format, C.ENCODING_PCM_16BIT) && !sinkSupportsFormat(format, C.ENCODING_PCM_FLOAT))) { - return FORMAT_UNSUPPORTED_SUBTYPE; + return C.FORMAT_UNSUPPORTED_SUBTYPE; } else if (format.exoMediaCryptoType != null) { - return FORMAT_UNSUPPORTED_DRM; + return C.FORMAT_UNSUPPORTED_DRM; } else { - return FORMAT_HANDLED; + return C.FORMAT_HANDLED; } } diff --git a/extensions/ffmpeg/src/main/java/com/google/android/exoplayer2/ext/ffmpeg/FfmpegLibrary.java b/extensions/ffmpeg/src/main/java/com/google/android/exoplayer2/ext/ffmpeg/FfmpegLibrary.java index a9b862b2b28..6fb47e962b4 100644 --- a/extensions/ffmpeg/src/main/java/com/google/android/exoplayer2/ext/ffmpeg/FfmpegLibrary.java +++ b/extensions/ffmpeg/src/main/java/com/google/android/exoplayer2/ext/ffmpeg/FfmpegLibrary.java @@ -34,8 +34,7 @@ public final class FfmpegLibrary { private static final String TAG = "FfmpegLibrary"; - private static final LibraryLoader LOADER = - new LibraryLoader("avutil", "swresample", "avcodec", "ffmpeg_jni"); + private static final LibraryLoader LOADER = new LibraryLoader("ffmpegJNI"); private static @MonotonicNonNull String version; private static int inputBufferPaddingSize = C.LENGTH_UNSET; @@ -145,10 +144,6 @@ public static boolean supportsFormat(String mimeType) { return "pcm_mulaw"; case MimeTypes.AUDIO_ALAW: return "pcm_alaw"; - case MimeTypes.VIDEO_H264: - return "h264"; - case MimeTypes.VIDEO_H265: - return "hevc"; default: return null; } diff --git a/extensions/ffmpeg/src/main/jni/CMakeLists.txt b/extensions/ffmpeg/src/main/jni/CMakeLists.txt index b60af4fa18a..fd3e398dd9a 100644 --- a/extensions/ffmpeg/src/main/jni/CMakeLists.txt +++ b/extensions/ffmpeg/src/main/jni/CMakeLists.txt @@ -3,17 +3,17 @@ cmake_minimum_required(VERSION 3.7.1 FATAL_ERROR) # Enable C++11 features. set(CMAKE_CXX_STANDARD 11) -project(libffmpeg_jni C CXX) +project(libffmpegJNI C CXX) set(ffmpeg_location "${CMAKE_CURRENT_SOURCE_DIR}/ffmpeg") set(ffmpeg_binaries "${ffmpeg_location}/android-libs/${ANDROID_ABI}") foreach(ffmpeg_lib avutil swresample avcodec) - set(ffmpeg_lib_filename lib${ffmpeg_lib}.so) + set(ffmpeg_lib_filename lib${ffmpeg_lib}.a) set(ffmpeg_lib_file_path ${ffmpeg_binaries}/${ffmpeg_lib_filename}) add_library( ${ffmpeg_lib} - SHARED + STATIC IMPORTED) set_target_properties( ${ffmpeg_lib} PROPERTIES @@ -24,13 +24,13 @@ endforeach() include_directories(${ffmpeg_location}) find_library(android_log_lib log) -add_library(ffmpeg_jni +add_library(ffmpegJNI SHARED ffmpeg_jni.cc) -target_link_libraries(ffmpeg_jni +target_link_libraries(ffmpegJNI PRIVATE android - PRIVATE avutil PRIVATE swresample PRIVATE avcodec + PRIVATE avutil PRIVATE ${android_log_lib}) diff --git a/extensions/ffmpeg/src/main/jni/build_ffmpeg.sh b/extensions/ffmpeg/src/main/jni/build_ffmpeg.sh index 4660669a336..7b2e9339021 100755 --- a/extensions/ffmpeg/src/main/jni/build_ffmpeg.sh +++ b/extensions/ffmpeg/src/main/jni/build_ffmpeg.sh @@ -19,10 +19,12 @@ FFMPEG_EXT_PATH=$1 NDK_PATH=$2 HOST_PLATFORM=$3 ENABLED_DECODERS=("${@:4}") +JOBS=$(nproc 2> /dev/null || sysctl -n hw.ncpu 2> /dev/null || echo 4) +echo "Using $JOBS jobs for make" COMMON_OPTIONS=" --target-os=android - --disable-static - --enable-shared + --enable-static + --disable-shared --disable-doc --disable-programs --disable-everything @@ -48,11 +50,13 @@ cd "${FFMPEG_EXT_PATH}/jni/ffmpeg" --cpu=armv7-a \ --cross-prefix="${TOOLCHAIN_PREFIX}/armv7a-linux-androideabi16-" \ --nm="${TOOLCHAIN_PREFIX}/arm-linux-androideabi-nm" \ + --ar="${TOOLCHAIN_PREFIX}/arm-linux-androideabi-ar" \ + --ranlib="${TOOLCHAIN_PREFIX}/arm-linux-androideabi-ranlib" \ --strip="${TOOLCHAIN_PREFIX}/arm-linux-androideabi-strip" \ --extra-cflags="-march=armv7-a -mfloat-abi=softfp" \ --extra-ldflags="-Wl,--fix-cortex-a8" \ ${COMMON_OPTIONS} -make -j4 +make -j$JOBS make install-libs make clean ./configure \ @@ -61,9 +65,11 @@ make clean --cpu=armv8-a \ --cross-prefix="${TOOLCHAIN_PREFIX}/aarch64-linux-android21-" \ --nm="${TOOLCHAIN_PREFIX}/aarch64-linux-android-nm" \ + --ar="${TOOLCHAIN_PREFIX}/aarch64-linux-android-ar" \ + --ranlib="${TOOLCHAIN_PREFIX}/aarch64-linux-android-ranlib" \ --strip="${TOOLCHAIN_PREFIX}/aarch64-linux-android-strip" \ ${COMMON_OPTIONS} -make -j4 +make -j$JOBS make install-libs make clean ./configure \ @@ -72,10 +78,12 @@ make clean --cpu=i686 \ --cross-prefix="${TOOLCHAIN_PREFIX}/i686-linux-android16-" \ --nm="${TOOLCHAIN_PREFIX}/i686-linux-android-nm" \ + --ar="${TOOLCHAIN_PREFIX}/i686-linux-android-ar" \ + --ranlib="${TOOLCHAIN_PREFIX}/i686-linux-android-ranlib" \ --strip="${TOOLCHAIN_PREFIX}/i686-linux-android-strip" \ --disable-asm \ ${COMMON_OPTIONS} -make -j4 +make -j$JOBS make install-libs make clean ./configure \ @@ -84,9 +92,11 @@ make clean --cpu=x86_64 \ --cross-prefix="${TOOLCHAIN_PREFIX}/x86_64-linux-android21-" \ --nm="${TOOLCHAIN_PREFIX}/x86_64-linux-android-nm" \ + --ar="${TOOLCHAIN_PREFIX}/x86_64-linux-android-ar" \ + --ranlib="${TOOLCHAIN_PREFIX}/x86_64-linux-android-ranlib" \ --strip="${TOOLCHAIN_PREFIX}/x86_64-linux-android-strip" \ --disable-asm \ ${COMMON_OPTIONS} -make -j4 +make -j$JOBS make install-libs make clean diff --git a/extensions/flac/README.md b/extensions/flac/README.md index a9d4c3094e2..47c74d11486 100644 --- a/extensions/flac/README.md +++ b/extensions/flac/README.md @@ -68,7 +68,7 @@ renderer. ### Using `FlacExtractor` ### -`FlacExtractor` is used via `ExtractorMediaSource`. If you're using +`FlacExtractor` is used via `ProgressiveMediaSource`. If you're using `DefaultExtractorsFactory`, `FlacExtractor` will automatically be used to read `.flac` files. If you're not using `DefaultExtractorsFactory`, return a `FlacExtractor` from your `ExtractorsFactory.createExtractors` implementation. diff --git a/extensions/flac/src/main/java/com/google/android/exoplayer2/ext/flac/FlacDecoder.java b/extensions/flac/src/main/java/com/google/android/exoplayer2/ext/flac/FlacDecoder.java index 84cb081b298..f6c5f6a6a73 100644 --- a/extensions/flac/src/main/java/com/google/android/exoplayer2/ext/flac/FlacDecoder.java +++ b/extensions/flac/src/main/java/com/google/android/exoplayer2/ext/flac/FlacDecoder.java @@ -15,7 +15,10 @@ */ package com.google.android.exoplayer2.ext.flac; +import static androidx.annotation.VisibleForTesting.PACKAGE_PRIVATE; + import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.ParserException; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; @@ -27,11 +30,10 @@ import java.nio.ByteBuffer; import java.util.List; -/** - * Flac decoder. - */ -/* package */ final class FlacDecoder extends - SimpleDecoder { +/** Flac decoder. */ +@VisibleForTesting(otherwise = PACKAGE_PRIVATE) +public final class FlacDecoder + extends SimpleDecoder { private final FlacStreamMetadata streamMetadata; private final FlacDecoderJni decoderJni; diff --git a/extensions/flac/src/main/java/com/google/android/exoplayer2/ext/flac/LibflacAudioRenderer.java b/extensions/flac/src/main/java/com/google/android/exoplayer2/ext/flac/LibflacAudioRenderer.java index df511866a38..4fea5ffa53a 100644 --- a/extensions/flac/src/main/java/com/google/android/exoplayer2/ext/flac/LibflacAudioRenderer.java +++ b/extensions/flac/src/main/java/com/google/android/exoplayer2/ext/flac/LibflacAudioRenderer.java @@ -79,11 +79,11 @@ public String getName() { } @Override - @FormatSupport + @C.FormatSupport protected int supportsFormatInternal(Format format) { if (!FlacLibrary.isAvailable() || !MimeTypes.AUDIO_FLAC.equalsIgnoreCase(format.sampleMimeType)) { - return FORMAT_UNSUPPORTED_TYPE; + return C.FORMAT_UNSUPPORTED_TYPE; } // Compute the format that the FLAC decoder will output. Format outputFormat; @@ -102,11 +102,11 @@ protected int supportsFormatInternal(Format format) { outputFormat = getOutputFormat(streamMetadata); } if (!sinkSupportsFormat(outputFormat)) { - return FORMAT_UNSUPPORTED_SUBTYPE; + return C.FORMAT_UNSUPPORTED_SUBTYPE; } else if (format.exoMediaCryptoType != null) { - return FORMAT_UNSUPPORTED_DRM; + return C.FORMAT_UNSUPPORTED_DRM; } else { - return FORMAT_HANDLED; + return C.FORMAT_HANDLED; } } diff --git a/extensions/ima/README.md b/extensions/ima/README.md index c67dfdbb5d5..016f848c7af 100644 --- a/extensions/ima/README.md +++ b/extensions/ima/README.md @@ -33,17 +33,19 @@ of the developer guide. The `AdsLoaderProvider` passed to the player's extension only supports players which are accessed on the application's main thread. -Resuming the player after entering the background requires some special handling -when playing ads. The player and its media source are released on entering the -background, and are recreated when returning to the foreground. When playing ads -it is necessary to persist ad playback state while in the background by keeping -a reference to the `ImaAdsLoader`. When re-entering the foreground, pass the -same instance back when `AdsLoaderProvider.getAdsLoader(Uri adTagUri)` is called -to restore the state. It is also important to persist the player position when -entering the background by storing the value of `player.getContentPosition()`. -On returning to the foreground, seek to that position before preparing the new -player instance. Finally, it is important to call `ImaAdsLoader.release()` when -playback has finished and will not be resumed. +Resuming the player after entering the background requires some special +handling when playing ads. The player and its media source are released on +entering the background, and are recreated when returning to the foreground. +When playing ads it is necessary to persist ad playback state while in the +background by keeping a reference to the `ImaAdsLoader`. When re-entering the +foreground, pass the same instance back when +`AdsLoaderProvider.getAdsLoader(MediaItem.AdsConfiguration adsConfiguration)` +is called to restore the state. It is also important to persist the player +position when entering the background by storing the value of +`player.getContentPosition()`. On returning to the foreground, seek to that +position before preparing the new player instance. Finally, it is important to +call `ImaAdsLoader.release()` when playback has finished and will not be +resumed. You can try the IMA extension in the ExoPlayer demo app, which has test content in the "IMA sample ad tags" section of the sample chooser. The demo app's diff --git a/extensions/ima/build.gradle b/extensions/ima/build.gradle index 8cdfb0dffc7..72d40b8f8f7 100644 --- a/extensions/ima/build.gradle +++ b/extensions/ima/build.gradle @@ -25,7 +25,7 @@ android { } dependencies { - api 'com.google.ads.interactivemedia.v3:interactivemedia:3.21.0' + api 'com.google.ads.interactivemedia.v3:interactivemedia:3.22.0' implementation project(modulePrefix + 'library-core') implementation 'androidx.annotation:annotation:' + androidxAnnotationVersion implementation 'com.google.android.gms:play-services-ads-identifier:17.0.0' diff --git a/extensions/ima/proguard-rules.txt b/extensions/ima/proguard-rules.txt index feef3daf7a0..a46014de755 100644 --- a/extensions/ima/proguard-rules.txt +++ b/extensions/ima/proguard-rules.txt @@ -1,5 +1,6 @@ # Proguard rules specific to the IMA extension. +-dontwarn com.google.ads.interactivemedia.** -keep class com.google.ads.interactivemedia.** { *; } -keep interface com.google.ads.interactivemedia.** { *; } -keep class com.google.obf.** { *; } diff --git a/extensions/ima/src/androidTest/java/com/google/android/exoplayer2/ext/ima/ImaPlaybackTest.java b/extensions/ima/src/androidTest/java/com/google/android/exoplayer2/ext/ima/ImaPlaybackTest.java index 9527d35cef9..839c8329516 100644 --- a/extensions/ima/src/androidTest/java/com/google/android/exoplayer2/ext/ima/ImaPlaybackTest.java +++ b/extensions/ima/src/androidTest/java/com/google/android/exoplayer2/ext/ima/ImaPlaybackTest.java @@ -248,6 +248,7 @@ protected MediaSource buildSource( return new AdsMediaSource( contentMediaSource, adTagDataSpec, + /* adsId= */ adTagDataSpec.uri, new DefaultMediaSourceFactory(dataSourceFactory), Assertions.checkNotNull(imaAdsLoader), new AdViewProvider() { diff --git a/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/AdTagLoader.java b/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/AdTagLoader.java new file mode 100644 index 00000000000..4ce0610fb67 --- /dev/null +++ b/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/AdTagLoader.java @@ -0,0 +1,1455 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.ext.ima; + +import static com.google.android.exoplayer2.ext.ima.ImaUtil.BITRATE_UNSET; +import static com.google.android.exoplayer2.ext.ima.ImaUtil.TIMEOUT_UNSET; +import static com.google.android.exoplayer2.ext.ima.ImaUtil.getAdGroupTimesUsForCuePoints; +import static com.google.android.exoplayer2.ext.ima.ImaUtil.getImaLooper; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static java.lang.Math.max; + +import android.content.Context; +import android.net.Uri; +import android.os.Handler; +import android.os.SystemClock; +import android.view.ViewGroup; +import androidx.annotation.IntDef; +import androidx.annotation.Nullable; +import com.google.ads.interactivemedia.v3.api.AdDisplayContainer; +import com.google.ads.interactivemedia.v3.api.AdError; +import com.google.ads.interactivemedia.v3.api.AdErrorEvent; +import com.google.ads.interactivemedia.v3.api.AdErrorEvent.AdErrorListener; +import com.google.ads.interactivemedia.v3.api.AdEvent; +import com.google.ads.interactivemedia.v3.api.AdEvent.AdEventListener; +import com.google.ads.interactivemedia.v3.api.AdEvent.AdEventType; +import com.google.ads.interactivemedia.v3.api.AdPodInfo; +import com.google.ads.interactivemedia.v3.api.AdsLoader; +import com.google.ads.interactivemedia.v3.api.AdsLoader.AdsLoadedListener; +import com.google.ads.interactivemedia.v3.api.AdsManager; +import com.google.ads.interactivemedia.v3.api.AdsManagerLoadedEvent; +import com.google.ads.interactivemedia.v3.api.AdsRenderingSettings; +import com.google.ads.interactivemedia.v3.api.AdsRequest; +import com.google.ads.interactivemedia.v3.api.ImaSdkSettings; +import com.google.ads.interactivemedia.v3.api.player.AdMediaInfo; +import com.google.ads.interactivemedia.v3.api.player.ContentProgressProvider; +import com.google.ads.interactivemedia.v3.api.player.VideoAdPlayer; +import com.google.ads.interactivemedia.v3.api.player.VideoProgressUpdate; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.ExoPlaybackException; +import com.google.android.exoplayer2.ExoPlayerLibraryInfo; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.source.ads.AdPlaybackState; +import com.google.android.exoplayer2.source.ads.AdsLoader.AdViewProvider; +import com.google.android.exoplayer2.source.ads.AdsLoader.EventListener; +import com.google.android.exoplayer2.source.ads.AdsLoader.OverlayInfo; +import com.google.android.exoplayer2.source.ads.AdsMediaSource.AdLoadException; +import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.upstream.DataSpec; +import com.google.android.exoplayer2.util.Log; +import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.BiMap; +import com.google.common.collect.HashBiMap; +import java.io.IOException; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +/** Handles loading and playback of a single ad tag. */ +/* package */ final class AdTagLoader implements Player.EventListener { + + private static final String TAG = "AdTagLoader"; + + private static final String IMA_SDK_SETTINGS_PLAYER_TYPE = "google/exo.ext.ima"; + private static final String IMA_SDK_SETTINGS_PLAYER_VERSION = ExoPlayerLibraryInfo.VERSION; + + /** + * Interval at which ad progress updates are provided to the IMA SDK, in milliseconds. 100 ms is + * the interval recommended by the IMA documentation. + * + * @see VideoAdPlayer.VideoAdPlayerCallback + */ + private static final int AD_PROGRESS_UPDATE_INTERVAL_MS = 100; + + /** The value used in {@link VideoProgressUpdate}s to indicate an unset duration. */ + private static final long IMA_DURATION_UNSET = -1L; + + /** + * Threshold before the end of content at which IMA is notified that content is complete if the + * player buffers, in milliseconds. + */ + private static final long THRESHOLD_END_OF_CONTENT_MS = 5000; + /** + * Threshold before the start of an ad at which IMA is expected to be able to preload the ad, in + * milliseconds. + */ + private static final long THRESHOLD_AD_PRELOAD_MS = 4000; + /** The threshold below which ad cue points are treated as matching, in microseconds. */ + private static final long THRESHOLD_AD_MATCH_US = 1000; + + /** The state of ad playback. */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({IMA_AD_STATE_NONE, IMA_AD_STATE_PLAYING, IMA_AD_STATE_PAUSED}) + private @interface ImaAdState {} + + /** The ad playback state when IMA is not playing an ad. */ + private static final int IMA_AD_STATE_NONE = 0; + /** + * The ad playback state when IMA has called {@link ComponentListener#playAd(AdMediaInfo)} and not + * {@link ComponentListener##pauseAd(AdMediaInfo)}. + */ + private static final int IMA_AD_STATE_PLAYING = 1; + /** + * The ad playback state when IMA has called {@link ComponentListener#pauseAd(AdMediaInfo)} while + * playing an ad. + */ + private static final int IMA_AD_STATE_PAUSED = 2; + + private final ImaUtil.Configuration configuration; + private final ImaUtil.ImaFactory imaFactory; + private final List supportedMimeTypes; + private final DataSpec adTagDataSpec; + private final Object adsId; + private final Timeline.Period period; + private final Handler handler; + private final ComponentListener componentListener; + private final List eventListeners; + private final List adCallbacks; + private final Runnable updateAdProgressRunnable; + private final BiMap adInfoByAdMediaInfo; + private final AdDisplayContainer adDisplayContainer; + private final AdsLoader adsLoader; + + @Nullable private Object pendingAdRequestContext; + @Nullable private Player player; + private VideoProgressUpdate lastContentProgress; + private VideoProgressUpdate lastAdProgress; + private int lastVolumePercent; + + @Nullable private AdsManager adsManager; + private boolean isAdsManagerInitialized; + @Nullable private AdLoadException pendingAdLoadError; + private Timeline timeline; + private long contentDurationMs; + private AdPlaybackState adPlaybackState; + + private boolean released; + + // Fields tracking IMA's state. + + /** Whether IMA has sent an ad event to pause content since the last resume content event. */ + private boolean imaPausedContent; + /** The current ad playback state. */ + private @ImaAdState int imaAdState; + /** The current ad media info, or {@code null} if in state {@link #IMA_AD_STATE_NONE}. */ + @Nullable private AdMediaInfo imaAdMediaInfo; + /** The current ad info, or {@code null} if in state {@link #IMA_AD_STATE_NONE}. */ + @Nullable private AdInfo imaAdInfo; + /** Whether IMA has been notified that playback of content has finished. */ + private boolean sentContentComplete; + + // Fields tracking the player/loader state. + + /** Whether the player is playing an ad. */ + private boolean playingAd; + /** Whether the player is buffering an ad. */ + private boolean bufferingAd; + /** + * If the player is playing an ad, stores the ad index in its ad group. {@link C#INDEX_UNSET} + * otherwise. + */ + private int playingAdIndexInAdGroup; + /** + * The ad info for a pending ad for which the media failed preparation, or {@code null} if no + * pending ads have failed to prepare. + */ + @Nullable private AdInfo pendingAdPrepareErrorAdInfo; + /** + * If a content period has finished but IMA has not yet called {@link + * ComponentListener#playAd(AdMediaInfo)}, stores the value of {@link + * SystemClock#elapsedRealtime()} when the content stopped playing. This can be used to determine + * a fake, increasing content position. {@link C#TIME_UNSET} otherwise. + */ + private long fakeContentProgressElapsedRealtimeMs; + /** + * If {@link #fakeContentProgressElapsedRealtimeMs} is set, stores the offset from which the + * content progress should increase. {@link C#TIME_UNSET} otherwise. + */ + private long fakeContentProgressOffsetMs; + /** Stores the pending content position when a seek operation was intercepted to play an ad. */ + private long pendingContentPositionMs; + /** + * Whether {@link ComponentListener#getContentProgress()} has sent {@link + * #pendingContentPositionMs} to IMA. + */ + private boolean sentPendingContentPositionMs; + /** + * Stores the real time in milliseconds at which the player started buffering, possibly due to not + * having preloaded an ad, or {@link C#TIME_UNSET} if not applicable. + */ + private long waitingForPreloadElapsedRealtimeMs; + + /** Creates a new ad tag loader, starting the ad request if the ad tag is valid. */ + @SuppressWarnings({"methodref.receiver.bound.invalid", "method.invocation.invalid"}) + public AdTagLoader( + Context context, + ImaUtil.Configuration configuration, + ImaUtil.ImaFactory imaFactory, + List supportedMimeTypes, + DataSpec adTagDataSpec, + Object adsId, + @Nullable ViewGroup adViewGroup) { + this.configuration = configuration; + this.imaFactory = imaFactory; + @Nullable ImaSdkSettings imaSdkSettings = configuration.imaSdkSettings; + if (imaSdkSettings == null) { + imaSdkSettings = imaFactory.createImaSdkSettings(); + if (configuration.debugModeEnabled) { + imaSdkSettings.setDebugMode(true); + } + } + imaSdkSettings.setPlayerType(IMA_SDK_SETTINGS_PLAYER_TYPE); + imaSdkSettings.setPlayerVersion(IMA_SDK_SETTINGS_PLAYER_VERSION); + this.supportedMimeTypes = supportedMimeTypes; + this.adTagDataSpec = adTagDataSpec; + this.adsId = adsId; + period = new Timeline.Period(); + handler = Util.createHandler(getImaLooper(), /* callback= */ null); + componentListener = new ComponentListener(); + eventListeners = new ArrayList<>(); + adCallbacks = new ArrayList<>(/* initialCapacity= */ 1); + if (configuration.applicationVideoAdPlayerCallback != null) { + adCallbacks.add(configuration.applicationVideoAdPlayerCallback); + } + updateAdProgressRunnable = this::updateAdProgress; + adInfoByAdMediaInfo = HashBiMap.create(); + lastContentProgress = VideoProgressUpdate.VIDEO_TIME_NOT_READY; + lastAdProgress = VideoProgressUpdate.VIDEO_TIME_NOT_READY; + fakeContentProgressElapsedRealtimeMs = C.TIME_UNSET; + fakeContentProgressOffsetMs = C.TIME_UNSET; + pendingContentPositionMs = C.TIME_UNSET; + waitingForPreloadElapsedRealtimeMs = C.TIME_UNSET; + contentDurationMs = C.TIME_UNSET; + timeline = Timeline.EMPTY; + adPlaybackState = AdPlaybackState.NONE; + if (adViewGroup != null) { + adDisplayContainer = + imaFactory.createAdDisplayContainer(adViewGroup, /* player= */ componentListener); + } else { + adDisplayContainer = + imaFactory.createAudioAdDisplayContainer(context, /* player= */ componentListener); + } + if (configuration.companionAdSlots != null) { + adDisplayContainer.setCompanionSlots(configuration.companionAdSlots); + } + adsLoader = requestAds(context, imaSdkSettings, adDisplayContainer); + } + + /** Returns the underlying IMA SDK ads loader. */ + public AdsLoader getAdsLoader() { + return adsLoader; + } + + /** Returns the IMA SDK ad display container. */ + public AdDisplayContainer getAdDisplayContainer() { + return adDisplayContainer; + } + + /** Skips the current skippable ad, if there is one. */ + public void skipAd() { + if (adsManager != null) { + adsManager.skip(); + } + } + + /** + * Starts passing events from this instance (including any pending ad playback state) and + * registers obstructions. + */ + public void addListenerWithAdView(EventListener eventListener, AdViewProvider adViewProvider) { + boolean isStarted = !eventListeners.isEmpty(); + eventListeners.add(eventListener); + if (isStarted) { + if (!AdPlaybackState.NONE.equals(adPlaybackState)) { + // Pass the existing ad playback state to the new listener. + eventListener.onAdPlaybackState(adPlaybackState); + } + return; + } + lastVolumePercent = 0; + lastAdProgress = VideoProgressUpdate.VIDEO_TIME_NOT_READY; + lastContentProgress = VideoProgressUpdate.VIDEO_TIME_NOT_READY; + maybeNotifyPendingAdLoadError(); + if (!AdPlaybackState.NONE.equals(adPlaybackState)) { + // Pass the ad playback state to the player, and resume ads if necessary. + eventListener.onAdPlaybackState(adPlaybackState); + } else if (adsManager != null) { + adPlaybackState = + new AdPlaybackState(adsId, getAdGroupTimesUsForCuePoints(adsManager.getAdCuePoints())); + updateAdPlaybackState(); + } + for (OverlayInfo overlayInfo : adViewProvider.getAdOverlayInfos()) { + adDisplayContainer.registerFriendlyObstruction( + imaFactory.createFriendlyObstruction( + overlayInfo.view, + ImaUtil.getFriendlyObstructionPurpose(overlayInfo.purpose), + overlayInfo.reasonDetail)); + } + } + + /** + * Populates the ad playback state with loaded cue points, if available. Any preroll will be + * paused immediately while waiting for this instance to be {@link #activate(Player) activated}. + */ + public void maybePreloadAds(long contentPositionMs, long contentDurationMs) { + maybeInitializeAdsManager(contentPositionMs, contentDurationMs); + } + + /** Activates playback. */ + public void activate(Player player) { + this.player = player; + player.addListener(this); + + boolean playWhenReady = player.getPlayWhenReady(); + onTimelineChanged(player.getCurrentTimeline(), Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE); + if (!AdPlaybackState.NONE.equals(adPlaybackState) + && adsManager != null + && imaPausedContent + && playWhenReady) { + adsManager.resume(); + } + } + + /** Deactivates playback. */ + public void deactivate() { + Player player = checkNotNull(this.player); + if (!AdPlaybackState.NONE.equals(adPlaybackState) && imaPausedContent) { + if (adsManager != null) { + adsManager.pause(); + } + adPlaybackState = + adPlaybackState.withAdResumePositionUs( + playingAd ? C.msToUs(player.getCurrentPosition()) : 0); + } + lastVolumePercent = getPlayerVolumePercent(); + lastAdProgress = getAdVideoProgressUpdate(); + lastContentProgress = getContentVideoProgressUpdate(); + + player.removeListener(this); + this.player = null; + } + + /** Stops passing of events from this instance and unregisters obstructions. */ + public void removeListener(EventListener eventListener) { + eventListeners.remove(eventListener); + if (eventListeners.isEmpty()) { + adDisplayContainer.unregisterAllFriendlyObstructions(); + } + } + + /** Releases all resources used by the ad tag loader. */ + public void release() { + if (released) { + return; + } + released = true; + pendingAdRequestContext = null; + destroyAdsManager(); + adsLoader.removeAdsLoadedListener(componentListener); + adsLoader.removeAdErrorListener(componentListener); + if (configuration.applicationAdErrorListener != null) { + adsLoader.removeAdErrorListener(configuration.applicationAdErrorListener); + } + adsLoader.release(); + imaPausedContent = false; + imaAdState = IMA_AD_STATE_NONE; + imaAdMediaInfo = null; + stopUpdatingAdProgress(); + imaAdInfo = null; + pendingAdLoadError = null; + adPlaybackState = new AdPlaybackState(adsId); + updateAdPlaybackState(); + } + + /** Notifies the IMA SDK that the specified ad has been prepared for playback. */ + public void handlePrepareComplete(int adGroupIndex, int adIndexInAdGroup) { + AdInfo adInfo = new AdInfo(adGroupIndex, adIndexInAdGroup); + if (configuration.debugModeEnabled) { + Log.d(TAG, "Prepared ad " + adInfo); + } + @Nullable AdMediaInfo adMediaInfo = adInfoByAdMediaInfo.inverse().get(adInfo); + if (adMediaInfo != null) { + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onLoaded(adMediaInfo); + } + } else { + Log.w(TAG, "Unexpected prepared ad " + adInfo); + } + } + + /** Notifies the IMA SDK that the specified ad has failed to prepare for playback. */ + public void handlePrepareError(int adGroupIndex, int adIndexInAdGroup, IOException exception) { + if (player == null) { + return; + } + try { + handleAdPrepareError(adGroupIndex, adIndexInAdGroup, exception); + } catch (RuntimeException e) { + maybeNotifyInternalError("handlePrepareError", e); + } + } + + // Player.EventListener implementation. + + @Override + public void onTimelineChanged(Timeline timeline, @Player.TimelineChangeReason int reason) { + if (timeline.isEmpty()) { + // The player is being reset or contains no media. + return; + } + this.timeline = timeline; + Player player = checkNotNull(this.player); + long contentDurationUs = timeline.getPeriod(player.getCurrentPeriodIndex(), period).durationUs; + contentDurationMs = C.usToMs(contentDurationUs); + if (contentDurationUs != adPlaybackState.contentDurationUs) { + adPlaybackState = adPlaybackState.withContentDurationUs(contentDurationUs); + updateAdPlaybackState(); + } + long contentPositionMs = getContentPeriodPositionMs(player, timeline, period); + maybeInitializeAdsManager(contentPositionMs, contentDurationMs); + handleTimelineOrPositionChanged(); + } + + @Override + public void onPositionDiscontinuity(@Player.DiscontinuityReason int reason) { + handleTimelineOrPositionChanged(); + } + + @Override + public void onPlaybackStateChanged(@Player.State int playbackState) { + @Nullable Player player = this.player; + if (adsManager == null || player == null) { + return; + } + + if (playbackState == Player.STATE_BUFFERING + && !player.isPlayingAd() + && isWaitingForAdToLoad()) { + waitingForPreloadElapsedRealtimeMs = SystemClock.elapsedRealtime(); + } else if (playbackState == Player.STATE_READY) { + waitingForPreloadElapsedRealtimeMs = C.TIME_UNSET; + } + + handlePlayerStateChanged(player.getPlayWhenReady(), playbackState); + } + + @Override + public void onPlayWhenReadyChanged( + boolean playWhenReady, @Player.PlayWhenReadyChangeReason int reason) { + if (adsManager == null || player == null) { + return; + } + + if (imaAdState == IMA_AD_STATE_PLAYING && !playWhenReady) { + adsManager.pause(); + return; + } + + if (imaAdState == IMA_AD_STATE_PAUSED && playWhenReady) { + adsManager.resume(); + return; + } + handlePlayerStateChanged(playWhenReady, player.getPlaybackState()); + } + + @Override + public void onPlayerError(ExoPlaybackException error) { + if (imaAdState != IMA_AD_STATE_NONE) { + AdMediaInfo adMediaInfo = checkNotNull(imaAdMediaInfo); + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onError(adMediaInfo); + } + } + } + + // Internal methods. + + private AdsLoader requestAds( + Context context, ImaSdkSettings imaSdkSettings, AdDisplayContainer adDisplayContainer) { + AdsLoader adsLoader = imaFactory.createAdsLoader(context, imaSdkSettings, adDisplayContainer); + adsLoader.addAdErrorListener(componentListener); + if (configuration.applicationAdErrorListener != null) { + adsLoader.addAdErrorListener(configuration.applicationAdErrorListener); + } + adsLoader.addAdsLoadedListener(componentListener); + AdsRequest request; + try { + request = ImaUtil.getAdsRequestForAdTagDataSpec(imaFactory, adTagDataSpec); + } catch (IOException e) { + adPlaybackState = new AdPlaybackState(adsId); + updateAdPlaybackState(); + pendingAdLoadError = AdLoadException.createForAllAds(e); + maybeNotifyPendingAdLoadError(); + return adsLoader; + } + pendingAdRequestContext = new Object(); + request.setUserRequestContext(pendingAdRequestContext); + if (configuration.enableContinuousPlayback != null) { + request.setContinuousPlayback(configuration.enableContinuousPlayback); + } + if (configuration.vastLoadTimeoutMs != TIMEOUT_UNSET) { + request.setVastLoadTimeout(configuration.vastLoadTimeoutMs); + } + request.setContentProgressProvider(componentListener); + adsLoader.requestAds(request); + return adsLoader; + } + + private void maybeInitializeAdsManager(long contentPositionMs, long contentDurationMs) { + @Nullable AdsManager adsManager = this.adsManager; + if (!isAdsManagerInitialized && adsManager != null) { + isAdsManagerInitialized = true; + @Nullable + AdsRenderingSettings adsRenderingSettings = + setupAdsRendering(contentPositionMs, contentDurationMs); + if (adsRenderingSettings == null) { + // There are no ads to play. + destroyAdsManager(); + } else { + adsManager.init(adsRenderingSettings); + adsManager.start(); + if (configuration.debugModeEnabled) { + Log.d(TAG, "Initialized with ads rendering settings: " + adsRenderingSettings); + } + } + updateAdPlaybackState(); + } + } + + /** + * Configures ads rendering for starting playback, returning the settings for the IMA SDK or + * {@code null} if no ads should play. + */ + @Nullable + private AdsRenderingSettings setupAdsRendering(long contentPositionMs, long contentDurationMs) { + AdsRenderingSettings adsRenderingSettings = imaFactory.createAdsRenderingSettings(); + adsRenderingSettings.setEnablePreloading(true); + adsRenderingSettings.setMimeTypes( + configuration.adMediaMimeTypes != null + ? configuration.adMediaMimeTypes + : supportedMimeTypes); + if (configuration.mediaLoadTimeoutMs != TIMEOUT_UNSET) { + adsRenderingSettings.setLoadVideoTimeout(configuration.mediaLoadTimeoutMs); + } + if (configuration.mediaBitrate != BITRATE_UNSET) { + adsRenderingSettings.setBitrateKbps(configuration.mediaBitrate / 1000); + } + adsRenderingSettings.setFocusSkipButtonWhenAvailable( + configuration.focusSkipButtonWhenAvailable); + if (configuration.adUiElements != null) { + adsRenderingSettings.setUiElements(configuration.adUiElements); + } + + // Skip ads based on the start position as required. + long[] adGroupTimesUs = adPlaybackState.adGroupTimesUs; + int adGroupForPositionIndex = + adPlaybackState.getAdGroupIndexForPositionUs( + C.msToUs(contentPositionMs), C.msToUs(contentDurationMs)); + if (adGroupForPositionIndex != C.INDEX_UNSET) { + boolean playAdWhenStartingPlayback = + configuration.playAdBeforeStartPosition + || adGroupTimesUs[adGroupForPositionIndex] == C.msToUs(contentPositionMs); + if (!playAdWhenStartingPlayback) { + adGroupForPositionIndex++; + } else if (hasMidrollAdGroups(adGroupTimesUs)) { + // Provide the player's initial position to trigger loading and playing the ad. If there are + // no midrolls, we are playing a preroll and any pending content position wouldn't be + // cleared. + pendingContentPositionMs = contentPositionMs; + } + if (adGroupForPositionIndex > 0) { + for (int i = 0; i < adGroupForPositionIndex; i++) { + adPlaybackState = adPlaybackState.withSkippedAdGroup(i); + } + if (adGroupForPositionIndex == adGroupTimesUs.length) { + // We don't need to play any ads. Because setPlayAdsAfterTime does not discard non-VMAP + // ads, we signal that no ads will render so the caller can destroy the ads manager. + return null; + } + long adGroupForPositionTimeUs = adGroupTimesUs[adGroupForPositionIndex]; + long adGroupBeforePositionTimeUs = adGroupTimesUs[adGroupForPositionIndex - 1]; + if (adGroupForPositionTimeUs == C.TIME_END_OF_SOURCE) { + // Play the postroll by offsetting the start position just past the last non-postroll ad. + adsRenderingSettings.setPlayAdsAfterTime( + (double) adGroupBeforePositionTimeUs / C.MICROS_PER_SECOND + 1d); + } else { + // Play ads after the midpoint between the ad to play and the one before it, to avoid + // issues with rounding one of the two ad times. + double midpointTimeUs = (adGroupForPositionTimeUs + adGroupBeforePositionTimeUs) / 2d; + adsRenderingSettings.setPlayAdsAfterTime(midpointTimeUs / C.MICROS_PER_SECOND); + } + } + } + return adsRenderingSettings; + } + + private VideoProgressUpdate getContentVideoProgressUpdate() { + boolean hasContentDuration = contentDurationMs != C.TIME_UNSET; + long contentPositionMs; + if (pendingContentPositionMs != C.TIME_UNSET) { + sentPendingContentPositionMs = true; + contentPositionMs = pendingContentPositionMs; + } else if (player == null) { + return lastContentProgress; + } else if (fakeContentProgressElapsedRealtimeMs != C.TIME_UNSET) { + long elapsedSinceEndMs = SystemClock.elapsedRealtime() - fakeContentProgressElapsedRealtimeMs; + contentPositionMs = fakeContentProgressOffsetMs + elapsedSinceEndMs; + } else if (imaAdState == IMA_AD_STATE_NONE && !playingAd && hasContentDuration) { + contentPositionMs = getContentPeriodPositionMs(player, timeline, period); + } else { + return VideoProgressUpdate.VIDEO_TIME_NOT_READY; + } + long contentDurationMs = hasContentDuration ? this.contentDurationMs : IMA_DURATION_UNSET; + return new VideoProgressUpdate(contentPositionMs, contentDurationMs); + } + + private VideoProgressUpdate getAdVideoProgressUpdate() { + if (player == null) { + return lastAdProgress; + } else if (imaAdState != IMA_AD_STATE_NONE && playingAd) { + long adDuration = player.getDuration(); + return adDuration == C.TIME_UNSET + ? VideoProgressUpdate.VIDEO_TIME_NOT_READY + : new VideoProgressUpdate(player.getCurrentPosition(), adDuration); + } else { + return VideoProgressUpdate.VIDEO_TIME_NOT_READY; + } + } + + private void updateAdProgress() { + VideoProgressUpdate videoProgressUpdate = getAdVideoProgressUpdate(); + if (configuration.debugModeEnabled) { + Log.d(TAG, "Ad progress: " + ImaUtil.getStringForVideoProgressUpdate(videoProgressUpdate)); + } + + AdMediaInfo adMediaInfo = checkNotNull(imaAdMediaInfo); + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onAdProgress(adMediaInfo, videoProgressUpdate); + } + handler.removeCallbacks(updateAdProgressRunnable); + handler.postDelayed(updateAdProgressRunnable, AD_PROGRESS_UPDATE_INTERVAL_MS); + } + + private void stopUpdatingAdProgress() { + handler.removeCallbacks(updateAdProgressRunnable); + } + + private int getPlayerVolumePercent() { + @Nullable Player player = this.player; + if (player == null) { + return lastVolumePercent; + } + + @Nullable Player.AudioComponent audioComponent = player.getAudioComponent(); + if (audioComponent != null) { + return (int) (audioComponent.getVolume() * 100); + } + + // Check for a selected track using an audio renderer. + TrackSelectionArray trackSelections = player.getCurrentTrackSelections(); + for (int i = 0; i < player.getRendererCount() && i < trackSelections.length; i++) { + if (player.getRendererType(i) == C.TRACK_TYPE_AUDIO && trackSelections.get(i) != null) { + return 100; + } + } + return 0; + } + + private void handleAdEvent(AdEvent adEvent) { + if (adsManager == null) { + // Drop events after release. + return; + } + switch (adEvent.getType()) { + case AD_BREAK_FETCH_ERROR: + String adGroupTimeSecondsString = checkNotNull(adEvent.getAdData().get("adBreakTime")); + if (configuration.debugModeEnabled) { + Log.d(TAG, "Fetch error for ad at " + adGroupTimeSecondsString + " seconds"); + } + double adGroupTimeSeconds = Double.parseDouble(adGroupTimeSecondsString); + int adGroupIndex = + adGroupTimeSeconds == -1.0 + ? adPlaybackState.adGroupCount - 1 + : getAdGroupIndexForCuePointTimeSeconds(adGroupTimeSeconds); + markAdGroupInErrorStateAndClearPendingContentPosition(adGroupIndex); + break; + case CONTENT_PAUSE_REQUESTED: + // After CONTENT_PAUSE_REQUESTED, IMA will playAd/pauseAd/stopAd to show one or more ads + // before sending CONTENT_RESUME_REQUESTED. + imaPausedContent = true; + pauseContentInternal(); + break; + case TAPPED: + for (int i = 0; i < eventListeners.size(); i++) { + eventListeners.get(i).onAdTapped(); + } + break; + case CLICKED: + for (int i = 0; i < eventListeners.size(); i++) { + eventListeners.get(i).onAdClicked(); + } + break; + case CONTENT_RESUME_REQUESTED: + imaPausedContent = false; + resumeContentInternal(); + break; + case LOG: + Map adData = adEvent.getAdData(); + String message = "AdEvent: " + adData; + Log.i(TAG, message); + break; + default: + break; + } + } + + private void pauseContentInternal() { + imaAdState = IMA_AD_STATE_NONE; + if (sentPendingContentPositionMs) { + pendingContentPositionMs = C.TIME_UNSET; + sentPendingContentPositionMs = false; + } + } + + private void resumeContentInternal() { + if (imaAdInfo != null) { + adPlaybackState = adPlaybackState.withSkippedAdGroup(imaAdInfo.adGroupIndex); + updateAdPlaybackState(); + } + } + + /** + * Returns whether this instance is expecting the first ad in an the upcoming ad group to load + * within the {@link ImaUtil.Configuration#adPreloadTimeoutMs preload timeout}. + */ + private boolean isWaitingForAdToLoad() { + @Nullable Player player = this.player; + if (player == null) { + return false; + } + int adGroupIndex = getLoadingAdGroupIndex(); + if (adGroupIndex == C.INDEX_UNSET) { + return false; + } + AdPlaybackState.AdGroup adGroup = adPlaybackState.adGroups[adGroupIndex]; + if (adGroup.count != C.LENGTH_UNSET + && adGroup.count != 0 + && adGroup.states[0] != AdPlaybackState.AD_STATE_UNAVAILABLE) { + // An ad is available already. + return false; + } + long adGroupTimeMs = C.usToMs(adPlaybackState.adGroupTimesUs[adGroupIndex]); + long contentPositionMs = getContentPeriodPositionMs(player, timeline, period); + long timeUntilAdMs = adGroupTimeMs - contentPositionMs; + return timeUntilAdMs < configuration.adPreloadTimeoutMs; + } + + private void handlePlayerStateChanged(boolean playWhenReady, @Player.State int playbackState) { + if (playingAd && imaAdState == IMA_AD_STATE_PLAYING) { + if (!bufferingAd && playbackState == Player.STATE_BUFFERING) { + bufferingAd = true; + AdMediaInfo adMediaInfo = checkNotNull(imaAdMediaInfo); + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onBuffering(adMediaInfo); + } + stopUpdatingAdProgress(); + } else if (bufferingAd && playbackState == Player.STATE_READY) { + bufferingAd = false; + updateAdProgress(); + } + } + + if (imaAdState == IMA_AD_STATE_NONE + && playbackState == Player.STATE_BUFFERING + && playWhenReady) { + ensureSentContentCompleteIfAtEndOfStream(); + } else if (imaAdState != IMA_AD_STATE_NONE && playbackState == Player.STATE_ENDED) { + @Nullable AdMediaInfo adMediaInfo = imaAdMediaInfo; + if (adMediaInfo == null) { + Log.w(TAG, "onEnded without ad media info"); + } else { + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onEnded(adMediaInfo); + } + } + if (configuration.debugModeEnabled) { + Log.d(TAG, "VideoAdPlayerCallback.onEnded in onPlaybackStateChanged"); + } + } + } + + private void handleTimelineOrPositionChanged() { + @Nullable Player player = this.player; + if (adsManager == null || player == null) { + return; + } + if (!playingAd && !player.isPlayingAd()) { + ensureSentContentCompleteIfAtEndOfStream(); + if (!sentContentComplete && !timeline.isEmpty()) { + long positionMs = getContentPeriodPositionMs(player, timeline, period); + timeline.getPeriod(/* periodIndex= */ 0, period); + int newAdGroupIndex = period.getAdGroupIndexForPositionUs(C.msToUs(positionMs)); + if (newAdGroupIndex != C.INDEX_UNSET) { + sentPendingContentPositionMs = false; + pendingContentPositionMs = positionMs; + } + } + } + + boolean wasPlayingAd = playingAd; + int oldPlayingAdIndexInAdGroup = playingAdIndexInAdGroup; + playingAd = player.isPlayingAd(); + playingAdIndexInAdGroup = playingAd ? player.getCurrentAdIndexInAdGroup() : C.INDEX_UNSET; + boolean adFinished = wasPlayingAd && playingAdIndexInAdGroup != oldPlayingAdIndexInAdGroup; + if (adFinished) { + // IMA is waiting for the ad playback to finish so invoke the callback now. + // Either CONTENT_RESUME_REQUESTED will be passed next, or playAd will be called again. + @Nullable AdMediaInfo adMediaInfo = imaAdMediaInfo; + if (adMediaInfo == null) { + Log.w(TAG, "onEnded without ad media info"); + } else { + @Nullable AdInfo adInfo = adInfoByAdMediaInfo.get(adMediaInfo); + if (playingAdIndexInAdGroup == C.INDEX_UNSET + || (adInfo != null && adInfo.adIndexInAdGroup < playingAdIndexInAdGroup)) { + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onEnded(adMediaInfo); + } + if (configuration.debugModeEnabled) { + Log.d( + TAG, "VideoAdPlayerCallback.onEnded in onTimelineChanged/onPositionDiscontinuity"); + } + } + } + } + if (!sentContentComplete && !wasPlayingAd && playingAd && imaAdState == IMA_AD_STATE_NONE) { + int adGroupIndex = player.getCurrentAdGroupIndex(); + if (adPlaybackState.adGroupTimesUs[adGroupIndex] == C.TIME_END_OF_SOURCE) { + sendContentComplete(); + } else { + // IMA hasn't called playAd yet, so fake the content position. + fakeContentProgressElapsedRealtimeMs = SystemClock.elapsedRealtime(); + fakeContentProgressOffsetMs = C.usToMs(adPlaybackState.adGroupTimesUs[adGroupIndex]); + if (fakeContentProgressOffsetMs == C.TIME_END_OF_SOURCE) { + fakeContentProgressOffsetMs = contentDurationMs; + } + } + } + } + + private void loadAdInternal(AdMediaInfo adMediaInfo, AdPodInfo adPodInfo) { + if (adsManager == null) { + // Drop events after release. + if (configuration.debugModeEnabled) { + Log.d( + TAG, + "loadAd after release " + getAdMediaInfoString(adMediaInfo) + ", ad pod " + adPodInfo); + } + return; + } + + int adGroupIndex = getAdGroupIndexForAdPod(adPodInfo); + int adIndexInAdGroup = adPodInfo.getAdPosition() - 1; + AdInfo adInfo = new AdInfo(adGroupIndex, adIndexInAdGroup); + adInfoByAdMediaInfo.put(adMediaInfo, adInfo); + if (configuration.debugModeEnabled) { + Log.d(TAG, "loadAd " + getAdMediaInfoString(adMediaInfo)); + } + if (adPlaybackState.isAdInErrorState(adGroupIndex, adIndexInAdGroup)) { + // We have already marked this ad as having failed to load, so ignore the request. IMA will + // timeout after its media load timeout. + return; + } + + // The ad count may increase on successive loads of ads in the same ad pod, for example, due to + // separate requests for ad tags with multiple ads within the ad pod completing after an earlier + // ad has loaded. See also https://github.com/google/ExoPlayer/issues/7477. + AdPlaybackState.AdGroup adGroup = adPlaybackState.adGroups[adInfo.adGroupIndex]; + adPlaybackState = + adPlaybackState.withAdCount( + adInfo.adGroupIndex, max(adPodInfo.getTotalAds(), adGroup.states.length)); + adGroup = adPlaybackState.adGroups[adInfo.adGroupIndex]; + for (int i = 0; i < adIndexInAdGroup; i++) { + // Any preceding ads that haven't loaded are not going to load. + if (adGroup.states[i] == AdPlaybackState.AD_STATE_UNAVAILABLE) { + adPlaybackState = adPlaybackState.withAdLoadError(adGroupIndex, /* adIndexInAdGroup= */ i); + } + } + + Uri adUri = Uri.parse(adMediaInfo.getUrl()); + adPlaybackState = + adPlaybackState.withAdUri(adInfo.adGroupIndex, adInfo.adIndexInAdGroup, adUri); + updateAdPlaybackState(); + } + + private void playAdInternal(AdMediaInfo adMediaInfo) { + if (configuration.debugModeEnabled) { + Log.d(TAG, "playAd " + getAdMediaInfoString(adMediaInfo)); + } + if (adsManager == null) { + // Drop events after release. + return; + } + + if (imaAdState == IMA_AD_STATE_PLAYING) { + // IMA does not always call stopAd before resuming content. + // See [Internal: b/38354028]. + Log.w(TAG, "Unexpected playAd without stopAd"); + } + + if (imaAdState == IMA_AD_STATE_NONE) { + // IMA is requesting to play the ad, so stop faking the content position. + fakeContentProgressElapsedRealtimeMs = C.TIME_UNSET; + fakeContentProgressOffsetMs = C.TIME_UNSET; + imaAdState = IMA_AD_STATE_PLAYING; + imaAdMediaInfo = adMediaInfo; + imaAdInfo = checkNotNull(adInfoByAdMediaInfo.get(adMediaInfo)); + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onPlay(adMediaInfo); + } + if (pendingAdPrepareErrorAdInfo != null && pendingAdPrepareErrorAdInfo.equals(imaAdInfo)) { + pendingAdPrepareErrorAdInfo = null; + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onError(adMediaInfo); + } + } + updateAdProgress(); + } else { + imaAdState = IMA_AD_STATE_PLAYING; + checkState(adMediaInfo.equals(imaAdMediaInfo)); + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onResume(adMediaInfo); + } + } + if (player == null || !player.getPlayWhenReady()) { + // Either this loader hasn't been activated yet, or the player is paused now. + checkNotNull(adsManager).pause(); + } + } + + private void pauseAdInternal(AdMediaInfo adMediaInfo) { + if (configuration.debugModeEnabled) { + Log.d(TAG, "pauseAd " + getAdMediaInfoString(adMediaInfo)); + } + if (adsManager == null) { + // Drop event after release. + return; + } + if (imaAdState == IMA_AD_STATE_NONE) { + // This method is called if loadAd has been called but the loaded ad won't play due to a seek + // to a different position, so drop the event. See also [Internal: b/159111848]. + return; + } + if (configuration.debugModeEnabled && !adMediaInfo.equals(imaAdMediaInfo)) { + Log.w( + TAG, + "Unexpected pauseAd for " + + getAdMediaInfoString(adMediaInfo) + + ", expected " + + getAdMediaInfoString(imaAdMediaInfo)); + } + imaAdState = IMA_AD_STATE_PAUSED; + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onPause(adMediaInfo); + } + } + + private void stopAdInternal(AdMediaInfo adMediaInfo) { + if (configuration.debugModeEnabled) { + Log.d(TAG, "stopAd " + getAdMediaInfoString(adMediaInfo)); + } + if (adsManager == null) { + // Drop event after release. + return; + } + if (imaAdState == IMA_AD_STATE_NONE) { + // This method is called if loadAd has been called but the preloaded ad won't play due to a + // seek to a different position, so drop the event and discard the ad. See also [Internal: + // b/159111848]. + @Nullable AdInfo adInfo = adInfoByAdMediaInfo.get(adMediaInfo); + if (adInfo != null) { + adPlaybackState = + adPlaybackState.withSkippedAd(adInfo.adGroupIndex, adInfo.adIndexInAdGroup); + updateAdPlaybackState(); + } + return; + } + imaAdState = IMA_AD_STATE_NONE; + stopUpdatingAdProgress(); + // TODO: Handle the skipped event so the ad can be marked as skipped rather than played. + checkNotNull(imaAdInfo); + int adGroupIndex = imaAdInfo.adGroupIndex; + int adIndexInAdGroup = imaAdInfo.adIndexInAdGroup; + if (adPlaybackState.isAdInErrorState(adGroupIndex, adIndexInAdGroup)) { + // We have already marked this ad as having failed to load, so ignore the request. + return; + } + adPlaybackState = + adPlaybackState.withPlayedAd(adGroupIndex, adIndexInAdGroup).withAdResumePositionUs(0); + updateAdPlaybackState(); + if (!playingAd) { + imaAdMediaInfo = null; + imaAdInfo = null; + } + } + + private void handleAdGroupLoadError(Exception error) { + int adGroupIndex = getLoadingAdGroupIndex(); + if (adGroupIndex == C.INDEX_UNSET) { + Log.w(TAG, "Unable to determine ad group index for ad group load error", error); + return; + } + markAdGroupInErrorStateAndClearPendingContentPosition(adGroupIndex); + if (pendingAdLoadError == null) { + pendingAdLoadError = AdLoadException.createForAdGroup(error, adGroupIndex); + } + } + + private void markAdGroupInErrorStateAndClearPendingContentPosition(int adGroupIndex) { + // Update the ad playback state so all ads in the ad group are in the error state. + AdPlaybackState.AdGroup adGroup = adPlaybackState.adGroups[adGroupIndex]; + if (adGroup.count == C.LENGTH_UNSET) { + adPlaybackState = adPlaybackState.withAdCount(adGroupIndex, max(1, adGroup.states.length)); + adGroup = adPlaybackState.adGroups[adGroupIndex]; + } + for (int i = 0; i < adGroup.count; i++) { + if (adGroup.states[i] == AdPlaybackState.AD_STATE_UNAVAILABLE) { + if (configuration.debugModeEnabled) { + Log.d(TAG, "Removing ad " + i + " in ad group " + adGroupIndex); + } + adPlaybackState = adPlaybackState.withAdLoadError(adGroupIndex, i); + } + } + updateAdPlaybackState(); + // Clear any pending content position that triggered attempting to load the ad group. + pendingContentPositionMs = C.TIME_UNSET; + fakeContentProgressElapsedRealtimeMs = C.TIME_UNSET; + } + + private void handleAdPrepareError(int adGroupIndex, int adIndexInAdGroup, Exception exception) { + if (configuration.debugModeEnabled) { + Log.d( + TAG, "Prepare error for ad " + adIndexInAdGroup + " in group " + adGroupIndex, exception); + } + if (adsManager == null) { + Log.w(TAG, "Ignoring ad prepare error after release"); + return; + } + if (imaAdState == IMA_AD_STATE_NONE) { + // Send IMA a content position at the ad group so that it will try to play it, at which point + // we can notify that it failed to load. + fakeContentProgressElapsedRealtimeMs = SystemClock.elapsedRealtime(); + fakeContentProgressOffsetMs = C.usToMs(adPlaybackState.adGroupTimesUs[adGroupIndex]); + if (fakeContentProgressOffsetMs == C.TIME_END_OF_SOURCE) { + fakeContentProgressOffsetMs = contentDurationMs; + } + pendingAdPrepareErrorAdInfo = new AdInfo(adGroupIndex, adIndexInAdGroup); + } else { + AdMediaInfo adMediaInfo = checkNotNull(imaAdMediaInfo); + // We're already playing an ad. + if (adIndexInAdGroup > playingAdIndexInAdGroup) { + // Mark the playing ad as ended so we can notify the error on the next ad and remove it, + // which means that the ad after will load (if any). + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onEnded(adMediaInfo); + } + } + playingAdIndexInAdGroup = adPlaybackState.adGroups[adGroupIndex].getFirstAdIndexToPlay(); + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onError(checkNotNull(adMediaInfo)); + } + } + adPlaybackState = adPlaybackState.withAdLoadError(adGroupIndex, adIndexInAdGroup); + updateAdPlaybackState(); + } + + private void ensureSentContentCompleteIfAtEndOfStream() { + if (!sentContentComplete + && contentDurationMs != C.TIME_UNSET + && pendingContentPositionMs == C.TIME_UNSET + && getContentPeriodPositionMs(checkNotNull(player), timeline, period) + + THRESHOLD_END_OF_CONTENT_MS + >= contentDurationMs) { + sendContentComplete(); + } + } + + private void sendContentComplete() { + for (int i = 0; i < adCallbacks.size(); i++) { + adCallbacks.get(i).onContentComplete(); + } + sentContentComplete = true; + if (configuration.debugModeEnabled) { + Log.d(TAG, "adsLoader.contentComplete"); + } + for (int i = 0; i < adPlaybackState.adGroupCount; i++) { + if (adPlaybackState.adGroupTimesUs[i] != C.TIME_END_OF_SOURCE) { + adPlaybackState = adPlaybackState.withSkippedAdGroup(/* adGroupIndex= */ i); + } + } + updateAdPlaybackState(); + } + + private void updateAdPlaybackState() { + for (int i = 0; i < eventListeners.size(); i++) { + eventListeners.get(i).onAdPlaybackState(adPlaybackState); + } + } + + private void maybeNotifyPendingAdLoadError() { + if (pendingAdLoadError != null) { + for (int i = 0; i < eventListeners.size(); i++) { + eventListeners.get(i).onAdLoadError(pendingAdLoadError, adTagDataSpec); + } + pendingAdLoadError = null; + } + } + + private void maybeNotifyInternalError(String name, Exception cause) { + String message = "Internal error in " + name; + Log.e(TAG, message, cause); + // We can't recover from an unexpected error in general, so skip all remaining ads. + for (int i = 0; i < adPlaybackState.adGroupCount; i++) { + adPlaybackState = adPlaybackState.withSkippedAdGroup(i); + } + updateAdPlaybackState(); + for (int i = 0; i < eventListeners.size(); i++) { + eventListeners + .get(i) + .onAdLoadError( + AdLoadException.createForUnexpected(new RuntimeException(message, cause)), + adTagDataSpec); + } + } + + private int getAdGroupIndexForAdPod(AdPodInfo adPodInfo) { + if (adPodInfo.getPodIndex() == -1) { + // This is a postroll ad. + return adPlaybackState.adGroupCount - 1; + } + + // adPodInfo.podIndex may be 0-based or 1-based, so for now look up the cue point instead. + return getAdGroupIndexForCuePointTimeSeconds(adPodInfo.getTimeOffset()); + } + + /** + * Returns the index of the ad group that will preload next, or {@link C#INDEX_UNSET} if there is + * no such ad group. + */ + private int getLoadingAdGroupIndex() { + if (player == null) { + return C.INDEX_UNSET; + } + long playerPositionUs = C.msToUs(getContentPeriodPositionMs(player, timeline, period)); + int adGroupIndex = + adPlaybackState.getAdGroupIndexForPositionUs(playerPositionUs, C.msToUs(contentDurationMs)); + if (adGroupIndex == C.INDEX_UNSET) { + adGroupIndex = + adPlaybackState.getAdGroupIndexAfterPositionUs( + playerPositionUs, C.msToUs(contentDurationMs)); + } + return adGroupIndex; + } + + private int getAdGroupIndexForCuePointTimeSeconds(double cuePointTimeSeconds) { + // We receive initial cue points from IMA SDK as floats. This code replicates the same + // calculation used to populate adGroupTimesUs (having truncated input back to float, to avoid + // failures if the behavior of the IMA SDK changes to provide greater precision). + float cuePointTimeSecondsFloat = (float) cuePointTimeSeconds; + long adPodTimeUs = Math.round((double) cuePointTimeSecondsFloat * C.MICROS_PER_SECOND); + for (int adGroupIndex = 0; adGroupIndex < adPlaybackState.adGroupCount; adGroupIndex++) { + long adGroupTimeUs = adPlaybackState.adGroupTimesUs[adGroupIndex]; + if (adGroupTimeUs != C.TIME_END_OF_SOURCE + && Math.abs(adGroupTimeUs - adPodTimeUs) < THRESHOLD_AD_MATCH_US) { + return adGroupIndex; + } + } + throw new IllegalStateException("Failed to find cue point"); + } + + private String getAdMediaInfoString(@Nullable AdMediaInfo adMediaInfo) { + @Nullable AdInfo adInfo = adInfoByAdMediaInfo.get(adMediaInfo); + return "AdMediaInfo[" + + (adMediaInfo == null ? "null" : adMediaInfo.getUrl()) + + ", " + + adInfo + + "]"; + } + + private static long getContentPeriodPositionMs( + Player player, Timeline timeline, Timeline.Period period) { + long contentWindowPositionMs = player.getContentPosition(); + if (timeline.isEmpty()) { + return contentWindowPositionMs; + } else { + return contentWindowPositionMs + - timeline.getPeriod(player.getCurrentPeriodIndex(), period).getPositionInWindowMs(); + } + } + + private static boolean hasMidrollAdGroups(long[] adGroupTimesUs) { + int count = adGroupTimesUs.length; + if (count == 1) { + return adGroupTimesUs[0] != 0 && adGroupTimesUs[0] != C.TIME_END_OF_SOURCE; + } else if (count == 2) { + return adGroupTimesUs[0] != 0 || adGroupTimesUs[1] != C.TIME_END_OF_SOURCE; + } else { + // There's at least one midroll ad group, as adGroupTimesUs is never empty. + return true; + } + } + + private void destroyAdsManager() { + if (adsManager != null) { + adsManager.removeAdErrorListener(componentListener); + if (configuration.applicationAdErrorListener != null) { + adsManager.removeAdErrorListener(configuration.applicationAdErrorListener); + } + adsManager.removeAdEventListener(componentListener); + if (configuration.applicationAdEventListener != null) { + adsManager.removeAdEventListener(configuration.applicationAdEventListener); + } + adsManager.destroy(); + adsManager = null; + } + } + + private final class ComponentListener + implements AdsLoadedListener, + ContentProgressProvider, + AdEventListener, + AdErrorListener, + VideoAdPlayer { + + // AdsLoader.AdsLoadedListener implementation. + + @Override + public void onAdsManagerLoaded(AdsManagerLoadedEvent adsManagerLoadedEvent) { + AdsManager adsManager = adsManagerLoadedEvent.getAdsManager(); + if (!Util.areEqual(pendingAdRequestContext, adsManagerLoadedEvent.getUserRequestContext())) { + adsManager.destroy(); + return; + } + pendingAdRequestContext = null; + AdTagLoader.this.adsManager = adsManager; + adsManager.addAdErrorListener(this); + if (configuration.applicationAdErrorListener != null) { + adsManager.addAdErrorListener(configuration.applicationAdErrorListener); + } + adsManager.addAdEventListener(this); + if (configuration.applicationAdEventListener != null) { + adsManager.addAdEventListener(configuration.applicationAdEventListener); + } + try { + adPlaybackState = + new AdPlaybackState(adsId, getAdGroupTimesUsForCuePoints(adsManager.getAdCuePoints())); + updateAdPlaybackState(); + } catch (RuntimeException e) { + maybeNotifyInternalError("onAdsManagerLoaded", e); + } + } + + // ContentProgressProvider implementation. + + @Override + public VideoProgressUpdate getContentProgress() { + VideoProgressUpdate videoProgressUpdate = getContentVideoProgressUpdate(); + if (configuration.debugModeEnabled) { + Log.d( + TAG, + "Content progress: " + ImaUtil.getStringForVideoProgressUpdate(videoProgressUpdate)); + } + + if (waitingForPreloadElapsedRealtimeMs != C.TIME_UNSET) { + // IMA is polling the player position but we are buffering for an ad to preload, so playback + // may be stuck. Detect this case and signal an error if applicable. + long stuckElapsedRealtimeMs = + SystemClock.elapsedRealtime() - waitingForPreloadElapsedRealtimeMs; + if (stuckElapsedRealtimeMs >= THRESHOLD_AD_PRELOAD_MS) { + waitingForPreloadElapsedRealtimeMs = C.TIME_UNSET; + handleAdGroupLoadError(new IOException("Ad preloading timed out")); + maybeNotifyPendingAdLoadError(); + } + } else if (pendingContentPositionMs != C.TIME_UNSET + && player != null + && player.getPlaybackState() == Player.STATE_BUFFERING + && isWaitingForAdToLoad()) { + // Prepare to timeout the load of an ad for the pending seek operation. + waitingForPreloadElapsedRealtimeMs = SystemClock.elapsedRealtime(); + } + + return videoProgressUpdate; + } + + // AdEvent.AdEventListener implementation. + + @Override + public void onAdEvent(AdEvent adEvent) { + AdEventType adEventType = adEvent.getType(); + if (configuration.debugModeEnabled && adEventType != AdEventType.AD_PROGRESS) { + Log.d(TAG, "onAdEvent: " + adEventType); + } + try { + handleAdEvent(adEvent); + } catch (RuntimeException e) { + maybeNotifyInternalError("onAdEvent", e); + } + } + + // AdErrorEvent.AdErrorListener implementation. + + @Override + public void onAdError(AdErrorEvent adErrorEvent) { + AdError error = adErrorEvent.getError(); + if (configuration.debugModeEnabled) { + Log.d(TAG, "onAdError", error); + } + if (adsManager == null) { + // No ads were loaded, so allow playback to start without any ads. + pendingAdRequestContext = null; + adPlaybackState = new AdPlaybackState(adsId); + updateAdPlaybackState(); + } else if (ImaUtil.isAdGroupLoadError(error)) { + try { + handleAdGroupLoadError(error); + } catch (RuntimeException e) { + maybeNotifyInternalError("onAdError", e); + } + } + if (pendingAdLoadError == null) { + pendingAdLoadError = AdLoadException.createForAllAds(error); + } + maybeNotifyPendingAdLoadError(); + } + + // VideoAdPlayer implementation. + + @Override + public void addCallback(VideoAdPlayerCallback videoAdPlayerCallback) { + adCallbacks.add(videoAdPlayerCallback); + } + + @Override + public void removeCallback(VideoAdPlayerCallback videoAdPlayerCallback) { + adCallbacks.remove(videoAdPlayerCallback); + } + + @Override + public VideoProgressUpdate getAdProgress() { + throw new IllegalStateException("Unexpected call to getAdProgress when using preloading"); + } + + @Override + public int getVolume() { + return getPlayerVolumePercent(); + } + + @Override + public void loadAd(AdMediaInfo adMediaInfo, AdPodInfo adPodInfo) { + try { + loadAdInternal(adMediaInfo, adPodInfo); + } catch (RuntimeException e) { + maybeNotifyInternalError("loadAd", e); + } + } + + @Override + public void playAd(AdMediaInfo adMediaInfo) { + try { + playAdInternal(adMediaInfo); + } catch (RuntimeException e) { + maybeNotifyInternalError("playAd", e); + } + } + + @Override + public void pauseAd(AdMediaInfo adMediaInfo) { + try { + pauseAdInternal(adMediaInfo); + } catch (RuntimeException e) { + maybeNotifyInternalError("pauseAd", e); + } + } + + @Override + public void stopAd(AdMediaInfo adMediaInfo) { + try { + stopAdInternal(adMediaInfo); + } catch (RuntimeException e) { + maybeNotifyInternalError("stopAd", e); + } + } + + @Override + public void release() { + // Do nothing. + } + } + + // TODO: Consider moving this into AdPlaybackState. + private static final class AdInfo { + + public final int adGroupIndex; + public final int adIndexInAdGroup; + + public AdInfo(int adGroupIndex, int adIndexInAdGroup) { + this.adGroupIndex = adGroupIndex; + this.adIndexInAdGroup = adIndexInAdGroup; + } + + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + AdInfo adInfo = (AdInfo) o; + if (adGroupIndex != adInfo.adGroupIndex) { + return false; + } + return adIndexInAdGroup == adInfo.adIndexInAdGroup; + } + + @Override + public int hashCode() { + int result = adGroupIndex; + result = 31 * result + adIndexInAdGroup; + return result; + } + + @Override + public String toString() { + return "(" + adGroupIndex + ", " + adIndexInAdGroup + ')'; + } + } +} diff --git a/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/ImaAdsLoader.java b/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/ImaAdsLoader.java index 574fbfa9175..e2adbaf2d02 100644 --- a/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/ImaAdsLoader.java +++ b/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/ImaAdsLoader.java @@ -15,33 +15,23 @@ */ package com.google.android.exoplayer2.ext.ima; +import static com.google.android.exoplayer2.ext.ima.ImaUtil.BITRATE_UNSET; +import static com.google.android.exoplayer2.ext.ima.ImaUtil.TIMEOUT_UNSET; +import static com.google.android.exoplayer2.ext.ima.ImaUtil.getImaLooper; import static com.google.android.exoplayer2.util.Assertions.checkArgument; import static com.google.android.exoplayer2.util.Assertions.checkNotNull; import static com.google.android.exoplayer2.util.Assertions.checkState; -import static java.lang.Math.max; import android.content.Context; -import android.net.Uri; -import android.os.Handler; import android.os.Looper; -import android.os.SystemClock; import android.view.View; import android.view.ViewGroup; -import androidx.annotation.IntDef; import androidx.annotation.Nullable; import androidx.annotation.VisibleForTesting; import com.google.ads.interactivemedia.v3.api.AdDisplayContainer; -import com.google.ads.interactivemedia.v3.api.AdError; -import com.google.ads.interactivemedia.v3.api.AdErrorEvent; import com.google.ads.interactivemedia.v3.api.AdErrorEvent.AdErrorListener; -import com.google.ads.interactivemedia.v3.api.AdEvent; import com.google.ads.interactivemedia.v3.api.AdEvent.AdEventListener; -import com.google.ads.interactivemedia.v3.api.AdEvent.AdEventType; -import com.google.ads.interactivemedia.v3.api.AdPodInfo; -import com.google.ads.interactivemedia.v3.api.AdsLoader; -import com.google.ads.interactivemedia.v3.api.AdsLoader.AdsLoadedListener; import com.google.ads.interactivemedia.v3.api.AdsManager; -import com.google.ads.interactivemedia.v3.api.AdsManagerLoadedEvent; import com.google.ads.interactivemedia.v3.api.AdsRenderingSettings; import com.google.ads.interactivemedia.v3.api.AdsRequest; import com.google.ads.interactivemedia.v3.api.CompanionAdSlot; @@ -50,45 +40,30 @@ import com.google.ads.interactivemedia.v3.api.ImaSdkFactory; import com.google.ads.interactivemedia.v3.api.ImaSdkSettings; import com.google.ads.interactivemedia.v3.api.UiElement; -import com.google.ads.interactivemedia.v3.api.player.AdMediaInfo; -import com.google.ads.interactivemedia.v3.api.player.ContentProgressProvider; import com.google.ads.interactivemedia.v3.api.player.VideoAdPlayer; -import com.google.ads.interactivemedia.v3.api.player.VideoProgressUpdate; import com.google.android.exoplayer2.C; -import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.source.MediaSourceFactory; -import com.google.android.exoplayer2.source.ads.AdPlaybackState; +import com.google.android.exoplayer2.source.ads.AdsLoader; import com.google.android.exoplayer2.source.ads.AdsMediaSource; -import com.google.android.exoplayer2.source.ads.AdsMediaSource.AdLoadException; -import com.google.android.exoplayer2.trackselection.TrackSelectionArray; import com.google.android.exoplayer2.upstream.DataSpec; -import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.Util; -import com.google.common.collect.BiMap; -import com.google.common.collect.HashBiMap; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import java.io.IOException; -import java.lang.annotation.Documented; -import java.lang.annotation.Retention; -import java.lang.annotation.RetentionPolicy; import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; import java.util.Collections; +import java.util.HashMap; import java.util.List; -import java.util.Map; -import java.util.Locale; import java.util.Set; -import org.checkerframework.checker.nullness.qual.MonotonicNonNull; /** - * {@link com.google.android.exoplayer2.source.ads.AdsLoader} using the IMA SDK. All methods must be - * called on the main thread. + * {@link AdsLoader} using the IMA SDK. All methods must be called on the main thread. * *

The player instance that will play the loaded ads must be set before playback using {@link * #setPlayer(Player)}. If the ads loader is no longer required, it must be released by calling @@ -109,8 +84,7 @@ * href="https://developers.google.com/interactive-media-ads/docs/sdks/android/client-side/omsdk">IMA * SDK Open Measurement documentation for more information. */ -public final class ImaAdsLoader - implements Player.EventListener, com.google.android.exoplayer2.source.ads.AdsLoader { +public final class ImaAdsLoader implements Player.EventListener, AdsLoader { static { ExoPlayerLibraryInfo.registerModule("goog.exo.ima"); @@ -139,6 +113,7 @@ public static final class Builder { @Nullable private List adMediaMimeTypes; @Nullable private Set adUiElements; @Nullable private Collection companionAdSlots; + @Nullable private Boolean enableContinuousPlayback; private long adPreloadTimeoutMs; private int vastLoadTimeoutMs; private int mediaLoadTimeoutMs; @@ -180,8 +155,8 @@ public Builder setImaSdkSettings(ImaSdkSettings imaSdkSettings) { /** * Sets a listener for ad errors that will be passed to {@link - * AdsLoader#addAdErrorListener(AdErrorListener)} and {@link - * AdsManager#addAdErrorListener(AdErrorListener)}. + * com.google.ads.interactivemedia.v3.api.AdsLoader#addAdErrorListener(AdErrorListener)} and + * {@link AdsManager#addAdErrorListener(AdErrorListener)}. * * @param adErrorListener The ad error listener. * @return This builder, for convenience. @@ -260,6 +235,20 @@ public Builder setAdMediaMimeTypes(List adMediaMimeTypes) { return this; } + /** + * Sets whether to enable continuous playback. Pass {@code true} if content videos will be + * played continuously, similar to a TV broadcast. This setting may modify the ads request but + * does not affect ad playback behavior. The requested value is unknown by default. + * + * @param enableContinuousPlayback Whether to enable continuous playback. + * @return This builder, for convenience. + * @see AdsRequest#setContinuousPlayback(boolean) + */ + public Builder setEnableContinuousPlayback(boolean enableContinuousPlayback) { + this.enableContinuousPlayback = enableContinuousPlayback; + return this; + } + /** * Sets the duration in milliseconds for which the player must buffer while preloading an ad * group before that ad group is skipped and marked as having failed to load. Pass {@link @@ -368,285 +357,63 @@ public Builder setDebugModeEnabled(boolean debugModeEnabled) { return this; } - /** - * Returns a new {@link ImaAdsLoader} for the specified ad tag. - * - * @param adTagUri The URI of a compatible ad tag to load. See - * https://developers.google.com/interactive-media-ads/docs/sdks/android/compatibility for - * information on compatible ad tags. - * @return The new {@link ImaAdsLoader}. - * @deprecated Pass the ad tag URI when setting media item playback properties (if using the - * media item API) or as a {@link DataSpec} when constructing the {@link AdsMediaSource} (if - * using media sources directly). - */ - @Deprecated - public ImaAdsLoader buildForAdTag(Uri adTagUri) { - return new ImaAdsLoader( - context, - getConfiguration(), - imaFactory, - /* adTagUri= */ adTagUri, - /* adsResponse= */ null); - } - - /** - * Returns a new {@link ImaAdsLoader} with the specified sideloaded ads response. - * - * @param adsResponse The sideloaded VAST, VMAP, or ad rules response to be used instead of - * making a request via an ad tag URL. - * @return The new {@link ImaAdsLoader}. - * @deprecated Pass the ads response as a data URI when setting media item playback properties - * (if using the media item API) or as a {@link DataSpec} when constructing the {@link - * AdsMediaSource} (if using media sources directly). {@link - * Util#getDataUriForString(String, String)} can be used to construct a data URI from - * literal string ads response (with MIME type text/xml). - */ - @Deprecated - public ImaAdsLoader buildForAdsResponse(String adsResponse) { - return new ImaAdsLoader( - context, getConfiguration(), imaFactory, /* adTagUri= */ null, adsResponse); - } - /** Returns a new {@link ImaAdsLoader}. */ public ImaAdsLoader build() { return new ImaAdsLoader( - context, getConfiguration(), imaFactory, /* adTagUri= */ null, /* adsResponse= */ null); - } - - // TODO(internal: b/169646419): Remove/hide once the deprecated constructor has been removed. - /* package */ ImaUtil.Configuration getConfiguration() { - return new ImaUtil.Configuration( - adPreloadTimeoutMs, - vastLoadTimeoutMs, - mediaLoadTimeoutMs, - focusSkipButtonWhenAvailable, - playAdBeforeStartPosition, - mediaBitrate, - adMediaMimeTypes, - adUiElements, - companionAdSlots, - adErrorListener, - adEventListener, - videoAdPlayerCallback, - imaSdkSettings, - debugModeEnabled); + context, + new ImaUtil.Configuration( + adPreloadTimeoutMs, + vastLoadTimeoutMs, + mediaLoadTimeoutMs, + focusSkipButtonWhenAvailable, + playAdBeforeStartPosition, + mediaBitrate, + enableContinuousPlayback, + adMediaMimeTypes, + adUiElements, + companionAdSlots, + adErrorListener, + adEventListener, + videoAdPlayerCallback, + imaSdkSettings, + debugModeEnabled), + imaFactory); } } - private static final String TAG = "ImaAdsLoader"; - - private static final String IMA_SDK_SETTINGS_PLAYER_TYPE = "google/exo.ext.ima"; - private static final String IMA_SDK_SETTINGS_PLAYER_VERSION = ExoPlayerLibraryInfo.VERSION; - - /** - * Interval at which ad progress updates are provided to the IMA SDK, in milliseconds. 100 ms is - * the interval recommended by the IMA documentation. - * - * @see VideoAdPlayer.VideoAdPlayerCallback - */ - private static final int AD_PROGRESS_UPDATE_INTERVAL_MS = 100; - - /** The value used in {@link VideoProgressUpdate}s to indicate an unset duration. */ - private static final long IMA_DURATION_UNSET = -1L; - - /** - * Threshold before the end of content at which IMA is notified that content is complete if the - * player buffers, in milliseconds. - */ - private static final long THRESHOLD_END_OF_CONTENT_MS = 5000; - /** - * Threshold before the start of an ad at which IMA is expected to be able to preload the ad, in - * milliseconds. - */ - private static final long THRESHOLD_AD_PRELOAD_MS = 4000; - /** The threshold below which ad cue points are treated as matching, in microseconds. */ - private static final long THRESHOLD_AD_MATCH_US = 1000; - - private static final int TIMEOUT_UNSET = -1; - private static final int BITRATE_UNSET = -1; - - /** The state of ad playback. */ - @Documented - @Retention(RetentionPolicy.SOURCE) - @IntDef({IMA_AD_STATE_NONE, IMA_AD_STATE_PLAYING, IMA_AD_STATE_PAUSED}) - private @interface ImaAdState {} - /** The ad playback state when IMA is not playing an ad. */ - private static final int IMA_AD_STATE_NONE = 0; - /** - * The ad playback state when IMA has called {@link ComponentListener#playAd(AdMediaInfo)} and not - * {@link ComponentListener##pauseAd(AdMediaInfo)}. - */ - private static final int IMA_AD_STATE_PLAYING = 1; - /** - * The ad playback state when IMA has called {@link ComponentListener#pauseAd(AdMediaInfo)} while - * playing an ad. - */ - private static final int IMA_AD_STATE_PAUSED = 2; - - private static final DataSpec EMPTY_AD_TAG_DATA_SPEC = new DataSpec(Uri.EMPTY); - private final ImaUtil.Configuration configuration; private final Context context; private final ImaUtil.ImaFactory imaFactory; - @Nullable private final Uri adTagUri; - @Nullable private final String adsResponse; - private final ImaSdkSettings imaSdkSettings; + private final HashMap adTagLoaderByAdsId; + private final HashMap adTagLoaderByAdsMediaSource; private final Timeline.Period period; - private final Handler handler; - private final ComponentListener componentListener; - private final List adCallbacks; - private final Runnable updateAdProgressRunnable; - private final BiMap adInfoByAdMediaInfo; + private final Timeline.Window window; - private @MonotonicNonNull AdDisplayContainer adDisplayContainer; - private @MonotonicNonNull AdsLoader adsLoader; private boolean wasSetPlayerCalled; @Nullable private Player nextPlayer; - @Nullable private Object pendingAdRequestContext; private List supportedMimeTypes; - @Nullable private EventListener eventListener; @Nullable private Player player; - private DataSpec adTagDataSpec; - private VideoProgressUpdate lastContentProgress; - private VideoProgressUpdate lastAdProgress; - private int lastVolumePercent; - - @Nullable private AdsManager adsManager; - private boolean isAdsManagerInitialized; - private boolean hasAdPlaybackState; - @Nullable private AdLoadException pendingAdLoadError; - private Timeline timeline; - private long contentDurationMs; - private AdPlaybackState adPlaybackState; - - // Fields tracking IMA's state. - - /** Whether IMA has sent an ad event to pause content since the last resume content event. */ - private boolean imaPausedContent; - /** The current ad playback state. */ - private @ImaAdState int imaAdState; - /** The current ad media info, or {@code null} if in state {@link #IMA_AD_STATE_NONE}. */ - @Nullable private AdMediaInfo imaAdMediaInfo; - /** The current ad info, or {@code null} if in state {@link #IMA_AD_STATE_NONE}. */ - @Nullable private AdInfo imaAdInfo; - /** Whether IMA has been notified that playback of content has finished. */ - private boolean sentContentComplete; - - // Fields tracking the player/loader state. - - /** Whether the player is playing an ad. */ - private boolean playingAd; - /** Whether the player is buffering an ad. */ - private boolean bufferingAd; - /** - * If the player is playing an ad, stores the ad index in its ad group. {@link C#INDEX_UNSET} - * otherwise. - */ - private int playingAdIndexInAdGroup; - /** - * The ad info for a pending ad for which the media failed preparation, or {@code null} if no - * pending ads have failed to prepare. - */ - @Nullable private AdInfo pendingAdPrepareErrorAdInfo; - /** - * If a content period has finished but IMA has not yet called {@link - * ComponentListener#playAd(AdMediaInfo)}, stores the value of {@link - * SystemClock#elapsedRealtime()} when the content stopped playing. This can be used to determine - * a fake, increasing content position. {@link C#TIME_UNSET} otherwise. - */ - private long fakeContentProgressElapsedRealtimeMs; - /** - * If {@link #fakeContentProgressElapsedRealtimeMs} is set, stores the offset from which the - * content progress should increase. {@link C#TIME_UNSET} otherwise. - */ - private long fakeContentProgressOffsetMs; - /** Stores the pending content position when a seek operation was intercepted to play an ad. */ - private long pendingContentPositionMs; - /** - * Whether {@link ComponentListener#getContentProgress()} has sent {@link - * #pendingContentPositionMs} to IMA. - */ - private boolean sentPendingContentPositionMs; - /** - * Stores the real time in milliseconds at which the player started buffering, possibly due to not - * having preloaded an ad, or {@link C#TIME_UNSET} if not applicable. - */ - private long waitingForPreloadElapsedRealtimeMs; + @Nullable private AdTagLoader currentAdTagLoader; - /** - * Creates a new IMA ads loader. - * - *

If you need to customize the ad request, use {@link ImaAdsLoader.Builder} instead. - * - * @param context The context. - * @param adTagUri The {@link Uri} of an ad tag compatible with the Android IMA SDK. See - * https://developers.google.com/interactive-media-ads/docs/sdks/android/compatibility for - * more information. - * @deprecated Use {@link Builder} to create an instance. Pass the ad tag URI when setting media - * item playback properties (if using the media item API) or as a {@link DataSpec} when - * constructing the {@link AdsMediaSource} (if using media sources directly). - */ - @Deprecated - public ImaAdsLoader(Context context, Uri adTagUri) { - this( - context, - new Builder(context).getConfiguration(), - new DefaultImaFactory(), - adTagUri, - /* adsResponse= */ null); - } - - @SuppressWarnings({"nullness:argument.type.incompatible", "methodref.receiver.bound.invalid"}) private ImaAdsLoader( - Context context, - ImaUtil.Configuration configuration, - ImaUtil.ImaFactory imaFactory, - @Nullable Uri adTagUri, - @Nullable String adsResponse) { + Context context, ImaUtil.Configuration configuration, ImaUtil.ImaFactory imaFactory) { this.context = context.getApplicationContext(); this.configuration = configuration; this.imaFactory = imaFactory; - this.adTagUri = adTagUri; - this.adsResponse = adsResponse; - @Nullable ImaSdkSettings imaSdkSettings = configuration.imaSdkSettings; - if (imaSdkSettings == null) { - imaSdkSettings = imaFactory.createImaSdkSettings(); - if (configuration.debugModeEnabled) { - imaSdkSettings.setDebugMode(true); - } - } - imaSdkSettings.setPlayerType(IMA_SDK_SETTINGS_PLAYER_TYPE); - imaSdkSettings.setPlayerVersion(IMA_SDK_SETTINGS_PLAYER_VERSION); - this.imaSdkSettings = imaSdkSettings; + supportedMimeTypes = ImmutableList.of(); + adTagLoaderByAdsId = new HashMap<>(); + adTagLoaderByAdsMediaSource = new HashMap<>(); period = new Timeline.Period(); - handler = Util.createHandler(getImaLooper(), /* callback= */ null); - componentListener = new ComponentListener(); - adCallbacks = new ArrayList<>(/* initialCapacity= */ 1); - if (configuration.applicationVideoAdPlayerCallback != null) { - adCallbacks.add(configuration.applicationVideoAdPlayerCallback); - } - updateAdProgressRunnable = this::updateAdProgress; - adInfoByAdMediaInfo = HashBiMap.create(); - supportedMimeTypes = Collections.emptyList(); - adTagDataSpec = EMPTY_AD_TAG_DATA_SPEC; - lastContentProgress = VideoProgressUpdate.VIDEO_TIME_NOT_READY; - lastAdProgress = VideoProgressUpdate.VIDEO_TIME_NOT_READY; - fakeContentProgressElapsedRealtimeMs = C.TIME_UNSET; - fakeContentProgressOffsetMs = C.TIME_UNSET; - pendingContentPositionMs = C.TIME_UNSET; - waitingForPreloadElapsedRealtimeMs = C.TIME_UNSET; - contentDurationMs = C.TIME_UNSET; - timeline = Timeline.EMPTY; - adPlaybackState = AdPlaybackState.NONE; + window = new Timeline.Window(); } /** - * Returns the underlying {@link AdsLoader} wrapped by this instance, or {@code null} if ads have - * not been requested yet. + * Returns the underlying {@link com.google.ads.interactivemedia.v3.api.AdsLoader} wrapped by this + * instance, or {@code null} if ads have not been requested yet. */ @Nullable - public AdsLoader getAdsLoader() { - return adsLoader; + public com.google.ads.interactivemedia.v3.api.AdsLoader getAdsLoader() { + return currentAdTagLoader != null ? currentAdTagLoader.getAdsLoader() : null; } /** @@ -657,30 +424,12 @@ public AdsLoader getAdsLoader() { * AdDisplayContainer#registerFriendlyObstruction(FriendlyObstruction)} will be unregistered * automatically when the media source detaches from this instance. It is therefore necessary to * re-register views each time the ads loader is reused. Alternatively, provide overlay views via - * the {@link com.google.android.exoplayer2.source.ads.AdsLoader.AdViewProvider} when creating the - * media source to benefit from automatic registration. + * the {@link AdViewProvider} when creating the media source to benefit from automatic + * registration. */ @Nullable public AdDisplayContainer getAdDisplayContainer() { - return adDisplayContainer; - } - - /** - * Requests ads, if they have not already been requested. Must be called on the main thread. - * - *

Ads will be requested automatically when the player is prepared if this method has not been - * called, so it is only necessary to call this method if you want to request ads before preparing - * the player. - * - * @param adViewGroup A {@link ViewGroup} on top of the player that will show any ad UI, or {@code - * null} if playing audio-only ads. - * @deprecated Use {@link #requestAds(DataSpec, ViewGroup)}, specifying the ad tag data spec to - * request, and migrate off deprecated builder methods/constructor that require an ad tag or - * ads response. - */ - @Deprecated - public void requestAds(@Nullable ViewGroup adViewGroup) { - requestAds(adTagDataSpec, adViewGroup); + return currentAdTagLoader != null ? currentAdTagLoader.getAdDisplayContainer() : null; } /** @@ -692,64 +441,23 @@ public void requestAds(@Nullable ViewGroup adViewGroup) { * * @param adTagDataSpec The data specification of the ad tag to load. See class javadoc for * information about compatible ad tag formats. + * @param adsId A opaque identifier for the ad playback state across start/stop calls. * @param adViewGroup A {@link ViewGroup} on top of the player that will show any ad UI, or {@code * null} if playing audio-only ads. */ - public void requestAds(DataSpec adTagDataSpec, @Nullable ViewGroup adViewGroup) { - if (hasAdPlaybackState || adsManager != null || pendingAdRequestContext != null) { - // Ads have already been requested. - return; - } - - if (EMPTY_AD_TAG_DATA_SPEC.equals(adTagDataSpec)) { - // Handle deprecated ways of specifying the ad tag. - if (adTagUri != null) { - adTagDataSpec = new DataSpec(adTagUri); - } else if (adsResponse != null) { - adTagDataSpec = - new DataSpec( - Util.getDataUriForString(/* mimeType= */ "text/xml", /* data= */ adsResponse)); - } else { - throw new IllegalStateException(); - } - } - - AdsRequest request; - try { - request = ImaUtil.getAdsRequestForAdTagDataSpec(imaFactory, adTagDataSpec); - } catch (IOException e) { - hasAdPlaybackState = true; - updateAdPlaybackState(); - pendingAdLoadError = AdLoadException.createForAllAds(e); - maybeNotifyPendingAdLoadError(); - return; - } - this.adTagDataSpec = adTagDataSpec; - pendingAdRequestContext = new Object(); - request.setUserRequestContext(pendingAdRequestContext); - if (configuration.vastLoadTimeoutMs != TIMEOUT_UNSET) { - request.setVastLoadTimeout(configuration.vastLoadTimeoutMs); - } - request.setContentProgressProvider(componentListener); - - if (adViewGroup != null) { - adDisplayContainer = - imaFactory.createAdDisplayContainer(adViewGroup, /* player= */ componentListener); - } else { - adDisplayContainer = - imaFactory.createAudioAdDisplayContainer(context, /* player= */ componentListener); - } - if (configuration.companionAdSlots != null) { - adDisplayContainer.setCompanionSlots(configuration.companionAdSlots); - } - - adsLoader = imaFactory.createAdsLoader(context, imaSdkSettings, adDisplayContainer); - adsLoader.addAdErrorListener(componentListener); - if (configuration.applicationAdErrorListener != null) { - adsLoader.addAdErrorListener(configuration.applicationAdErrorListener); + public void requestAds(DataSpec adTagDataSpec, Object adsId, @Nullable ViewGroup adViewGroup) { + if (!adTagLoaderByAdsId.containsKey(adsId)) { + AdTagLoader adTagLoader = + new AdTagLoader( + context, + configuration, + imaFactory, + supportedMimeTypes, + adTagDataSpec, + adsId, + adViewGroup); + adTagLoaderByAdsId.put(adsId, adTagLoader); } - adsLoader.addAdsLoadedListener(componentListener); - adsLoader.requestAds(request); } /** @@ -760,12 +468,12 @@ public void requestAds(DataSpec adTagDataSpec, @Nullable ViewGroup adViewGroup) * IMA SDK provides the UI to skip ads in the ad view group passed via {@link AdViewProvider}. */ public void skipAd() { - if (adsManager != null) { - adsManager.skip(); + if (currentAdTagLoader != null) { + currentAdTagLoader.skipAd(); } } - // com.google.android.exoplayer2.source.ads.AdsLoader implementation. + // AdsLoader implementation. @Override public void setPlayer(@Nullable Player player) { @@ -798,121 +506,88 @@ public void setSupportedContentTypes(@C.ContentType int... contentTypes) { } @Override - public void setAdTagDataSpec(DataSpec adTagDataSpec) { - this.adTagDataSpec = adTagDataSpec; - } - - @Override - public void start(EventListener eventListener, AdViewProvider adViewProvider) { + public void start( + AdsMediaSource adsMediaSource, + DataSpec adTagDataSpec, + Object adsId, + AdViewProvider adViewProvider, + EventListener eventListener) { checkState( wasSetPlayerCalled, "Set player using adsLoader.setPlayer before preparing the player."); - player = nextPlayer; - if (player == null) { - return; - } - player.addListener(this); - boolean playWhenReady = player.getPlayWhenReady(); - this.eventListener = eventListener; - lastVolumePercent = 0; - lastAdProgress = VideoProgressUpdate.VIDEO_TIME_NOT_READY; - lastContentProgress = VideoProgressUpdate.VIDEO_TIME_NOT_READY; - maybeNotifyPendingAdLoadError(); - if (hasAdPlaybackState) { - // Pass the ad playback state to the player, and resume ads if necessary. - eventListener.onAdPlaybackState(adPlaybackState); - if (adsManager != null && imaPausedContent && playWhenReady) { - adsManager.resume(); + if (adTagLoaderByAdsMediaSource.isEmpty()) { + player = nextPlayer; + @Nullable Player player = this.player; + if (player == null) { + return; } - } else if (adsManager != null) { - adPlaybackState = ImaUtil.getInitialAdPlaybackStateForCuePoints(adsManager.getAdCuePoints()); - updateAdPlaybackState(); - } else { - // Ads haven't loaded yet, so request them. - requestAds(adTagDataSpec, adViewProvider.getAdViewGroup()); + player.addListener(this); } - if (adDisplayContainer != null) { - for (OverlayInfo overlayInfo : adViewProvider.getAdOverlayInfos()) { - adDisplayContainer.registerFriendlyObstruction( - imaFactory.createFriendlyObstruction( - overlayInfo.view, - ImaUtil.getFriendlyObstructionPurpose(overlayInfo.purpose), - overlayInfo.reasonDetail)); - } + + @Nullable AdTagLoader adTagLoader = adTagLoaderByAdsId.get(adsId); + if (adTagLoader == null) { + requestAds(adTagDataSpec, adsId, adViewProvider.getAdViewGroup()); + adTagLoader = adTagLoaderByAdsId.get(adsId); } + adTagLoaderByAdsMediaSource.put(adsMediaSource, checkNotNull(adTagLoader)); + adTagLoader.addListenerWithAdView(eventListener, adViewProvider); + maybeUpdateCurrentAdTagLoader(); } @Override - public void stop() { - @Nullable Player player = this.player; - if (player == null) { - return; + public void stop(AdsMediaSource adsMediaSource, EventListener eventListener) { + @Nullable AdTagLoader removedAdTagLoader = adTagLoaderByAdsMediaSource.remove(adsMediaSource); + maybeUpdateCurrentAdTagLoader(); + if (removedAdTagLoader != null) { + removedAdTagLoader.removeListener(eventListener); } - if (adsManager != null && imaPausedContent) { - adsManager.pause(); - adPlaybackState = - adPlaybackState.withAdResumePositionUs( - playingAd ? C.msToUs(player.getCurrentPosition()) : 0); - } - lastVolumePercent = getPlayerVolumePercent(); - lastAdProgress = getAdVideoProgressUpdate(); - lastContentProgress = getContentVideoProgressUpdate(); - if (adDisplayContainer != null) { - adDisplayContainer.unregisterAllFriendlyObstructions(); + + if (player != null && adTagLoaderByAdsMediaSource.isEmpty()) { + player.removeListener(this); + player = null; } - player.removeListener(this); - this.player = null; - eventListener = null; } @Override public void release() { - pendingAdRequestContext = null; - destroyAdsManager(); - if (adsLoader != null) { - adsLoader.removeAdsLoadedListener(componentListener); - adsLoader.removeAdErrorListener(componentListener); - if (configuration.applicationAdErrorListener != null) { - adsLoader.removeAdErrorListener(configuration.applicationAdErrorListener); - } - adsLoader.release(); + if (player != null) { + player.removeListener(this); + player = null; + maybeUpdateCurrentAdTagLoader(); } - imaPausedContent = false; - imaAdState = IMA_AD_STATE_NONE; - imaAdMediaInfo = null; - stopUpdatingAdProgress(); - imaAdInfo = null; - pendingAdLoadError = null; - adPlaybackState = AdPlaybackState.NONE; - hasAdPlaybackState = true; - updateAdPlaybackState(); - } + nextPlayer = null; - @Override - public void handlePrepareComplete(int adGroupIndex, int adIndexInAdGroup) { - AdInfo adInfo = new AdInfo(adGroupIndex, adIndexInAdGroup); - if (configuration.debugModeEnabled) { - Log.d(TAG, "Prepared ad " + adInfo); + for (AdTagLoader adTagLoader : adTagLoaderByAdsMediaSource.values()) { + adTagLoader.release(); } - @Nullable AdMediaInfo adMediaInfo = adInfoByAdMediaInfo.inverse().get(adInfo); - if (adMediaInfo != null) { - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onLoaded(adMediaInfo); - } - } else { - Log.w(TAG, "Unexpected prepared ad " + adInfo); + adTagLoaderByAdsMediaSource.clear(); + + for (AdTagLoader adTagLoader : adTagLoaderByAdsId.values()) { + adTagLoader.release(); } + adTagLoaderByAdsId.clear(); } @Override - public void handlePrepareError(int adGroupIndex, int adIndexInAdGroup, IOException exception) { + public void handlePrepareComplete( + AdsMediaSource adsMediaSource, int adGroupIndex, int adIndexInAdGroup) { if (player == null) { return; } - try { - handleAdPrepareError(adGroupIndex, adIndexInAdGroup, exception); - } catch (RuntimeException e) { - maybeNotifyInternalError("handlePrepareError", e); + checkNotNull(adTagLoaderByAdsMediaSource.get(adsMediaSource)) + .handlePrepareComplete(adGroupIndex, adIndexInAdGroup); + } + + @Override + public void handlePrepareError( + AdsMediaSource adsMediaSource, + int adGroupIndex, + int adIndexInAdGroup, + IOException exception) { + if (player == null) { + return; } + checkNotNull(adTagLoaderByAdsMediaSource.get(adsMediaSource)) + .handlePrepareError(adGroupIndex, adIndexInAdGroup, exception); } // Player.EventListener implementation. @@ -923,988 +598,97 @@ public void onTimelineChanged(Timeline timeline, @Player.TimelineChangeReason in // The player is being reset or contains no media. return; } - checkArgument(timeline.getPeriodCount() == 1); - this.timeline = timeline; - long contentDurationUs = timeline.getPeriod(/* periodIndex= */ 0, period).durationUs; - contentDurationMs = C.usToMs(contentDurationUs); - if (contentDurationUs != C.TIME_UNSET) { - adPlaybackState = adPlaybackState.withContentDurationUs(contentDurationUs); - } - @Nullable AdsManager adsManager = this.adsManager; - if (!isAdsManagerInitialized && adsManager != null) { - isAdsManagerInitialized = true; - @Nullable AdsRenderingSettings adsRenderingSettings = setupAdsRendering(); - if (adsRenderingSettings == null) { - // There are no ads to play. - destroyAdsManager(); - } else { - adsManager.init(adsRenderingSettings); - adsManager.start(); - if (configuration.debugModeEnabled) { - Log.d(TAG, "Initialized with ads rendering settings: " + adsRenderingSettings); - } - } - updateAdPlaybackState(); - } - handleTimelineOrPositionChanged(); + maybeUpdateCurrentAdTagLoader(); + maybePreloadNextPeriodAds(); } @Override public void onPositionDiscontinuity(@Player.DiscontinuityReason int reason) { - handleTimelineOrPositionChanged(); + maybeUpdateCurrentAdTagLoader(); + maybePreloadNextPeriodAds(); } @Override - public void onPlaybackStateChanged(@Player.State int playbackState) { - @Nullable Player player = this.player; - if (adsManager == null || player == null) { - return; - } - - if (playbackState == Player.STATE_BUFFERING && !player.isPlayingAd()) { - // Check whether we are waiting for an ad to preload. - int adGroupIndex = getLoadingAdGroupIndex(); - if (adGroupIndex == C.INDEX_UNSET) { - return; - } - AdPlaybackState.AdGroup adGroup = adPlaybackState.adGroups[adGroupIndex]; - if (adGroup.count != C.LENGTH_UNSET - && adGroup.count != 0 - && adGroup.states[0] != AdPlaybackState.AD_STATE_UNAVAILABLE) { - // An ad is available already so we must be buffering for some other reason. - return; - } - long adGroupTimeMs = C.usToMs(adPlaybackState.adGroupTimesUs[adGroupIndex]); - long contentPositionMs = getContentPeriodPositionMs(player, timeline, period); - long timeUntilAdMs = adGroupTimeMs - contentPositionMs; - if (timeUntilAdMs < configuration.adPreloadTimeoutMs) { - waitingForPreloadElapsedRealtimeMs = SystemClock.elapsedRealtime(); - } - } else if (playbackState == Player.STATE_READY) { - waitingForPreloadElapsedRealtimeMs = C.TIME_UNSET; - } - - handlePlayerStateChanged(player.getPlayWhenReady(), playbackState); - } - - @Override - public void onPlayWhenReadyChanged( - boolean playWhenReady, @Player.PlayWhenReadyChangeReason int reason) { - if (adsManager == null || player == null) { - return; - } - - if (imaAdState == IMA_AD_STATE_PLAYING && !playWhenReady) { - adsManager.pause(); - return; - } - - if (imaAdState == IMA_AD_STATE_PAUSED && playWhenReady) { - adsManager.resume(); - return; - } - handlePlayerStateChanged(playWhenReady, player.getPlaybackState()); + public void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) { + maybePreloadNextPeriodAds(); } @Override - public void onPlayerError(ExoPlaybackException error) { - if (imaAdState != IMA_AD_STATE_NONE) { - AdMediaInfo adMediaInfo = checkNotNull(imaAdMediaInfo); - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onError(adMediaInfo); - } - } + public void onRepeatModeChanged(@Player.RepeatMode int repeatMode) { + maybePreloadNextPeriodAds(); } // Internal methods. - /** - * Configures ads rendering for starting playback, returning the settings for the IMA SDK or - * {@code null} if no ads should play. - */ - @Nullable - private AdsRenderingSettings setupAdsRendering() { - AdsRenderingSettings adsRenderingSettings = imaFactory.createAdsRenderingSettings(); - adsRenderingSettings.setEnablePreloading(true); - adsRenderingSettings.setMimeTypes( - configuration.adMediaMimeTypes != null - ? configuration.adMediaMimeTypes - : supportedMimeTypes); - if (configuration.mediaLoadTimeoutMs != TIMEOUT_UNSET) { - adsRenderingSettings.setLoadVideoTimeout(configuration.mediaLoadTimeoutMs); - } - if (configuration.mediaBitrate != BITRATE_UNSET) { - adsRenderingSettings.setBitrateKbps(configuration.mediaBitrate / 1000); - } - adsRenderingSettings.setFocusSkipButtonWhenAvailable( - configuration.focusSkipButtonWhenAvailable); - if (configuration.adUiElements != null) { - adsRenderingSettings.setUiElements(configuration.adUiElements); - } - - // Skip ads based on the start position as required. - long[] adGroupTimesUs = adPlaybackState.adGroupTimesUs; - long contentPositionMs = getContentPeriodPositionMs(checkNotNull(player), timeline, period); - int adGroupForPositionIndex = - adPlaybackState.getAdGroupIndexForPositionUs( - C.msToUs(contentPositionMs), C.msToUs(contentDurationMs)); - if (adGroupForPositionIndex != C.INDEX_UNSET) { - boolean playAdWhenStartingPlayback = - configuration.playAdBeforeStartPosition - || adGroupTimesUs[adGroupForPositionIndex] == C.msToUs(contentPositionMs); - if (!playAdWhenStartingPlayback) { - adGroupForPositionIndex++; - } else if (hasMidrollAdGroups(adGroupTimesUs)) { - // Provide the player's initial position to trigger loading and playing the ad. If there are - // no midrolls, we are playing a preroll and any pending content position wouldn't be - // cleared. - pendingContentPositionMs = contentPositionMs; + private void maybeUpdateCurrentAdTagLoader() { + @Nullable AdTagLoader oldAdTagLoader = currentAdTagLoader; + @Nullable AdTagLoader newAdTagLoader = getCurrentAdTagLoader(); + if (!Util.areEqual(oldAdTagLoader, newAdTagLoader)) { + if (oldAdTagLoader != null) { + oldAdTagLoader.deactivate(); } - if (adGroupForPositionIndex > 0) { - for (int i = 0; i < adGroupForPositionIndex; i++) { - adPlaybackState = adPlaybackState.withSkippedAdGroup(i); - } - if (adGroupForPositionIndex == adGroupTimesUs.length) { - // We don't need to play any ads. Because setPlayAdsAfterTime does not discard non-VMAP - // ads, we signal that no ads will render so the caller can destroy the ads manager. - return null; - } - long adGroupForPositionTimeUs = adGroupTimesUs[adGroupForPositionIndex]; - long adGroupBeforePositionTimeUs = adGroupTimesUs[adGroupForPositionIndex - 1]; - if (adGroupForPositionTimeUs == C.TIME_END_OF_SOURCE) { - // Play the postroll by offsetting the start position just past the last non-postroll ad. - adsRenderingSettings.setPlayAdsAfterTime( - (double) adGroupBeforePositionTimeUs / C.MICROS_PER_SECOND + 1d); - } else { - // Play ads after the midpoint between the ad to play and the one before it, to avoid - // issues with rounding one of the two ad times. - double midpointTimeUs = (adGroupForPositionTimeUs + adGroupBeforePositionTimeUs) / 2d; - adsRenderingSettings.setPlayAdsAfterTime(midpointTimeUs / C.MICROS_PER_SECOND); - } + currentAdTagLoader = newAdTagLoader; + if (newAdTagLoader != null) { + newAdTagLoader.activate(checkNotNull(player)); } } - return adsRenderingSettings; - } - - private VideoProgressUpdate getContentVideoProgressUpdate() { - if (player == null) { - return lastContentProgress; - } - boolean hasContentDuration = contentDurationMs != C.TIME_UNSET; - long contentPositionMs; - if (pendingContentPositionMs != C.TIME_UNSET) { - sentPendingContentPositionMs = true; - contentPositionMs = pendingContentPositionMs; - } else if (fakeContentProgressElapsedRealtimeMs != C.TIME_UNSET) { - long elapsedSinceEndMs = SystemClock.elapsedRealtime() - fakeContentProgressElapsedRealtimeMs; - contentPositionMs = fakeContentProgressOffsetMs + elapsedSinceEndMs; - } else if (imaAdState == IMA_AD_STATE_NONE && !playingAd && hasContentDuration) { - contentPositionMs = getContentPeriodPositionMs(player, timeline, period); - } else { - return VideoProgressUpdate.VIDEO_TIME_NOT_READY; - } - long contentDurationMs = hasContentDuration ? this.contentDurationMs : IMA_DURATION_UNSET; - return new VideoProgressUpdate(contentPositionMs, contentDurationMs); - } - - private VideoProgressUpdate getAdVideoProgressUpdate() { - if (player == null) { - return lastAdProgress; - } else if (imaAdState != IMA_AD_STATE_NONE && playingAd) { - long adDuration = player.getDuration(); - return adDuration == C.TIME_UNSET - ? VideoProgressUpdate.VIDEO_TIME_NOT_READY - : new VideoProgressUpdate(player.getCurrentPosition(), adDuration); - } else { - return VideoProgressUpdate.VIDEO_TIME_NOT_READY; - } - } - - private void updateAdProgress() { - VideoProgressUpdate videoProgressUpdate = getAdVideoProgressUpdate(); - if (configuration.debugModeEnabled) { - Log.d(TAG, "Ad progress: " + ImaUtil.getStringForVideoProgressUpdate(videoProgressUpdate)); - } - - AdMediaInfo adMediaInfo = checkNotNull(imaAdMediaInfo); - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onAdProgress(adMediaInfo, videoProgressUpdate); - } - handler.removeCallbacks(updateAdProgressRunnable); - handler.postDelayed(updateAdProgressRunnable, AD_PROGRESS_UPDATE_INTERVAL_MS); } - private void stopUpdatingAdProgress() { - handler.removeCallbacks(updateAdProgressRunnable); - } - - private int getPlayerVolumePercent() { + @Nullable + private AdTagLoader getCurrentAdTagLoader() { @Nullable Player player = this.player; if (player == null) { - return lastVolumePercent; - } - - @Nullable Player.AudioComponent audioComponent = player.getAudioComponent(); - if (audioComponent != null) { - return (int) (audioComponent.getVolume() * 100); - } - - // Check for a selected track using an audio renderer. - TrackSelectionArray trackSelections = player.getCurrentTrackSelections(); - for (int i = 0; i < player.getRendererCount() && i < trackSelections.length; i++) { - if (player.getRendererType(i) == C.TRACK_TYPE_AUDIO && trackSelections.get(i) != null) { - return 100; - } - } - return 0; - } - - private void handleAdEvent(AdEvent adEvent) { - if (adsManager == null) { - // Drop events after release. - return; + return null; } - switch (adEvent.getType()) { - case AD_BREAK_FETCH_ERROR: - String adGroupTimeSecondsString = checkNotNull(adEvent.getAdData().get("adBreakTime")); - if (configuration.debugModeEnabled) { - Log.d(TAG, "Fetch error for ad at " + adGroupTimeSecondsString + " seconds"); - } - double adGroupTimeSeconds = Double.parseDouble(adGroupTimeSecondsString); - int adGroupIndex = - adGroupTimeSeconds == -1.0 - ? adPlaybackState.adGroupCount - 1 - : getAdGroupIndexForCuePointTimeSeconds(adGroupTimeSeconds); - markAdGroupInErrorStateAndClearPendingContentPosition(adGroupIndex); - break; - case CONTENT_PAUSE_REQUESTED: - // After CONTENT_PAUSE_REQUESTED, IMA will playAd/pauseAd/stopAd to show one or more ads - // before sending CONTENT_RESUME_REQUESTED. - imaPausedContent = true; - pauseContentInternal(); - break; - case TAPPED: - if (eventListener != null) { - eventListener.onAdTapped(); - } - break; - case CLICKED: - if (eventListener != null) { - eventListener.onAdClicked(); - } - break; - case CONTENT_RESUME_REQUESTED: - imaPausedContent = false; - resumeContentInternal(); - break; - case LOG: - Map adData = adEvent.getAdData(); - String message = "AdEvent: " + adData; - Log.i(TAG, message); - break; - default: - break; - } - } - - private void pauseContentInternal() { - imaAdState = IMA_AD_STATE_NONE; - if (sentPendingContentPositionMs) { - pendingContentPositionMs = C.TIME_UNSET; - sentPendingContentPositionMs = false; - } - } - - private void resumeContentInternal() { - if (imaAdInfo != null) { - adPlaybackState = adPlaybackState.withSkippedAdGroup(imaAdInfo.adGroupIndex); - updateAdPlaybackState(); - } else { - // Mark any ads for the current/reported player position that haven't loaded as being in the - // error state, to force resuming content. This includes VPAID ads that never load. - long playerPositionUs; - if (player != null) { - playerPositionUs = C.msToUs(getContentPeriodPositionMs(player, timeline, period)); - } else if (!VideoProgressUpdate.VIDEO_TIME_NOT_READY.equals(lastContentProgress)) { - // Playback is backgrounded so use the last reported content position. - playerPositionUs = C.msToUs(lastContentProgress.getCurrentTimeMs()); - } else { - return; - } - int adGroupIndex = - adPlaybackState.getAdGroupIndexForPositionUs( - playerPositionUs, C.msToUs(contentDurationMs)); - if (adGroupIndex != C.INDEX_UNSET) { - markAdGroupInErrorStateAndClearPendingContentPosition(adGroupIndex); - } + Timeline timeline = player.getCurrentTimeline(); + if (timeline.isEmpty()) { + return null; } - } - - private void handlePlayerStateChanged(boolean playWhenReady, @Player.State int playbackState) { - if (playingAd && imaAdState == IMA_AD_STATE_PLAYING) { - if (!bufferingAd && playbackState == Player.STATE_BUFFERING) { - bufferingAd = true; - AdMediaInfo adMediaInfo = checkNotNull(imaAdMediaInfo); - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onBuffering(adMediaInfo); - } - stopUpdatingAdProgress(); - } else if (bufferingAd && playbackState == Player.STATE_READY) { - bufferingAd = false; - updateAdProgress(); - } + int periodIndex = player.getCurrentPeriodIndex(); + @Nullable Object adsId = timeline.getPeriod(periodIndex, period).getAdsId(); + if (adsId == null) { + return null; } - - if (imaAdState == IMA_AD_STATE_NONE - && playbackState == Player.STATE_BUFFERING - && playWhenReady) { - ensureSentContentCompleteIfAtEndOfStream(); - } else if (imaAdState != IMA_AD_STATE_NONE && playbackState == Player.STATE_ENDED) { - @Nullable AdMediaInfo adMediaInfo = imaAdMediaInfo; - if (adMediaInfo == null) { - Log.w(TAG, "onEnded without ad media info"); - } else { - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onEnded(adMediaInfo); - } - } - if (configuration.debugModeEnabled) { - Log.d(TAG, "VideoAdPlayerCallback.onEnded in onPlaybackStateChanged"); - } + @Nullable AdTagLoader adTagLoader = adTagLoaderByAdsId.get(adsId); + if (adTagLoader == null || !adTagLoaderByAdsMediaSource.containsValue(adTagLoader)) { + return null; } + return adTagLoader; } - private void handleTimelineOrPositionChanged() { + private void maybePreloadNextPeriodAds() { @Nullable Player player = this.player; - if (adsManager == null || player == null) { - return; - } - if (!playingAd && !player.isPlayingAd()) { - ensureSentContentCompleteIfAtEndOfStream(); - if (!sentContentComplete && !timeline.isEmpty()) { - long positionMs = getContentPeriodPositionMs(player, timeline, period); - timeline.getPeriod(/* periodIndex= */ 0, period); - int newAdGroupIndex = period.getAdGroupIndexForPositionUs(C.msToUs(positionMs)); - if (newAdGroupIndex != C.INDEX_UNSET) { - sentPendingContentPositionMs = false; - pendingContentPositionMs = positionMs; - } - } - } - - boolean wasPlayingAd = playingAd; - int oldPlayingAdIndexInAdGroup = playingAdIndexInAdGroup; - playingAd = player.isPlayingAd(); - playingAdIndexInAdGroup = playingAd ? player.getCurrentAdIndexInAdGroup() : C.INDEX_UNSET; - boolean adFinished = wasPlayingAd && playingAdIndexInAdGroup != oldPlayingAdIndexInAdGroup; - if (adFinished) { - // IMA is waiting for the ad playback to finish so invoke the callback now. - // Either CONTENT_RESUME_REQUESTED will be passed next, or playAd will be called again. - @Nullable AdMediaInfo adMediaInfo = imaAdMediaInfo; - if (adMediaInfo == null) { - Log.w(TAG, "onEnded without ad media info"); - } else { - @Nullable AdInfo adInfo = adInfoByAdMediaInfo.get(adMediaInfo); - if (playingAdIndexInAdGroup == C.INDEX_UNSET - || (adInfo != null && adInfo.adIndexInAdGroup < playingAdIndexInAdGroup)) { - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onEnded(adMediaInfo); - } - if (configuration.debugModeEnabled) { - Log.d( - TAG, "VideoAdPlayerCallback.onEnded in onTimelineChanged/onPositionDiscontinuity"); - } - } - } - } - if (!sentContentComplete && !wasPlayingAd && playingAd && imaAdState == IMA_AD_STATE_NONE) { - int adGroupIndex = player.getCurrentAdGroupIndex(); - if (adPlaybackState.adGroupTimesUs[adGroupIndex] == C.TIME_END_OF_SOURCE) { - sendContentComplete(); - } else { - // IMA hasn't called playAd yet, so fake the content position. - fakeContentProgressElapsedRealtimeMs = SystemClock.elapsedRealtime(); - fakeContentProgressOffsetMs = C.usToMs(adPlaybackState.adGroupTimesUs[adGroupIndex]); - if (fakeContentProgressOffsetMs == C.TIME_END_OF_SOURCE) { - fakeContentProgressOffsetMs = contentDurationMs; - } - } - } - } - - private void loadAdInternal(AdMediaInfo adMediaInfo, AdPodInfo adPodInfo) { - if (adsManager == null) { - // Drop events after release. - if (configuration.debugModeEnabled) { - Log.d( - TAG, - "loadAd after release " + getAdMediaInfoString(adMediaInfo) + ", ad pod " + adPodInfo); - } - return; - } - - int adGroupIndex = getAdGroupIndexForAdPod(adPodInfo); - int adIndexInAdGroup = adPodInfo.getAdPosition() - 1; - AdInfo adInfo = new AdInfo(adGroupIndex, adIndexInAdGroup); - adInfoByAdMediaInfo.put(adMediaInfo, adInfo); - if (configuration.debugModeEnabled) { - Log.d(TAG, "loadAd " + getAdMediaInfoString(adMediaInfo)); - } - if (adPlaybackState.isAdInErrorState(adGroupIndex, adIndexInAdGroup)) { - // We have already marked this ad as having failed to load, so ignore the request. IMA will - // timeout after its media load timeout. - return; - } - - // The ad count may increase on successive loads of ads in the same ad pod, for example, due to - // separate requests for ad tags with multiple ads within the ad pod completing after an earlier - // ad has loaded. See also https://github.com/google/ExoPlayer/issues/7477. - AdPlaybackState.AdGroup adGroup = adPlaybackState.adGroups[adInfo.adGroupIndex]; - adPlaybackState = - adPlaybackState.withAdCount( - adInfo.adGroupIndex, max(adPodInfo.getTotalAds(), adGroup.states.length)); - adGroup = adPlaybackState.adGroups[adInfo.adGroupIndex]; - for (int i = 0; i < adIndexInAdGroup; i++) { - // Any preceding ads that haven't loaded are not going to load. - if (adGroup.states[i] == AdPlaybackState.AD_STATE_UNAVAILABLE) { - adPlaybackState = adPlaybackState.withAdLoadError(adGroupIndex, /* adIndexInAdGroup= */ i); - } - } - - Uri adUri = Uri.parse(adMediaInfo.getUrl()); - adPlaybackState = - adPlaybackState.withAdUri(adInfo.adGroupIndex, adInfo.adIndexInAdGroup, adUri); - updateAdPlaybackState(); - } - - private void playAdInternal(AdMediaInfo adMediaInfo) { - if (configuration.debugModeEnabled) { - Log.d(TAG, "playAd " + getAdMediaInfoString(adMediaInfo)); - } - if (adsManager == null) { - // Drop events after release. - return; - } - - if (imaAdState == IMA_AD_STATE_PLAYING) { - // IMA does not always call stopAd before resuming content. - // See [Internal: b/38354028]. - Log.w(TAG, "Unexpected playAd without stopAd"); - } - - if (imaAdState == IMA_AD_STATE_NONE) { - // IMA is requesting to play the ad, so stop faking the content position. - fakeContentProgressElapsedRealtimeMs = C.TIME_UNSET; - fakeContentProgressOffsetMs = C.TIME_UNSET; - imaAdState = IMA_AD_STATE_PLAYING; - imaAdMediaInfo = adMediaInfo; - imaAdInfo = checkNotNull(adInfoByAdMediaInfo.get(adMediaInfo)); - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onPlay(adMediaInfo); - } - if (pendingAdPrepareErrorAdInfo != null && pendingAdPrepareErrorAdInfo.equals(imaAdInfo)) { - pendingAdPrepareErrorAdInfo = null; - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onError(adMediaInfo); - } - } - updateAdProgress(); - } else { - imaAdState = IMA_AD_STATE_PLAYING; - checkState(adMediaInfo.equals(imaAdMediaInfo)); - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onResume(adMediaInfo); - } - } - if (!checkNotNull(player).getPlayWhenReady()) { - checkNotNull(adsManager).pause(); - } - } - - private void pauseAdInternal(AdMediaInfo adMediaInfo) { - if (configuration.debugModeEnabled) { - Log.d(TAG, "pauseAd " + getAdMediaInfoString(adMediaInfo)); - } - if (adsManager == null) { - // Drop event after release. - return; - } - if (imaAdState == IMA_AD_STATE_NONE) { - // This method is called if loadAd has been called but the loaded ad won't play due to a seek - // to a different position, so drop the event. See also [Internal: b/159111848]. - return; - } - checkState(adMediaInfo.equals(imaAdMediaInfo)); - imaAdState = IMA_AD_STATE_PAUSED; - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onPause(adMediaInfo); - } - } - - private void stopAdInternal(AdMediaInfo adMediaInfo) { - if (configuration.debugModeEnabled) { - Log.d(TAG, "stopAd " + getAdMediaInfoString(adMediaInfo)); - } - if (adsManager == null) { - // Drop event after release. + if (player == null) { return; } - if (imaAdState == IMA_AD_STATE_NONE) { - // This method is called if loadAd has been called but the preloaded ad won't play due to a - // seek to a different position, so drop the event and discard the ad. See also [Internal: - // b/159111848]. - @Nullable AdInfo adInfo = adInfoByAdMediaInfo.get(adMediaInfo); - if (adInfo != null) { - adPlaybackState = - adPlaybackState.withSkippedAd(adInfo.adGroupIndex, adInfo.adIndexInAdGroup); - updateAdPlaybackState(); - } + Timeline timeline = player.getCurrentTimeline(); + if (timeline.isEmpty()) { return; } - checkNotNull(player); - imaAdState = IMA_AD_STATE_NONE; - stopUpdatingAdProgress(); - // TODO: Handle the skipped event so the ad can be marked as skipped rather than played. - checkNotNull(imaAdInfo); - int adGroupIndex = imaAdInfo.adGroupIndex; - int adIndexInAdGroup = imaAdInfo.adIndexInAdGroup; - if (adPlaybackState.isAdInErrorState(adGroupIndex, adIndexInAdGroup)) { - // We have already marked this ad as having failed to load, so ignore the request. + int nextPeriodIndex = + timeline.getNextPeriodIndex( + player.getCurrentPeriodIndex(), + period, + window, + player.getRepeatMode(), + player.getShuffleModeEnabled()); + if (nextPeriodIndex == C.INDEX_UNSET) { return; } - adPlaybackState = - adPlaybackState.withPlayedAd(adGroupIndex, adIndexInAdGroup).withAdResumePositionUs(0); - updateAdPlaybackState(); - if (!playingAd) { - imaAdMediaInfo = null; - imaAdInfo = null; - } - } - - private void handleAdGroupLoadError(Exception error) { - int adGroupIndex = getLoadingAdGroupIndex(); - if (adGroupIndex == C.INDEX_UNSET) { - Log.w(TAG, "Unable to determine ad group index for ad group load error", error); + timeline.getPeriod(nextPeriodIndex, period); + @Nullable Object nextAdsId = period.getAdsId(); + if (nextAdsId == null) { return; } - markAdGroupInErrorStateAndClearPendingContentPosition(adGroupIndex); - if (pendingAdLoadError == null) { - pendingAdLoadError = AdLoadException.createForAdGroup(error, adGroupIndex); - } - } - - private void markAdGroupInErrorStateAndClearPendingContentPosition(int adGroupIndex) { - // Update the ad playback state so all ads in the ad group are in the error state. - AdPlaybackState.AdGroup adGroup = adPlaybackState.adGroups[adGroupIndex]; - if (adGroup.count == C.LENGTH_UNSET) { - adPlaybackState = adPlaybackState.withAdCount(adGroupIndex, max(1, adGroup.states.length)); - adGroup = adPlaybackState.adGroups[adGroupIndex]; - } - for (int i = 0; i < adGroup.count; i++) { - if (adGroup.states[i] == AdPlaybackState.AD_STATE_UNAVAILABLE) { - if (configuration.debugModeEnabled) { - Log.d(TAG, "Removing ad " + i + " in ad group " + adGroupIndex); - } - adPlaybackState = adPlaybackState.withAdLoadError(adGroupIndex, i); - } - } - updateAdPlaybackState(); - // Clear any pending content position that triggered attempting to load the ad group. - pendingContentPositionMs = C.TIME_UNSET; - fakeContentProgressElapsedRealtimeMs = C.TIME_UNSET; - } - - private void handleAdPrepareError(int adGroupIndex, int adIndexInAdGroup, Exception exception) { - if (configuration.debugModeEnabled) { - Log.d( - TAG, "Prepare error for ad " + adIndexInAdGroup + " in group " + adGroupIndex, exception); - } - if (adsManager == null) { - Log.w(TAG, "Ignoring ad prepare error after release"); + @Nullable AdTagLoader nextAdTagLoader = adTagLoaderByAdsId.get(nextAdsId); + if (nextAdTagLoader == null || nextAdTagLoader == currentAdTagLoader) { return; } - if (imaAdState == IMA_AD_STATE_NONE) { - // Send IMA a content position at the ad group so that it will try to play it, at which point - // we can notify that it failed to load. - fakeContentProgressElapsedRealtimeMs = SystemClock.elapsedRealtime(); - fakeContentProgressOffsetMs = C.usToMs(adPlaybackState.adGroupTimesUs[adGroupIndex]); - if (fakeContentProgressOffsetMs == C.TIME_END_OF_SOURCE) { - fakeContentProgressOffsetMs = contentDurationMs; - } - pendingAdPrepareErrorAdInfo = new AdInfo(adGroupIndex, adIndexInAdGroup); - } else { - AdMediaInfo adMediaInfo = checkNotNull(imaAdMediaInfo); - // We're already playing an ad. - if (adIndexInAdGroup > playingAdIndexInAdGroup) { - // Mark the playing ad as ended so we can notify the error on the next ad and remove it, - // which means that the ad after will load (if any). - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onEnded(adMediaInfo); - } - } - playingAdIndexInAdGroup = adPlaybackState.adGroups[adGroupIndex].getFirstAdIndexToPlay(); - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onError(checkNotNull(adMediaInfo)); - } - } - adPlaybackState = adPlaybackState.withAdLoadError(adGroupIndex, adIndexInAdGroup); - updateAdPlaybackState(); - } - - private void ensureSentContentCompleteIfAtEndOfStream() { - if (!sentContentComplete - && contentDurationMs != C.TIME_UNSET - && pendingContentPositionMs == C.TIME_UNSET - && getContentPeriodPositionMs(checkNotNull(player), timeline, period) - + THRESHOLD_END_OF_CONTENT_MS - >= contentDurationMs) { - sendContentComplete(); - } - } - - private void sendContentComplete() { - for (int i = 0; i < adCallbacks.size(); i++) { - adCallbacks.get(i).onContentComplete(); - } - sentContentComplete = true; - if (configuration.debugModeEnabled) { - Log.d(TAG, "adsLoader.contentComplete"); - } - for (int i = 0; i < adPlaybackState.adGroupCount; i++) { - if (adPlaybackState.adGroupTimesUs[i] != C.TIME_END_OF_SOURCE) { - adPlaybackState = adPlaybackState.withSkippedAdGroup(/* adGroupIndex= */ i); - } - } - updateAdPlaybackState(); - } - - private void updateAdPlaybackState() { - // Ignore updates while detached. When a player is attached it will receive the latest state. - if (eventListener != null) { - eventListener.onAdPlaybackState(adPlaybackState); - } - } - - private void maybeNotifyPendingAdLoadError() { - if (pendingAdLoadError != null && eventListener != null) { - eventListener.onAdLoadError(pendingAdLoadError, adTagDataSpec); - pendingAdLoadError = null; - } - } - - private void maybeNotifyInternalError(String name, Exception cause) { - String message = "Internal error in " + name; - Log.e(TAG, message, cause); - // We can't recover from an unexpected error in general, so skip all remaining ads. - for (int i = 0; i < adPlaybackState.adGroupCount; i++) { - adPlaybackState = adPlaybackState.withSkippedAdGroup(i); - } - updateAdPlaybackState(); - if (eventListener != null) { - eventListener.onAdLoadError( - AdLoadException.createForUnexpected(new RuntimeException(message, cause)), adTagDataSpec); - } - } - - private int getAdGroupIndexForAdPod(AdPodInfo adPodInfo) { - if (adPodInfo.getPodIndex() == -1) { - // This is a postroll ad. - return adPlaybackState.adGroupCount - 1; - } - - // adPodInfo.podIndex may be 0-based or 1-based, so for now look up the cue point instead. - return getAdGroupIndexForCuePointTimeSeconds(adPodInfo.getTimeOffset()); - } - - /** - * Returns the index of the ad group that will preload next, or {@link C#INDEX_UNSET} if there is - * no such ad group. - */ - private int getLoadingAdGroupIndex() { - if (player == null) { - return C.INDEX_UNSET; - } - long playerPositionUs = C.msToUs(getContentPeriodPositionMs(player, timeline, period)); - int adGroupIndex = - adPlaybackState.getAdGroupIndexForPositionUs(playerPositionUs, C.msToUs(contentDurationMs)); - if (adGroupIndex == C.INDEX_UNSET) { - adGroupIndex = - adPlaybackState.getAdGroupIndexAfterPositionUs( - playerPositionUs, C.msToUs(contentDurationMs)); - } - return adGroupIndex; - } - - private int getAdGroupIndexForCuePointTimeSeconds(double cuePointTimeSeconds) { - // We receive initial cue points from IMA SDK as floats. This code replicates the same - // calculation used to populate adGroupTimesUs (having truncated input back to float, to avoid - // failures if the behavior of the IMA SDK changes to provide greater precision). - float cuePointTimeSecondsFloat = (float) cuePointTimeSeconds; - long adPodTimeUs = Math.round((double) cuePointTimeSecondsFloat * C.MICROS_PER_SECOND); - for (int adGroupIndex = 0; adGroupIndex < adPlaybackState.adGroupCount; adGroupIndex++) { - long adGroupTimeUs = adPlaybackState.adGroupTimesUs[adGroupIndex]; - if (adGroupTimeUs != C.TIME_END_OF_SOURCE - && Math.abs(adGroupTimeUs - adPodTimeUs) < THRESHOLD_AD_MATCH_US) { - return adGroupIndex; - } - } - throw new IllegalStateException("Failed to find cue point"); - } - - private String getAdMediaInfoString(AdMediaInfo adMediaInfo) { - @Nullable AdInfo adInfo = adInfoByAdMediaInfo.get(adMediaInfo); - return "AdMediaInfo[" + adMediaInfo.getUrl() + (adInfo != null ? ", " + adInfo : "") + "]"; - } - - private static long getContentPeriodPositionMs( - Player player, Timeline timeline, Timeline.Period period) { - long contentWindowPositionMs = player.getContentPosition(); - return contentWindowPositionMs - - (timeline.isEmpty() - ? 0 - : timeline.getPeriod(/* periodIndex= */ 0, period).getPositionInWindowMs()); - } - - private static Looper getImaLooper() { - // IMA SDK callbacks occur on the main thread. This method can be used to check that the player - // is using the same looper, to ensure all interaction with this class is on the main thread. - return Looper.getMainLooper(); - } - - private static boolean hasMidrollAdGroups(long[] adGroupTimesUs) { - int count = adGroupTimesUs.length; - if (count == 1) { - return adGroupTimesUs[0] != 0 && adGroupTimesUs[0] != C.TIME_END_OF_SOURCE; - } else if (count == 2) { - return adGroupTimesUs[0] != 0 || adGroupTimesUs[1] != C.TIME_END_OF_SOURCE; - } else { - // There's at least one midroll ad group, as adGroupTimesUs is never empty. - return true; - } - } - - private void destroyAdsManager() { - if (adsManager != null) { - adsManager.removeAdErrorListener(componentListener); - if (configuration.applicationAdErrorListener != null) { - adsManager.removeAdErrorListener(configuration.applicationAdErrorListener); - } - adsManager.removeAdEventListener(componentListener); - if (configuration.applicationAdEventListener != null) { - adsManager.removeAdEventListener(configuration.applicationAdEventListener); - } - adsManager.destroy(); - adsManager = null; - } - } - - private final class ComponentListener - implements AdsLoadedListener, - ContentProgressProvider, - AdEventListener, - AdErrorListener, - VideoAdPlayer { - - // AdsLoader.AdsLoadedListener implementation. - - @Override - public void onAdsManagerLoaded(AdsManagerLoadedEvent adsManagerLoadedEvent) { - AdsManager adsManager = adsManagerLoadedEvent.getAdsManager(); - if (!Util.areEqual(pendingAdRequestContext, adsManagerLoadedEvent.getUserRequestContext())) { - adsManager.destroy(); - return; - } - pendingAdRequestContext = null; - ImaAdsLoader.this.adsManager = adsManager; - adsManager.addAdErrorListener(this); - if (configuration.applicationAdErrorListener != null) { - adsManager.addAdErrorListener(configuration.applicationAdErrorListener); - } - adsManager.addAdEventListener(this); - if (configuration.applicationAdEventListener != null) { - adsManager.addAdEventListener(configuration.applicationAdEventListener); - } - if (player != null) { - // If a player is attached already, start playback immediately. - try { - adPlaybackState = - ImaUtil.getInitialAdPlaybackStateForCuePoints(adsManager.getAdCuePoints()); - hasAdPlaybackState = true; - updateAdPlaybackState(); - } catch (RuntimeException e) { - maybeNotifyInternalError("onAdsManagerLoaded", e); - } - } - } - - // ContentProgressProvider implementation. - - @Override - public VideoProgressUpdate getContentProgress() { - VideoProgressUpdate videoProgressUpdate = getContentVideoProgressUpdate(); - if (configuration.debugModeEnabled) { - Log.d( - TAG, - "Content progress: " + ImaUtil.getStringForVideoProgressUpdate(videoProgressUpdate)); - } - - if (waitingForPreloadElapsedRealtimeMs != C.TIME_UNSET) { - // IMA is polling the player position but we are buffering for an ad to preload, so playback - // may be stuck. Detect this case and signal an error if applicable. - long stuckElapsedRealtimeMs = - SystemClock.elapsedRealtime() - waitingForPreloadElapsedRealtimeMs; - if (stuckElapsedRealtimeMs >= THRESHOLD_AD_PRELOAD_MS) { - waitingForPreloadElapsedRealtimeMs = C.TIME_UNSET; - handleAdGroupLoadError(new IOException("Ad preloading timed out")); - maybeNotifyPendingAdLoadError(); - } - } - - return videoProgressUpdate; - } - - // AdEvent.AdEventListener implementation. - - @Override - public void onAdEvent(AdEvent adEvent) { - AdEventType adEventType = adEvent.getType(); - if (configuration.debugModeEnabled && adEventType != AdEventType.AD_PROGRESS) { - Log.d(TAG, "onAdEvent: " + adEventType); - } - try { - handleAdEvent(adEvent); - } catch (RuntimeException e) { - maybeNotifyInternalError("onAdEvent", e); - } - } - - // AdErrorEvent.AdErrorListener implementation. - - @Override - public void onAdError(AdErrorEvent adErrorEvent) { - AdError error = adErrorEvent.getError(); - if (configuration.debugModeEnabled) { - Log.d(TAG, "onAdError", error); - } - if (adsManager == null) { - // No ads were loaded, so allow playback to start without any ads. - pendingAdRequestContext = null; - adPlaybackState = AdPlaybackState.NONE; - hasAdPlaybackState = true; - updateAdPlaybackState(); - } else if (ImaUtil.isAdGroupLoadError(error)) { - try { - handleAdGroupLoadError(error); - } catch (RuntimeException e) { - maybeNotifyInternalError("onAdError", e); - } - } - if (pendingAdLoadError == null) { - pendingAdLoadError = AdLoadException.createForAllAds(error); - } - maybeNotifyPendingAdLoadError(); - } - - // VideoAdPlayer implementation. - - @Override - public void addCallback(VideoAdPlayerCallback videoAdPlayerCallback) { - adCallbacks.add(videoAdPlayerCallback); - } - - @Override - public void removeCallback(VideoAdPlayerCallback videoAdPlayerCallback) { - adCallbacks.remove(videoAdPlayerCallback); - } - - @Override - public VideoProgressUpdate getAdProgress() { - throw new IllegalStateException("Unexpected call to getAdProgress when using preloading"); - } - - @Override - public int getVolume() { - return getPlayerVolumePercent(); - } - - @Override - public void loadAd(AdMediaInfo adMediaInfo, AdPodInfo adPodInfo) { - try { - loadAdInternal(adMediaInfo, adPodInfo); - } catch (RuntimeException e) { - maybeNotifyInternalError("loadAd", e); - } - } - - @Override - public void playAd(AdMediaInfo adMediaInfo) { - try { - playAdInternal(adMediaInfo); - } catch (RuntimeException e) { - maybeNotifyInternalError("playAd", e); - } - } - - @Override - public void pauseAd(AdMediaInfo adMediaInfo) { - try { - pauseAdInternal(adMediaInfo); - } catch (RuntimeException e) { - maybeNotifyInternalError("pauseAd", e); - } - } - - @Override - public void stopAd(AdMediaInfo adMediaInfo) { - try { - stopAdInternal(adMediaInfo); - } catch (RuntimeException e) { - maybeNotifyInternalError("stopAd", e); - } - } - - @Override - public void release() { - // Do nothing. - } - } - - // TODO: Consider moving this into AdPlaybackState. - private static final class AdInfo { - public final int adGroupIndex; - public final int adIndexInAdGroup; - - public AdInfo(int adGroupIndex, int adIndexInAdGroup) { - this.adGroupIndex = adGroupIndex; - this.adIndexInAdGroup = adIndexInAdGroup; - } - - @Override - public boolean equals(@Nullable Object o) { - if (this == o) { - return true; - } - if (o == null || getClass() != o.getClass()) { - return false; - } - AdInfo adInfo = (AdInfo) o; - if (adGroupIndex != adInfo.adGroupIndex) { - return false; - } - return adIndexInAdGroup == adInfo.adIndexInAdGroup; - } - - @Override - public int hashCode() { - int result = adGroupIndex; - result = 31 * result + adIndexInAdGroup; - return result; - } - - @Override - public String toString() { - return "(" + adGroupIndex + ", " + adIndexInAdGroup + ')'; - } + long periodPositionUs = + timeline.getPeriodPosition( + window, period, period.windowIndex, /* windowPositionUs= */ C.TIME_UNSET) + .second; + nextAdTagLoader.maybePreloadAds(C.usToMs(periodPositionUs), C.usToMs(period.durationUs)); } /** @@ -1915,7 +699,7 @@ private static final class DefaultImaFactory implements ImaUtil.ImaFactory { @Override public ImaSdkSettings createImaSdkSettings() { ImaSdkSettings settings = ImaSdkFactory.getInstance().createImaSdkSettings(); - settings.setLanguage(getImaLanguageCodeForDefaultLocale()); + settings.setLanguage(Util.getSystemLanguageCodes()[0]); return settings; } @@ -1952,22 +736,10 @@ public AdsRequest createAdsRequest() { } @Override - public AdsLoader createAdsLoader( + public com.google.ads.interactivemedia.v3.api.AdsLoader createAdsLoader( Context context, ImaSdkSettings imaSdkSettings, AdDisplayContainer adDisplayContainer) { return ImaSdkFactory.getInstance() .createAdsLoader(context, imaSdkSettings, adDisplayContainer); } - - /** - * Returns a language code that's suitable for passing to {@link ImaSdkSettings#setLanguage} and - * corresponds to the device's {@link Locale#getDefault() default Locale}. IMA will fall back to - * its default language code ("en") if the value returned is unsupported. - */ - // TODO: It may be possible to define a better mapping onto IMA's supported language codes. See: - // https://developers.google.com/interactive-media-ads/docs/sdks/android/client-side/localization. - // [Internal ref: b/174042000] will help if implemented. - private static String getImaLanguageCodeForDefaultLocale() { - return Util.splitAtFirst(Util.getSystemLanguageCodes()[0], "-")[0]; - } } } diff --git a/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/ImaUtil.java b/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/ImaUtil.java index 6d69547278b..0324e93713a 100644 --- a/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/ImaUtil.java +++ b/extensions/ima/src/main/java/com/google/android/exoplayer2/ext/ima/ImaUtil.java @@ -16,6 +16,7 @@ package com.google.android.exoplayer2.ext.ima; import android.content.Context; +import android.os.Looper; import android.view.View; import android.view.ViewGroup; import androidx.annotation.Nullable; @@ -35,7 +36,6 @@ import com.google.ads.interactivemedia.v3.api.player.VideoAdPlayer; import com.google.ads.interactivemedia.v3.api.player.VideoProgressUpdate; import com.google.android.exoplayer2.C; -import com.google.android.exoplayer2.source.ads.AdPlaybackState; import com.google.android.exoplayer2.source.ads.AdsLoader.OverlayInfo; import com.google.android.exoplayer2.upstream.DataSchemeDataSource; import com.google.android.exoplayer2.upstream.DataSpec; @@ -89,6 +89,7 @@ public static final class Configuration { public final boolean focusSkipButtonWhenAvailable; public final boolean playAdBeforeStartPosition; public final int mediaBitrate; + @Nullable public final Boolean enableContinuousPlayback; @Nullable public final List adMediaMimeTypes; @Nullable public final Set adUiElements; @Nullable public final Collection companionAdSlots; @@ -105,6 +106,7 @@ public Configuration( boolean focusSkipButtonWhenAvailable, boolean playAdBeforeStartPosition, int mediaBitrate, + @Nullable Boolean enableContinuousPlayback, @Nullable List adMediaMimeTypes, @Nullable Set adUiElements, @Nullable Collection companionAdSlots, @@ -119,6 +121,7 @@ public Configuration( this.focusSkipButtonWhenAvailable = focusSkipButtonWhenAvailable; this.playAdBeforeStartPosition = playAdBeforeStartPosition; this.mediaBitrate = mediaBitrate; + this.enableContinuousPlayback = enableContinuousPlayback; this.adMediaMimeTypes = adMediaMimeTypes; this.adUiElements = adUiElements; this.companionAdSlots = companionAdSlots; @@ -130,6 +133,9 @@ public Configuration( } } + public static final int TIMEOUT_UNSET = -1; + public static final int BITRATE_UNSET = -1; + /** * Returns the IMA {@link FriendlyObstructionPurpose} corresponding to the given {@link * OverlayInfo#purpose}. @@ -150,15 +156,14 @@ public static FriendlyObstructionPurpose getFriendlyObstructionPurpose( } /** - * Returns an initial {@link AdPlaybackState} with ad groups at the provided {@code cuePoints}. + * Returns the microsecond ad group timestamps corresponding to the specified cue points. * - * @param cuePoints The cue points of the ads in seconds. - * @return The {@link AdPlaybackState}. + * @param cuePoints The cue points of the ads in seconds, provided by the IMA SDK. + * @return The corresponding microsecond ad group timestamps. */ - public static AdPlaybackState getInitialAdPlaybackStateForCuePoints(List cuePoints) { + public static long[] getAdGroupTimesUsForCuePoints(List cuePoints) { if (cuePoints.isEmpty()) { - // If no cue points are specified, there is a preroll ad. - return new AdPlaybackState(/* adGroupTimesUs...= */ 0); + return new long[] {0L}; } int count = cuePoints.size(); @@ -174,7 +179,7 @@ public static AdPlaybackState getInitialAdPlaybackStateForCuePoints(List } // Cue points may be out of order, so sort them. Arrays.sort(adGroupTimesUs, 0, adGroupIndex); - return new AdPlaybackState(adGroupTimesUs); + return adGroupTimesUs; } /** Returns an {@link AdsRequest} based on the specified ad tag {@link DataSpec}. */ @@ -203,6 +208,13 @@ public static boolean isAdGroupLoadError(AdError adError) { || adError.getErrorCode() == AdError.AdErrorCode.UNKNOWN_ERROR; } + /** Returns the looper on which all IMA SDK interaction must occur. */ + public static Looper getImaLooper() { + // IMA SDK callbacks occur on the main thread. This method can be used to check that the player + // is using the same looper, to ensure all interaction with this class is on the main thread. + return Looper.getMainLooper(); + } + /** Returns a human-readable representation of a video progress update. */ public static String getStringForVideoProgressUpdate(VideoProgressUpdate videoProgressUpdate) { if (VideoProgressUpdate.VIDEO_TIME_NOT_READY.equals(videoProgressUpdate)) { diff --git a/extensions/ima/src/test/java/com/google/android/exoplayer2/ext/ima/FakePlayer.java b/extensions/ima/src/test/java/com/google/android/exoplayer2/ext/ima/FakePlayer.java index 4c98233acb8..6b62af93f3c 100644 --- a/extensions/ima/src/test/java/com/google/android/exoplayer2/ext/ima/FakePlayer.java +++ b/extensions/ima/src/test/java/com/google/android/exoplayer2/ext/ima/FakePlayer.java @@ -21,25 +21,32 @@ import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.testutil.StubExoPlayer; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; -import java.util.ArrayList; +import com.google.android.exoplayer2.util.Clock; +import com.google.android.exoplayer2.util.ListenerSet; /** A fake player for testing content/ad playback. */ /* package */ final class FakePlayer extends StubExoPlayer { - private final ArrayList listeners; + private final ListenerSet listeners; private final Timeline.Period period; - private final Timeline timeline; + private Timeline timeline; @Player.State private int state; private boolean playWhenReady; - private long position; - private long contentPosition; + private int periodIndex; + private long positionMs; + private long contentPositionMs; private boolean isPlayingAd; private int adGroupIndex; private int adIndexInAdGroup; public FakePlayer() { - listeners = new ArrayList<>(); + listeners = + new ListenerSet<>( + Looper.getMainLooper(), + Clock.DEFAULT, + Player.Events::new, + (listener, eventFlags) -> listener.onEvents(/* player= */ this, eventFlags)); period = new Timeline.Period(); state = Player.STATE_IDLE; playWhenReady = true; @@ -48,26 +55,27 @@ public FakePlayer() { /** Sets the timeline on this fake player, which notifies listeners with the changed timeline. */ public void updateTimeline(Timeline timeline, @TimelineChangeReason int reason) { - for (Player.EventListener listener : listeners) { - listener.onTimelineChanged(timeline, reason); - } + this.timeline = timeline; + listeners.sendEvent( + Player.EVENT_TIMELINE_CHANGED, listener -> listener.onTimelineChanged(timeline, reason)); } /** * Sets the state of this player as if it were playing content at the given {@code position}. If * an ad is currently playing, this will trigger a position discontinuity. */ - public void setPlayingContentPosition(long position) { + public void setPlayingContentPosition(int periodIndex, long positionMs) { boolean notify = isPlayingAd; isPlayingAd = false; adGroupIndex = C.INDEX_UNSET; adIndexInAdGroup = C.INDEX_UNSET; - this.position = position; - contentPosition = position; + this.periodIndex = periodIndex; + this.positionMs = positionMs; + contentPositionMs = positionMs; if (notify) { - for (Player.EventListener listener : listeners) { - listener.onPositionDiscontinuity(DISCONTINUITY_REASON_AD_INSERTION); - } + listeners.sendEvent( + Player.EVENT_POSITION_DISCONTINUITY, + listener -> listener.onPositionDiscontinuity(DISCONTINUITY_REASON_AD_INSERTION)); } } @@ -77,17 +85,22 @@ public void setPlayingContentPosition(long position) { * position discontinuity. */ public void setPlayingAdPosition( - int adGroupIndex, int adIndexInAdGroup, long position, long contentPosition) { + int periodIndex, + int adGroupIndex, + int adIndexInAdGroup, + long positionMs, + long contentPositionMs) { boolean notify = !isPlayingAd || this.adIndexInAdGroup != adIndexInAdGroup; isPlayingAd = true; + this.periodIndex = periodIndex; this.adGroupIndex = adGroupIndex; this.adIndexInAdGroup = adIndexInAdGroup; - this.position = position; - this.contentPosition = contentPosition; + this.positionMs = positionMs; + this.contentPositionMs = contentPositionMs; if (notify) { - for (Player.EventListener listener : listeners) { - listener.onPositionDiscontinuity(DISCONTINUITY_REASON_AD_INSERTION); - } + listeners.sendEvent( + EVENT_POSITION_DISCONTINUITY, + listener -> listener.onPositionDiscontinuity(DISCONTINUITY_REASON_AD_INSERTION)); } } @@ -99,16 +112,18 @@ public void setState(@Player.State int state, boolean playWhenReady) { this.state = state; this.playWhenReady = playWhenReady; if (playbackStateChanged || playWhenReadyChanged) { - for (Player.EventListener listener : listeners) { - listener.onPlayerStateChanged(playWhenReady, state); - if (playbackStateChanged) { - listener.onPlaybackStateChanged(state); - } - if (playWhenReadyChanged) { - listener.onPlayWhenReadyChanged( - playWhenReady, PLAY_WHEN_READY_CHANGE_REASON_USER_REQUEST); - } - } + listeners.sendEvent( + Player.EVENT_PLAYBACK_STATE_CHANGED, + listener -> { + listener.onPlayerStateChanged(playWhenReady, state); + if (playbackStateChanged) { + listener.onPlaybackStateChanged(state); + } + if (playWhenReadyChanged) { + listener.onPlayWhenReadyChanged( + playWhenReady, PLAY_WHEN_READY_CHANGE_REASON_USER_REQUEST); + } + }); } } @@ -145,6 +160,17 @@ public boolean getPlayWhenReady() { return playWhenReady; } + @Override + @RepeatMode + public int getRepeatMode() { + return REPEAT_MODE_OFF; + } + + @Override + public boolean getShuffleModeEnabled() { + return false; + } + @Override public int getRendererCount() { return 0; @@ -162,7 +188,7 @@ public Timeline getCurrentTimeline() { @Override public int getCurrentPeriodIndex() { - return 0; + return periodIndex; } @Override @@ -186,7 +212,7 @@ public long getDuration() { @Override public long getCurrentPosition() { - return position; + return positionMs; } @Override @@ -206,6 +232,6 @@ public int getCurrentAdIndexInAdGroup() { @Override public long getContentPosition() { - return contentPosition; + return contentPositionMs; } } diff --git a/extensions/ima/src/test/java/com/google/android/exoplayer2/ext/ima/ImaAdsLoaderTest.java b/extensions/ima/src/test/java/com/google/android/exoplayer2/ext/ima/ImaAdsLoaderTest.java index d64f6c4b67a..e7b6603694f 100644 --- a/extensions/ima/src/test/java/com/google/android/exoplayer2/ext/ima/ImaAdsLoaderTest.java +++ b/extensions/ima/src/test/java/com/google/android/exoplayer2/ext/ima/ImaAdsLoaderTest.java @@ -16,8 +16,10 @@ package com.google.android.exoplayer2.ext.ima; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.android.exoplayer2.ext.ima.ImaUtil.getAdGroupTimesUsForCuePoints; import static com.google.common.truth.Truth.assertThat; import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyBoolean; import static org.mockito.ArgumentMatchers.anyDouble; import static org.mockito.Mockito.atLeastOnce; import static org.mockito.Mockito.doAnswer; @@ -27,8 +29,11 @@ import static org.mockito.Mockito.never; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; +import static org.robolectric.Shadows.shadowOf; +import android.content.Context; import android.net.Uri; +import android.os.Looper; import android.view.View; import android.view.ViewGroup; import android.widget.FrameLayout; @@ -51,16 +56,16 @@ import com.google.ads.interactivemedia.v3.api.player.VideoProgressUpdate; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlaybackException; -import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.Timeline.Period; import com.google.android.exoplayer2.ext.ima.ImaUtil.ImaFactory; -import com.google.android.exoplayer2.source.MaskingMediaSource.PlaceholderTimeline; +import com.google.android.exoplayer2.source.DefaultMediaSourceFactory; import com.google.android.exoplayer2.source.ads.AdPlaybackState; import com.google.android.exoplayer2.source.ads.AdsLoader; +import com.google.android.exoplayer2.source.ads.AdsMediaSource; import com.google.android.exoplayer2.source.ads.AdsMediaSource.AdLoadException; -import com.google.android.exoplayer2.source.ads.SinglePeriodAdTimeline; +import com.google.android.exoplayer2.testutil.FakeMediaSource; import com.google.android.exoplayer2.testutil.FakeTimeline; import com.google.android.exoplayer2.testutil.FakeTimeline.TimelineWindowDefinition; import com.google.android.exoplayer2.upstream.DataSpec; @@ -101,6 +106,7 @@ public final class ImaAdsLoaderTest { CONTENT_TIMELINE.getPeriod(/* periodIndex= */ 0, new Period()).durationUs; private static final Uri TEST_URI = Uri.parse("https://www.google.com"); private static final DataSpec TEST_DATA_SPEC = new DataSpec(TEST_URI); + private static final Object TEST_ADS_ID = new Object(); private static final AdMediaInfo TEST_AD_MEDIA_INFO = new AdMediaInfo("https://www.google.com"); private static final long TEST_AD_DURATION_US = 5 * C.MICROS_PER_SECOND; private static final ImmutableList PREROLL_CUE_POINTS_SECONDS = ImmutableList.of(0f); @@ -114,11 +120,14 @@ public final class ImaAdsLoaderTest { @Mock private AdsRequest mockAdsRequest; @Mock private AdsManagerLoadedEvent mockAdsManagerLoadedEvent; @Mock private com.google.ads.interactivemedia.v3.api.AdsLoader mockAdsLoader; + @Mock private VideoAdPlayer.VideoAdPlayerCallback mockVideoAdPlayerCallback; @Mock private FriendlyObstruction mockFriendlyObstruction; @Mock private ImaFactory mockImaFactory; @Mock private AdPodInfo mockAdPodInfo; @Mock private Ad mockPrerollSingleAd; + private TimelineWindowDefinition[] timelineWindowDefinitions; + private AdsMediaSource adsMediaSource; private ViewGroup adViewGroup; private AdsLoader.AdViewProvider adViewProvider; private AdsLoader.AdViewProvider audioAdsAdViewProvider; @@ -126,12 +135,13 @@ public final class ImaAdsLoaderTest { private ContentProgressProvider contentProgressProvider; private VideoAdPlayer videoAdPlayer; private TestAdsLoaderListener adsLoaderListener; - private FakePlayer fakeExoPlayer; + private FakePlayer fakePlayer; private ImaAdsLoader imaAdsLoader; @Before public void setUp() { setupMocks(); + fakePlayer = new FakePlayer(); adViewGroup = new FrameLayout(getApplicationContext()); View adOverlayView = new View(getApplicationContext()); adViewProvider = @@ -159,27 +169,45 @@ public ImmutableList getAdOverlayInfos() { return ImmutableList.of(); } }; + imaAdsLoader = + new ImaAdsLoader.Builder(getApplicationContext()) + .setImaFactory(mockImaFactory) + .setImaSdkSettings(mockImaSdkSettings) + .setVideoAdPlayerCallback(mockVideoAdPlayerCallback) + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); + timelineWindowDefinitions = + new TimelineWindowDefinition[] {getInitialTimelineWindowDefinition(TEST_ADS_ID)}; + adsLoaderListener = new TestAdsLoaderListener(/* periodIndex= */ 0); + when(mockAdsManager.getAdCuePoints()).thenReturn(PREROLL_CUE_POINTS_SECONDS); } @After public void teardown() { - if (imaAdsLoader != null) { - imaAdsLoader.release(); - } + imaAdsLoader.release(); } @Test - public void builder_overridesPlayerType() { + public void loader_overridesCustomPlayerType() { when(mockImaSdkSettings.getPlayerType()).thenReturn("test player type"); - setupPlayback(CONTENT_TIMELINE, PREROLL_CUE_POINTS_SECONDS); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockImaSdkSettings).setPlayerType("google/exo.ext.ima"); } @Test public void start_setsAdUiViewGroup() { - setupPlayback(CONTENT_TIMELINE, PREROLL_CUE_POINTS_SECONDS); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockImaFactory, atLeastOnce()).createAdDisplayContainer(adViewGroup, videoAdPlayer); verify(mockImaFactory, never()).createAudioAdDisplayContainer(any(), any()); @@ -188,8 +216,8 @@ public void start_setsAdUiViewGroup() { @Test public void startForAudioOnlyAds_createsAudioOnlyAdDisplayContainer() { - setupPlayback(CONTENT_TIMELINE, PREROLL_CUE_POINTS_SECONDS); - imaAdsLoader.start(adsLoaderListener, audioAdsAdViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, audioAdsAdViewProvider, adsLoaderListener); verify(mockImaFactory, atLeastOnce()) .createAudioAdDisplayContainer(getApplicationContext(), videoAdPlayer); @@ -199,9 +227,14 @@ public void startForAudioOnlyAds_createsAudioOnlyAdDisplayContainer() { @Test public void start_withPlaceholderContent_initializedAdsLoader() { - Timeline placeholderTimeline = new PlaceholderTimeline(MediaItem.fromUri(Uri.EMPTY)); - setupPlayback(placeholderTimeline, PREROLL_CUE_POINTS_SECONDS); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + timelineWindowDefinitions = + new TimelineWindowDefinition[] { + getInitialTimelineWindowDefinition(TEST_ADS_ID, /* isPlaceholder= */ true) + }; + + when(mockAdsManager.getAdCuePoints()).thenReturn(PREROLL_CUE_POINTS_SECONDS); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); // We'll only create the rendering settings when initializing the ads loader. verify(mockImaFactory).createAdsRenderingSettings(); @@ -209,37 +242,37 @@ public void start_withPlaceholderContent_initializedAdsLoader() { @Test public void start_updatesAdPlaybackState() { - setupPlayback(CONTENT_TIMELINE, PREROLL_CUE_POINTS_SECONDS); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - new AdPlaybackState(/* adGroupTimesUs...= */ 0) + new AdPlaybackState(TEST_ADS_ID, /* adGroupTimesUs...= */ 0) .withContentDurationUs(CONTENT_PERIOD_DURATION_US)); } @Test public void startAfterRelease() { - setupPlayback(CONTENT_TIMELINE, PREROLL_CUE_POINTS_SECONDS); imaAdsLoader.release(); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); } @Test public void startAndCallbacksAfterRelease() { - setupPlayback(CONTENT_TIMELINE, PREROLL_CUE_POINTS_SECONDS); // Request ads in order to get a reference to the ad event listener. - imaAdsLoader.requestAds(adViewGroup); + imaAdsLoader.requestAds(TEST_DATA_SPEC, TEST_ADS_ID, adViewGroup); imaAdsLoader.release(); - imaAdsLoader.start(adsLoaderListener, adViewProvider); - fakeExoPlayer.setPlayingContentPosition(/* position= */ 0); - fakeExoPlayer.setState(Player.STATE_READY, true); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, /* positionMs= */ 0); + fakePlayer.setState(Player.STATE_READY, /* playWhenReady= */ true); // If callbacks are invoked there is no crash. // Note: we can't currently call getContentProgress/getAdProgress as a VerifyError is thrown // when using Robolectric and accessing VideoProgressUpdate.VIDEO_TIME_NOT_READY, due to the IMA // SDK being proguarded. - imaAdsLoader.requestAds(adViewGroup); + imaAdsLoader.requestAds(TEST_DATA_SPEC, TEST_ADS_ID, adViewGroup); adEventListener.onAdEvent(getAdEvent(AdEventType.LOADED, mockPrerollSingleAd)); videoAdPlayer.loadAd(TEST_AD_MEDIA_INFO, mockAdPodInfo); adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_PAUSE_REQUESTED, mockPrerollSingleAd)); @@ -251,41 +284,41 @@ public void startAndCallbacksAfterRelease() { imaAdsLoader.onPositionDiscontinuity(Player.DISCONTINUITY_REASON_SEEK); adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_RESUME_REQUESTED, /* ad= */ null)); imaAdsLoader.handlePrepareError( - /* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, new IOException()); + adsMediaSource, /* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, new IOException()); } @Test public void playback_withPrerollAd_marksAdAsPlayed() { - setupPlayback(CONTENT_TIMELINE, PREROLL_CUE_POINTS_SECONDS); - // Load the preroll ad. - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); adEventListener.onAdEvent(getAdEvent(AdEventType.LOADED, mockPrerollSingleAd)); videoAdPlayer.loadAd(TEST_AD_MEDIA_INFO, mockAdPodInfo); adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_PAUSE_REQUESTED, mockPrerollSingleAd)); // Play the preroll ad. videoAdPlayer.playAd(TEST_AD_MEDIA_INFO); - fakeExoPlayer.setPlayingAdPosition( + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, /* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, /* position= */ 0, /* contentPosition= */ 0); - fakeExoPlayer.setState(Player.STATE_READY, true); + fakePlayer.setState(Player.STATE_READY, /* playWhenReady= */ true); adEventListener.onAdEvent(getAdEvent(AdEventType.STARTED, mockPrerollSingleAd)); adEventListener.onAdEvent(getAdEvent(AdEventType.FIRST_QUARTILE, mockPrerollSingleAd)); adEventListener.onAdEvent(getAdEvent(AdEventType.MIDPOINT, mockPrerollSingleAd)); adEventListener.onAdEvent(getAdEvent(AdEventType.THIRD_QUARTILE, mockPrerollSingleAd)); // Play the content. - fakeExoPlayer.setPlayingContentPosition(0); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, /* positionMs= */ 0); videoAdPlayer.stopAd(TEST_AD_MEDIA_INFO); adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_RESUME_REQUESTED, /* ad= */ null)); // Verify that the preroll ad has been marked as played. - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - new AdPlaybackState(/* adGroupTimesUs...= */ 0) + new AdPlaybackState(TEST_ADS_ID, /* adGroupTimesUs...= */ 0) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdUri(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, TEST_URI) @@ -300,15 +333,16 @@ public void playback_withMidrollFetchError_marksAdAsInErrorState() { when(mockMidrollFetchErrorAdEvent.getType()).thenReturn(AdEventType.AD_BREAK_FETCH_ERROR); when(mockMidrollFetchErrorAdEvent.getAdData()) .thenReturn(ImmutableMap.of("adBreakTime", "20.5")); - setupPlayback(CONTENT_TIMELINE, ImmutableList.of(20.5f)); + when(mockAdsManager.getAdCuePoints()).thenReturn(ImmutableList.of(20.5f)); // Simulate loading an empty midroll ad. - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); adEventListener.onAdEvent(mockMidrollFetchErrorAdEvent); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - new AdPlaybackState(/* adGroupTimesUs...= */ 20_500_000) + new AdPlaybackState(TEST_ADS_ID, /* adGroupTimesUs...= */ 20_500_000) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withAdDurationsUs(new long[][] {{TEST_AD_DURATION_US}}) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) @@ -321,17 +355,18 @@ public void playback_withMidrollFetchError_updatesContentProgress() { when(mockMidrollFetchErrorAdEvent.getType()).thenReturn(AdEventType.AD_BREAK_FETCH_ERROR); when(mockMidrollFetchErrorAdEvent.getAdData()) .thenReturn(ImmutableMap.of("adBreakTime", "5.5")); - setupPlayback(CONTENT_TIMELINE, ImmutableList.of(5.5f)); + when(mockAdsManager.getAdCuePoints()).thenReturn(ImmutableList.of(5.5f)); // Simulate loading an empty midroll ad and advancing the player position. - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); adEventListener.onAdEvent(mockMidrollFetchErrorAdEvent); long playerPositionUs = CONTENT_DURATION_US - C.MICROS_PER_SECOND; long playerPositionInPeriodUs = playerPositionUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; long periodDurationUs = CONTENT_TIMELINE.getPeriod(/* periodIndex= */ 0, new Period()).durationUs; - fakeExoPlayer.setPlayingContentPosition(C.usToMs(playerPositionUs)); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(playerPositionUs)); // Verify the content progress is updated to reflect the new player position. assertThat(contentProgressProvider.getContentProgress()) @@ -346,15 +381,16 @@ public void playback_withPostrollFetchError_marksAdAsInErrorState() { when(mockPostrollFetchErrorAdEvent.getType()).thenReturn(AdEventType.AD_BREAK_FETCH_ERROR); when(mockPostrollFetchErrorAdEvent.getAdData()) .thenReturn(ImmutableMap.of("adBreakTime", "-1")); - setupPlayback(CONTENT_TIMELINE, ImmutableList.of(-1f)); + when(mockAdsManager.getAdCuePoints()).thenReturn(ImmutableList.of(-1f)); // Simulate loading an empty postroll ad. - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); adEventListener.onAdEvent(mockPostrollFetchErrorAdEvent); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - new AdPlaybackState(/* adGroupTimesUs...= */ C.TIME_END_OF_SOURCE) + new AdPlaybackState(TEST_ADS_ID, /* adGroupTimesUs...= */ C.TIME_END_OF_SOURCE) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withAdDurationsUs(new long[][] {{TEST_AD_DURATION_US}}) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) @@ -369,19 +405,20 @@ public void playback_withAdNotPreloadingBeforeTimeout_hasNoError() { adGroupPositionInWindowUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; ImmutableList cuePoints = ImmutableList.of((float) adGroupTimeUs / C.MICROS_PER_SECOND); - setupPlayback(CONTENT_TIMELINE, cuePoints); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); // Advance playback to just before the midroll and simulate buffering. - imaAdsLoader.start(adsLoaderListener, adViewProvider); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(adGroupPositionInWindowUs)); - fakeExoPlayer.setState(Player.STATE_BUFFERING, /* playWhenReady= */ true); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(adGroupPositionInWindowUs)); + fakePlayer.setState(Player.STATE_BUFFERING, /* playWhenReady= */ true); // Advance before the timeout and simulating polling content progress. ShadowSystemClock.advanceBy(Duration.ofSeconds(1)); contentProgressProvider.getContentProgress(); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US)); } @@ -393,25 +430,158 @@ public void playback_withAdNotPreloadingAfterTimeout_hasErrorAdGroup() { adGroupPositionInWindowUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; ImmutableList cuePoints = ImmutableList.of((float) adGroupTimeUs / C.MICROS_PER_SECOND); - setupPlayback(CONTENT_TIMELINE, cuePoints); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); // Advance playback to just before the midroll and simulate buffering. - imaAdsLoader.start(adsLoaderListener, adViewProvider); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(adGroupPositionInWindowUs)); - fakeExoPlayer.setState(Player.STATE_BUFFERING, /* playWhenReady= */ true); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(adGroupPositionInWindowUs)); + fakePlayer.setState(Player.STATE_BUFFERING, /* playWhenReady= */ true); // Advance past the timeout and simulate polling content progress. ShadowSystemClock.advanceBy(Duration.ofSeconds(5)); contentProgressProvider.getContentProgress(); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) + .isEqualTo( + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) + .withContentDurationUs(CONTENT_PERIOD_DURATION_US) + .withAdDurationsUs(new long[][] {{TEST_AD_DURATION_US}}) + .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) + .withAdLoadError(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0)); + } + + @Test + public void startPlaybackAfterMidroll_doesNotSkipMidroll() { + // Simulate an ad at 2 seconds, and starting playback with an initial seek position at the ad. + long adGroupPositionInWindowUs = 2 * C.MICROS_PER_SECOND; + long adGroupTimeUs = + adGroupPositionInWindowUs + + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; + ImmutableList cuePoints = ImmutableList.of((float) adGroupTimeUs / C.MICROS_PER_SECOND); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); + fakePlayer.setState(Player.STATE_BUFFERING, /* playWhenReady= */ true); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(adGroupPositionInWindowUs)); + + // Start ad loading while still buffering and simulate the calls from the IMA SDK to resume then + // immediately pause content playback. + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + contentProgressProvider.getContentProgress(); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_RESUME_REQUESTED, /* ad= */ null)); + contentProgressProvider.getContentProgress(); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_PAUSE_REQUESTED, /* ad= */ null)); + contentProgressProvider.getContentProgress(); + + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) + .isEqualTo( + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) + .withContentDurationUs(CONTENT_PERIOD_DURATION_US)); + } + + @Test + public void startPlaybackAfterMidroll_withAdNotPreloadingAfterTimeout_hasErrorAdGroup() { + // Simulate an ad at 2 seconds, and starting playback with an initial seek position at the ad. + long adGroupPositionInWindowUs = 2 * C.MICROS_PER_SECOND; + long adGroupTimeUs = + adGroupPositionInWindowUs + + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; + ImmutableList cuePoints = ImmutableList.of((float) adGroupTimeUs / C.MICROS_PER_SECOND); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); + fakePlayer.setState(Player.STATE_BUFFERING, /* playWhenReady= */ true); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(adGroupPositionInWindowUs)); + + // Start ad loading while still buffering and poll progress without the ad loading. + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + contentProgressProvider.getContentProgress(); + ShadowSystemClock.advanceBy(Duration.ofSeconds(5)); + contentProgressProvider.getContentProgress(); + + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withAdDurationsUs(new long[][] {{TEST_AD_DURATION_US}}) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdLoadError(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0)); } + @Test + public void bufferingDuringAd_callsOnBuffering() { + // Load the preroll ad. + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + adEventListener.onAdEvent(getAdEvent(AdEventType.LOADED, mockPrerollSingleAd)); + videoAdPlayer.loadAd(TEST_AD_MEDIA_INFO, mockAdPodInfo); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_PAUSE_REQUESTED, mockPrerollSingleAd)); + + // Play the preroll ad then simulate buffering. + videoAdPlayer.playAd(TEST_AD_MEDIA_INFO); + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* positionMs= */ 0, + /* contentPositionMs= */ 0); + fakePlayer.setState(Player.STATE_READY, /* playWhenReady= */ true); + adEventListener.onAdEvent(getAdEvent(AdEventType.STARTED, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.FIRST_QUARTILE, mockPrerollSingleAd)); + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* positionMs= */ 1_000, + /* contentPositionMs= */ 0); + fakePlayer.setState(Player.STATE_BUFFERING, /* playWhenReady= */ true); + + verify(mockVideoAdPlayerCallback).onBuffering(any()); + } + + @Test + public void resumeAfterBufferingDuringAd_updatesPosition() { + // Load the preroll ad. + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + adEventListener.onAdEvent(getAdEvent(AdEventType.LOADED, mockPrerollSingleAd)); + videoAdPlayer.loadAd(TEST_AD_MEDIA_INFO, mockAdPodInfo); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_PAUSE_REQUESTED, mockPrerollSingleAd)); + + // Play the preroll ad. + videoAdPlayer.playAd(TEST_AD_MEDIA_INFO); + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* positionMs= */ 0, + /* contentPositionMs= */ 0); + fakePlayer.setState(Player.STATE_READY, /* playWhenReady= */ true); + + // Simulate buffering. + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* positionMs= */ 2_000, + /* contentPositionMs= */ 0); + fakePlayer.setState(Player.STATE_BUFFERING, /* playWhenReady= */ true); + + // Simulate resuming and force pending ad progress updates to happen immediately. + int newPlayerPositionMs = 3_000; + fakePlayer.setState(Player.STATE_READY, /* playWhenReady= */ true); + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + newPlayerPositionMs, + /* contentPositionMs= */ 0); + shadowOf(Looper.getMainLooper()).runToEndOfTasks(); + + verify(mockVideoAdPlayerCallback) + .onAdProgress( + TEST_AD_MEDIA_INFO, + new VideoProgressUpdate(newPlayerPositionMs, C.usToMs(TEST_AD_DURATION_US))); + } + @Test public void resumePlaybackBeforeMidroll_playsPreroll() { long midrollWindowTimeUs = 2 * C.MICROS_PER_SECOND; @@ -419,15 +589,17 @@ public void resumePlaybackBeforeMidroll_playsPreroll() { midrollWindowTimeUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; ImmutableList cuePoints = ImmutableList.of(0f, (float) midrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback(CONTENT_TIMELINE, cuePoints); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(midrollWindowTimeUs) - 1_000); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition( + /* periodIndex= */ 0, C.usToMs(midrollWindowTimeUs) - 1_000); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockAdsRenderingSettings, never()).setPlayAdsAfterTime(anyDouble()); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US)); } @@ -438,10 +610,11 @@ public void resumePlaybackAtMidroll_skipsPreroll() { midrollWindowTimeUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; ImmutableList cuePoints = ImmutableList.of(0f, (float) midrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback(CONTENT_TIMELINE, cuePoints); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(midrollWindowTimeUs)); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(midrollWindowTimeUs)); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); ArgumentCaptor playAdsAfterTimeCaptor = ArgumentCaptor.forClass(Double.class); verify(mockAdsRenderingSettings).setPlayAdsAfterTime(playAdsAfterTimeCaptor.capture()); @@ -449,9 +622,9 @@ public void resumePlaybackAtMidroll_skipsPreroll() { assertThat(playAdsAfterTimeCaptor.getValue()) .isWithin(0.1) .of(expectedPlayAdsAfterTimeUs / C.MICROS_PER_SECOND); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withSkippedAdGroup(/* adGroupIndex= */ 0)); } @@ -463,10 +636,12 @@ public void resumePlaybackAfterMidroll_skipsPreroll() { midrollWindowTimeUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; ImmutableList cuePoints = ImmutableList.of(0f, (float) midrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback(CONTENT_TIMELINE, cuePoints); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(midrollWindowTimeUs) + 1_000); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition( + /* periodIndex= */ 0, C.usToMs(midrollWindowTimeUs) + 1_000); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); ArgumentCaptor playAdsAfterTimeCaptor = ArgumentCaptor.forClass(Double.class); verify(mockAdsRenderingSettings).setPlayAdsAfterTime(playAdsAfterTimeCaptor.capture()); @@ -474,9 +649,9 @@ public void resumePlaybackAfterMidroll_skipsPreroll() { assertThat(playAdsAfterTimeCaptor.getValue()) .isWithin(0.1) .of(expectedPlayAdsAfterTimeUs / C.MICROS_PER_SECOND); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withSkippedAdGroup(/* adGroupIndex= */ 0)); } @@ -495,15 +670,17 @@ public void resumePlaybackBeforeSecondMidroll_playsFirstMidroll() { ImmutableList.of( (float) firstMidrollPeriodTimeUs / C.MICROS_PER_SECOND, (float) secondMidrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback(CONTENT_TIMELINE, cuePoints); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(secondMidrollWindowTimeUs) - 1_000); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition( + /* periodIndex= */ 0, C.usToMs(secondMidrollWindowTimeUs) - 1_000); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockAdsRenderingSettings, never()).setPlayAdsAfterTime(anyDouble()); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US)); } @@ -521,10 +698,11 @@ public void resumePlaybackAtSecondMidroll_skipsFirstMidroll() { ImmutableList.of( (float) firstMidrollPeriodTimeUs / C.MICROS_PER_SECOND, (float) secondMidrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback(CONTENT_TIMELINE, cuePoints); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(secondMidrollWindowTimeUs)); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(secondMidrollWindowTimeUs)); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); ArgumentCaptor playAdsAfterTimeCaptor = ArgumentCaptor.forClass(Double.class); verify(mockAdsRenderingSettings).setPlayAdsAfterTime(playAdsAfterTimeCaptor.capture()); @@ -532,32 +710,41 @@ public void resumePlaybackAtSecondMidroll_skipsFirstMidroll() { assertThat(playAdsAfterTimeCaptor.getValue()) .isWithin(0.1) .of(expectedPlayAdsAfterTimeUs / C.MICROS_PER_SECOND); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withSkippedAdGroup(/* adGroupIndex= */ 0)); } @Test public void resumePlaybackBeforeMidroll_withoutPlayAdBeforeStartPosition_skipsPreroll() { + imaAdsLoader = + new ImaAdsLoader.Builder(getApplicationContext()) + .setPlayAdBeforeStartPosition(false) + .setImaFactory(mockImaFactory) + .setImaSdkSettings(mockImaSdkSettings) + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); long midrollWindowTimeUs = 2 * C.MICROS_PER_SECOND; long midrollPeriodTimeUs = midrollWindowTimeUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; ImmutableList cuePoints = ImmutableList.of(0f, (float) midrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback( - CONTENT_TIMELINE, - cuePoints, - new ImaAdsLoader.Builder(getApplicationContext()) - .setPlayAdBeforeStartPosition(false) - .setImaFactory(mockImaFactory) - .setImaSdkSettings(mockImaSdkSettings) - .build(), - TEST_DATA_SPEC); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(midrollWindowTimeUs) - 1_000); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition( + /* periodIndex= */ 0, C.usToMs(midrollWindowTimeUs) - 1_000); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); ArgumentCaptor playAdsAfterTimeCaptor = ArgumentCaptor.forClass(Double.class); verify(mockAdsRenderingSettings).setPlayAdsAfterTime(playAdsAfterTimeCaptor.capture()); @@ -565,32 +752,40 @@ public void resumePlaybackBeforeMidroll_withoutPlayAdBeforeStartPosition_skipsPr assertThat(playAdsAfterTimeCaptor.getValue()) .isWithin(0.1d) .of(expectedPlayAdsAfterTimeUs / C.MICROS_PER_SECOND); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withSkippedAdGroup(/* adGroupIndex= */ 0) .withContentDurationUs(CONTENT_PERIOD_DURATION_US)); } @Test public void resumePlaybackAtMidroll_withoutPlayAdBeforeStartPosition_skipsPreroll() { + imaAdsLoader = + new ImaAdsLoader.Builder(getApplicationContext()) + .setPlayAdBeforeStartPosition(false) + .setImaFactory(mockImaFactory) + .setImaSdkSettings(mockImaSdkSettings) + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); long midrollWindowTimeUs = 2 * C.MICROS_PER_SECOND; long midrollPeriodTimeUs = midrollWindowTimeUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; ImmutableList cuePoints = ImmutableList.of(0f, (float) midrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback( - CONTENT_TIMELINE, - cuePoints, - new ImaAdsLoader.Builder(getApplicationContext()) - .setPlayAdBeforeStartPosition(false) - .setImaFactory(mockImaFactory) - .setImaSdkSettings(mockImaSdkSettings) - .build(), - TEST_DATA_SPEC); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(midrollWindowTimeUs)); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(midrollWindowTimeUs)); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); ArgumentCaptor playAdsAfterTimeCaptor = ArgumentCaptor.forClass(Double.class); verify(mockAdsRenderingSettings).setPlayAdsAfterTime(playAdsAfterTimeCaptor.capture()); @@ -598,37 +793,46 @@ public void resumePlaybackAtMidroll_withoutPlayAdBeforeStartPosition_skipsPrerol assertThat(playAdsAfterTimeCaptor.getValue()) .isWithin(0.1d) .of(expectedPlayAdsAfterTimeUs / C.MICROS_PER_SECOND); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withSkippedAdGroup(/* adGroupIndex= */ 0)); } @Test public void resumePlaybackAfterMidroll_withoutPlayAdBeforeStartPosition_skipsMidroll() { + imaAdsLoader = + new ImaAdsLoader.Builder(getApplicationContext()) + .setPlayAdBeforeStartPosition(false) + .setImaFactory(mockImaFactory) + .setImaSdkSettings(mockImaSdkSettings) + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); long midrollWindowTimeUs = 2 * C.MICROS_PER_SECOND; long midrollPeriodTimeUs = midrollWindowTimeUs + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; ImmutableList cuePoints = ImmutableList.of(0f, (float) midrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback( - CONTENT_TIMELINE, - cuePoints, - new ImaAdsLoader.Builder(getApplicationContext()) - .setPlayAdBeforeStartPosition(false) - .setImaFactory(mockImaFactory) - .setImaSdkSettings(mockImaSdkSettings) - .build(), - TEST_DATA_SPEC); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(midrollWindowTimeUs) + 1_000); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition( + /* periodIndex= */ 0, C.usToMs(midrollWindowTimeUs) + 1_000); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockAdsManager).destroy(); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withSkippedAdGroup(/* adGroupIndex= */ 0) .withSkippedAdGroup(/* adGroupIndex= */ 1)); @@ -637,6 +841,21 @@ public void resumePlaybackAfterMidroll_withoutPlayAdBeforeStartPosition_skipsMid @Test public void resumePlaybackBeforeSecondMidroll_withoutPlayAdBeforeStartPosition_skipsFirstMidroll() { + imaAdsLoader = + new ImaAdsLoader.Builder(getApplicationContext()) + .setPlayAdBeforeStartPosition(false) + .setImaFactory(mockImaFactory) + .setImaSdkSettings(mockImaSdkSettings) + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); long firstMidrollWindowTimeUs = 2 * C.MICROS_PER_SECOND; long firstMidrollPeriodTimeUs = firstMidrollWindowTimeUs @@ -649,18 +868,12 @@ public void resumePlaybackAfterMidroll_withoutPlayAdBeforeStartPosition_skipsMid ImmutableList.of( (float) firstMidrollPeriodTimeUs / C.MICROS_PER_SECOND, (float) secondMidrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback( - CONTENT_TIMELINE, - cuePoints, - new ImaAdsLoader.Builder(getApplicationContext()) - .setPlayAdBeforeStartPosition(false) - .setImaFactory(mockImaFactory) - .setImaSdkSettings(mockImaSdkSettings) - .build(), - TEST_DATA_SPEC); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(secondMidrollWindowTimeUs) - 1_000); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition( + /* periodIndex= */ 0, C.usToMs(secondMidrollWindowTimeUs) - 1_000); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); ArgumentCaptor playAdsAfterTimeCaptor = ArgumentCaptor.forClass(Double.class); verify(mockAdsRenderingSettings).setPlayAdsAfterTime(playAdsAfterTimeCaptor.capture()); @@ -668,15 +881,30 @@ public void resumePlaybackAfterMidroll_withoutPlayAdBeforeStartPosition_skipsMid assertThat(playAdsAfterTimeCaptor.getValue()) .isWithin(0.1d) .of(expectedPlayAdsAfterTimeUs / C.MICROS_PER_SECOND); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withSkippedAdGroup(/* adGroupIndex= */ 0) .withContentDurationUs(CONTENT_PERIOD_DURATION_US)); } @Test public void resumePlaybackAtSecondMidroll_withoutPlayAdBeforeStartPosition_skipsFirstMidroll() { + imaAdsLoader = + new ImaAdsLoader.Builder(getApplicationContext()) + .setPlayAdBeforeStartPosition(false) + .setImaFactory(mockImaFactory) + .setImaSdkSettings(mockImaSdkSettings) + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); long firstMidrollWindowTimeUs = 2 * C.MICROS_PER_SECOND; long firstMidrollPeriodTimeUs = firstMidrollWindowTimeUs @@ -689,18 +917,11 @@ public void resumePlaybackAtSecondMidroll_withoutPlayAdBeforeStartPosition_skips ImmutableList.of( (float) firstMidrollPeriodTimeUs / C.MICROS_PER_SECOND, (float) secondMidrollPeriodTimeUs / C.MICROS_PER_SECOND); - setupPlayback( - CONTENT_TIMELINE, - cuePoints, - new ImaAdsLoader.Builder(getApplicationContext()) - .setPlayAdBeforeStartPosition(false) - .setImaFactory(mockImaFactory) - .setImaSdkSettings(mockImaSdkSettings) - .build(), - TEST_DATA_SPEC); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - fakeExoPlayer.setPlayingContentPosition(C.usToMs(secondMidrollWindowTimeUs)); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, C.usToMs(secondMidrollWindowTimeUs)); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); ArgumentCaptor playAdsAfterTimeCaptor = ArgumentCaptor.forClass(Double.class); verify(mockAdsRenderingSettings).setPlayAdsAfterTime(playAdsAfterTimeCaptor.capture()); @@ -708,9 +929,9 @@ public void resumePlaybackAtSecondMidroll_withoutPlayAdBeforeStartPosition_skips assertThat(playAdsAfterTimeCaptor.getValue()) .isWithin(0.1d) .of(expectedPlayAdsAfterTimeUs / C.MICROS_PER_SECOND); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withSkippedAdGroup(/* adGroupIndex= */ 0)); } @@ -726,40 +947,24 @@ public void requestAdTagWithDataScheme_requestsWithAdsResponse() throws Exceptio + " \n" + ""; DataSpec adDataSpec = new DataSpec(Util.getDataUriForString("text/xml", adsResponse)); - - setupPlayback( - CONTENT_TIMELINE, - ImmutableList.of(0f), - new ImaAdsLoader.Builder(getApplicationContext()) - .setImaFactory(mockImaFactory) - .setImaSdkSettings(mockImaSdkSettings) - .build(), - adDataSpec); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start(adsMediaSource, adDataSpec, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockAdsRequest).setAdsResponse(adsResponse); } @Test public void requestAdTagWithUri_requestsWithAdTagUrl() throws Exception { - setupPlayback( - CONTENT_TIMELINE, - ImmutableList.of(0f), - new ImaAdsLoader.Builder(getApplicationContext()) - .setImaFactory(mockImaFactory) - .setImaSdkSettings(mockImaSdkSettings) - .build(), - TEST_DATA_SPEC); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockAdsRequest).setAdTagUrl(TEST_DATA_SPEC.uri.toString()); } @Test public void setsDefaultMimeTypes() throws Exception { - setupPlayback(CONTENT_TIMELINE, ImmutableList.of(0f)); imaAdsLoader.setSupportedContentTypes(C.TYPE_DASH, C.TYPE_OTHER); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockAdsRenderingSettings) .setMimeTypes( @@ -774,27 +979,36 @@ public void setsDefaultMimeTypes() throws Exception { @Test public void buildWithAdMediaMimeTypes_setsMimeTypes() throws Exception { - setupPlayback( - CONTENT_TIMELINE, - ImmutableList.of(0f), + imaAdsLoader = new ImaAdsLoader.Builder(getApplicationContext()) + .setAdMediaMimeTypes(ImmutableList.of(MimeTypes.AUDIO_MPEG)) .setImaFactory(mockImaFactory) .setImaSdkSettings(mockImaSdkSettings) - .setAdMediaMimeTypes(ImmutableList.of(MimeTypes.AUDIO_MPEG)) - .build(), - TEST_DATA_SPEC); + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); + when(mockAdsManager.getAdCuePoints()).thenReturn(PREROLL_CUE_POINTS_SECONDS); + imaAdsLoader.setSupportedContentTypes(C.TYPE_OTHER); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); verify(mockAdsRenderingSettings).setMimeTypes(ImmutableList.of(MimeTypes.AUDIO_MPEG)); } @Test public void stop_unregistersAllVideoControlOverlays() { - setupPlayback(CONTENT_TIMELINE, PREROLL_CUE_POINTS_SECONDS); - imaAdsLoader.start(adsLoaderListener, adViewProvider); - imaAdsLoader.requestAds(TEST_DATA_SPEC, adViewGroup); - imaAdsLoader.stop(); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + imaAdsLoader.requestAds(TEST_DATA_SPEC, TEST_ADS_ID, adViewGroup); + imaAdsLoader.stop(adsMediaSource, adsLoaderListener); InOrder inOrder = inOrder(mockAdDisplayContainer); inOrder.verify(mockAdDisplayContainer).registerFriendlyObstruction(mockFriendlyObstruction); @@ -805,9 +1019,11 @@ public void stop_unregistersAllVideoControlOverlays() { public void loadAd_withLargeAdCuePoint_updatesAdPlaybackStateWithLoadedAd() { // Use a large enough value to test correct truncating of large cue points. float midrollTimeSecs = Float.MAX_VALUE; - ImmutableList cuePoints = ImmutableList.of(midrollTimeSecs); - setupPlayback(CONTENT_TIMELINE, cuePoints); - imaAdsLoader.start(adsLoaderListener, adViewProvider); + List cuePoints = ImmutableList.of(midrollTimeSecs); + when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); + + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); videoAdPlayer.loadAd( TEST_AD_MEDIA_INFO, new AdPodInfo() { @@ -842,37 +1058,236 @@ public double getTimeOffset() { } }); - assertThat(adsLoaderListener.adPlaybackState) + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) .isEqualTo( - ImaUtil.getInitialAdPlaybackStateForCuePoints(cuePoints) + new AdPlaybackState(TEST_ADS_ID, getAdGroupTimesUsForCuePoints(cuePoints)) .withContentDurationUs(CONTENT_PERIOD_DURATION_US) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdUri(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, TEST_URI) .withAdDurationsUs(new long[][] {{TEST_AD_DURATION_US}})); } - private void setupPlayback(Timeline contentTimeline, List cuePoints) { - setupPlayback( - contentTimeline, - cuePoints, + @Test + public void playbackWithTwoAdsMediaSources_preloadsSecondAdTag() { + Object secondAdsId = new Object(); + AdsMediaSource secondAdsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + secondAdsId, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); + timelineWindowDefinitions = + new TimelineWindowDefinition[] { + getInitialTimelineWindowDefinition(TEST_ADS_ID), + getInitialTimelineWindowDefinition(secondAdsId) + }; + TestAdsLoaderListener secondAdsLoaderListener = new TestAdsLoaderListener(/* periodIndex= */ 1); + + // Load and play the preroll ad then content. + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + adEventListener.onAdEvent(getAdEvent(AdEventType.LOADED, mockPrerollSingleAd)); + videoAdPlayer.loadAd(TEST_AD_MEDIA_INFO, mockAdPodInfo); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_PAUSE_REQUESTED, mockPrerollSingleAd)); + videoAdPlayer.playAd(TEST_AD_MEDIA_INFO); + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* position= */ 0, + /* contentPosition= */ 0); + fakePlayer.setState(Player.STATE_READY, /* playWhenReady= */ true); + adEventListener.onAdEvent(getAdEvent(AdEventType.STARTED, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.FIRST_QUARTILE, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.MIDPOINT, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.THIRD_QUARTILE, mockPrerollSingleAd)); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, /* positionMs= */ 0); + videoAdPlayer.stopAd(TEST_AD_MEDIA_INFO); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_RESUME_REQUESTED, /* ad= */ null)); + + // Simulate starting to buffer the second ads media source. + imaAdsLoader.start( + secondAdsMediaSource, TEST_DATA_SPEC, secondAdsId, adViewProvider, secondAdsLoaderListener); + + // Verify that the preroll ad has been marked as played. + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) + .isEqualTo( + new AdPlaybackState(TEST_ADS_ID, /* adGroupTimesUs...= */ 0) + .withContentDurationUs(CONTENT_PERIOD_DURATION_US) + .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) + .withAdUri(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, TEST_URI) + .withAdDurationsUs(new long[][] {{TEST_AD_DURATION_US}}) + .withPlayedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0) + .withAdResumePositionUs(/* adResumePositionUs= */ 0)); + // Verify that the second source's ad cue points have preloaded. + assertThat(getAdPlaybackState(/* periodIndex= */ 1)) + .isEqualTo(new AdPlaybackState(secondAdsId, /* adGroupTimesUs...= */ 0)); + } + + @Test + public void playbackWithTwoAdsMediaSources_preloadsSecondAdTagWithBackgroundResume() { + Object secondAdsId = new Object(); + AdsMediaSource secondAdsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + secondAdsId, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); + timelineWindowDefinitions = + new TimelineWindowDefinition[] { + getInitialTimelineWindowDefinition(TEST_ADS_ID), + getInitialTimelineWindowDefinition(secondAdsId) + }; + TestAdsLoaderListener secondAdsLoaderListener = new TestAdsLoaderListener(/* periodIndex= */ 1); + + // Load and play the preroll ad then content. + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + adEventListener.onAdEvent(getAdEvent(AdEventType.LOADED, mockPrerollSingleAd)); + videoAdPlayer.loadAd(TEST_AD_MEDIA_INFO, mockAdPodInfo); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_PAUSE_REQUESTED, mockPrerollSingleAd)); + videoAdPlayer.playAd(TEST_AD_MEDIA_INFO); + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* position= */ 0, + /* contentPosition= */ 0); + fakePlayer.setState(Player.STATE_READY, /* playWhenReady= */ true); + adEventListener.onAdEvent(getAdEvent(AdEventType.STARTED, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.FIRST_QUARTILE, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.MIDPOINT, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.THIRD_QUARTILE, mockPrerollSingleAd)); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, /* positionMs= */ 0); + videoAdPlayer.stopAd(TEST_AD_MEDIA_INFO); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_RESUME_REQUESTED, /* ad= */ null)); + + // Simulate starting to buffer the second ads media source. + imaAdsLoader.start( + secondAdsMediaSource, TEST_DATA_SPEC, secondAdsId, adViewProvider, secondAdsLoaderListener); + + // Simulate backgrounding/resuming. + imaAdsLoader.stop(adsMediaSource, adsLoaderListener); + imaAdsLoader.stop(secondAdsMediaSource, secondAdsLoaderListener); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + imaAdsLoader.start( + secondAdsMediaSource, TEST_DATA_SPEC, secondAdsId, adViewProvider, secondAdsLoaderListener); + + // Verify that the preroll ad has been marked as played. + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) + .isEqualTo( + new AdPlaybackState(TEST_ADS_ID, /* adGroupTimesUs...= */ 0) + .withContentDurationUs(CONTENT_PERIOD_DURATION_US) + .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) + .withAdUri(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, TEST_URI) + .withAdDurationsUs(new long[][] {{TEST_AD_DURATION_US}}) + .withPlayedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0) + .withAdResumePositionUs(/* adResumePositionUs= */ 0)); + // Verify that the second source's ad cue points have preloaded. + assertThat(getAdPlaybackState(/* periodIndex= */ 1)) + .isEqualTo(new AdPlaybackState(secondAdsId, /* adGroupTimesUs...= */ 0)); + } + + @Test + public void playbackWithTwoAdsMediaSourcesAndMatchingAdsIds_hasMatchingAdPlaybackState() { + AdsMediaSource secondAdsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); + timelineWindowDefinitions = + new TimelineWindowDefinition[] { + getInitialTimelineWindowDefinition(TEST_ADS_ID), + getInitialTimelineWindowDefinition(TEST_ADS_ID) + }; + TestAdsLoaderListener secondAdsLoaderListener = new TestAdsLoaderListener(/* periodIndex= */ 1); + + // Load and play the preroll ad then content. + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + adEventListener.onAdEvent(getAdEvent(AdEventType.LOADED, mockPrerollSingleAd)); + videoAdPlayer.loadAd(TEST_AD_MEDIA_INFO, mockAdPodInfo); + imaAdsLoader.start( + secondAdsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, secondAdsLoaderListener); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_PAUSE_REQUESTED, mockPrerollSingleAd)); + videoAdPlayer.playAd(TEST_AD_MEDIA_INFO); + fakePlayer.setPlayingAdPosition( + /* periodIndex= */ 0, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* positionMs= */ 0, + /* contentPositionMs= */ 0); + fakePlayer.setState(Player.STATE_READY, /* playWhenReady= */ true); + adEventListener.onAdEvent(getAdEvent(AdEventType.STARTED, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.FIRST_QUARTILE, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.MIDPOINT, mockPrerollSingleAd)); + adEventListener.onAdEvent(getAdEvent(AdEventType.THIRD_QUARTILE, mockPrerollSingleAd)); + fakePlayer.setPlayingContentPosition(/* periodIndex= */ 0, /* positionMs= */ 0); + videoAdPlayer.stopAd(TEST_AD_MEDIA_INFO); + adEventListener.onAdEvent(getAdEvent(AdEventType.CONTENT_RESUME_REQUESTED, /* ad= */ null)); + + // Verify that the ad playback states for the two periods match. + assertThat(getAdPlaybackState(/* periodIndex= */ 0)) + .isEqualTo(getAdPlaybackState(/* periodIndex= */ 1)); + } + + @Test + public void buildWithDefaultEnableContinuousPlayback_doesNotSetAdsRequestProperty() { + imaAdsLoader = new ImaAdsLoader.Builder(getApplicationContext()) .setImaFactory(mockImaFactory) .setImaSdkSettings(mockImaSdkSettings) - .build(), - TEST_DATA_SPEC); + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); + when(mockAdsManager.getAdCuePoints()).thenReturn(PREROLL_CUE_POINTS_SECONDS); + + imaAdsLoader.setSupportedContentTypes(C.TYPE_OTHER); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + + verify(mockAdsRequest, never()).setContinuousPlayback(anyBoolean()); } - private void setupPlayback( - Timeline contentTimeline, - List cuePoints, - ImaAdsLoader imaAdsLoader, - DataSpec adTagDataSpec) { - fakeExoPlayer = new FakePlayer(); - adsLoaderListener = new TestAdsLoaderListener(fakeExoPlayer, contentTimeline); - when(mockAdsManager.getAdCuePoints()).thenReturn(cuePoints); - this.imaAdsLoader = imaAdsLoader; - imaAdsLoader.setPlayer(fakeExoPlayer); - imaAdsLoader.setAdTagDataSpec(adTagDataSpec); + @Test + public void buildWithEnableContinuousPlayback_setsAdsRequestProperty() { + imaAdsLoader = + new ImaAdsLoader.Builder(getApplicationContext()) + .setEnableContinuousPlayback(true) + .setImaFactory(mockImaFactory) + .setImaSdkSettings(mockImaSdkSettings) + .build(); + imaAdsLoader.setPlayer(fakePlayer); + adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(CONTENT_TIMELINE), + TEST_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) getApplicationContext()), + imaAdsLoader, + adViewProvider); + when(mockAdsManager.getAdCuePoints()).thenReturn(PREROLL_CUE_POINTS_SECONDS); + + imaAdsLoader.setSupportedContentTypes(C.TYPE_OTHER); + imaAdsLoader.start( + adsMediaSource, TEST_DATA_SPEC, TEST_ADS_ID, adViewProvider, adsLoaderListener); + + verify(mockAdsRequest).setContinuousPlayback(true); } private void setupMocks() { @@ -949,6 +1364,10 @@ private void setupMocks() { when(mockPrerollSingleAd.getAdPodInfo()).thenReturn(mockAdPodInfo); } + private AdPlaybackState getAdPlaybackState(int periodIndex) { + return timelineWindowDefinitions[periodIndex].adPlaybackState; + } + private static AdEvent getAdEvent(AdEventType adEventType, @Nullable Ad ad) { return new AdEvent() { @Override @@ -970,16 +1389,12 @@ public Map getAdData() { } /** Ad loader event listener that forwards ad playback state to a fake player. */ - private static final class TestAdsLoaderListener implements AdsLoader.EventListener { + private final class TestAdsLoaderListener implements AdsLoader.EventListener { - private final FakePlayer fakeExoPlayer; - private final Timeline contentTimeline; + private final int periodIndex; - public AdPlaybackState adPlaybackState; - - public TestAdsLoaderListener(FakePlayer fakeExoPlayer, Timeline contentTimeline) { - this.fakeExoPlayer = fakeExoPlayer; - this.contentTimeline = contentTimeline; + public TestAdsLoaderListener(int periodIndex) { + this.periodIndex = periodIndex; } @Override @@ -990,10 +1405,23 @@ public void onAdPlaybackState(AdPlaybackState adPlaybackState) { Arrays.fill(adDurationsUs[adGroupIndex], TEST_AD_DURATION_US); } adPlaybackState = adPlaybackState.withAdDurationsUs(adDurationsUs); - this.adPlaybackState = adPlaybackState; - fakeExoPlayer.updateTimeline( - new SinglePeriodAdTimeline(contentTimeline, adPlaybackState), - Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE); + + TimelineWindowDefinition timelineWindowDefinition = timelineWindowDefinitions[periodIndex]; + assertThat(adPlaybackState.adsId).isEqualTo(timelineWindowDefinition.adPlaybackState.adsId); + timelineWindowDefinitions[periodIndex] = + new TimelineWindowDefinition( + timelineWindowDefinition.periodCount, + timelineWindowDefinition.id, + timelineWindowDefinition.isSeekable, + timelineWindowDefinition.isDynamic, + timelineWindowDefinition.isLive, + timelineWindowDefinition.isPlaceholder, + timelineWindowDefinition.durationUs, + timelineWindowDefinition.defaultPositionUs, + timelineWindowDefinition.windowOffsetInFirstPeriodUs, + adPlaybackState); + fakePlayer.updateTimeline( + new FakeTimeline(timelineWindowDefinitions), Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE); } @Override @@ -1011,4 +1439,24 @@ public void onAdTapped() { // Do nothing. } } + + private static TimelineWindowDefinition getInitialTimelineWindowDefinition(Object adsId) { + return getInitialTimelineWindowDefinition(adsId, /* isPlaceholder= */ false); + } + + private static TimelineWindowDefinition getInitialTimelineWindowDefinition( + Object adsId, boolean isPlaceholder) { + return new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ new Object(), + /* isSeekable= */ true, + /* isDynamic= */ false, + /* isLive= */ false, + /* isPlaceholder= */ isPlaceholder, + /* durationUs= */ CONTENT_DURATION_US, + /* defaultPositionUs= */ 0, + /* windowOffsetInFirstPeriodUs= */ TimelineWindowDefinition + .DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, + new AdPlaybackState(adsId)); + } } diff --git a/extensions/media2/src/androidTest/AndroidManifest.xml b/extensions/media2/src/androidTest/AndroidManifest.xml index b699de67b16..4a9d03e991b 100644 --- a/extensions/media2/src/androidTest/AndroidManifest.xml +++ b/extensions/media2/src/androidTest/AndroidManifest.xml @@ -22,15 +22,6 @@ - - - - diff --git a/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/MediaStubActivity.java b/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/MediaStubActivity.java deleted file mode 100644 index 23a44913891..00000000000 --- a/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/MediaStubActivity.java +++ /dev/null @@ -1,83 +0,0 @@ -/* - * Copyright 2019 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package com.google.android.exoplayer2.ext.media2; - -import android.app.Activity; -import android.app.KeyguardManager; -import android.content.Context; -import android.os.Bundle; -import android.util.Log; -import android.view.SurfaceHolder; -import android.view.SurfaceView; -import android.view.WindowManager; -import com.google.android.exoplayer2.ext.media2.test.R; -import com.google.android.exoplayer2.util.Util; - -/** Stub activity to play media contents on. */ -public final class MediaStubActivity extends Activity { - - private static final String TAG = "MediaStubActivity"; - - @Override - protected void onCreate(Bundle savedInstanceState) { - super.onCreate(savedInstanceState); - setContentView(R.layout.mediaplayer); - - // disable enter animation. - overridePendingTransition(0, 0); - - if (Util.SDK_INT >= 27) { - getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON); - setTurnScreenOn(true); - setShowWhenLocked(true); - KeyguardManager keyguardManager = - (KeyguardManager) getSystemService(Context.KEYGUARD_SERVICE); - keyguardManager.requestDismissKeyguard(this, null); - } else { - getWindow() - .addFlags( - WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON - | WindowManager.LayoutParams.FLAG_TURN_SCREEN_ON - | WindowManager.LayoutParams.FLAG_SHOW_WHEN_LOCKED - | WindowManager.LayoutParams.FLAG_DISMISS_KEYGUARD); - } - } - - @Override - public void finish() { - super.finish(); - - // disable exit animation. - overridePendingTransition(0, 0); - } - - @Override - protected void onResume() { - Log.i(TAG, "onResume"); - super.onResume(); - } - - @Override - protected void onPause() { - Log.i(TAG, "onPause"); - super.onPause(); - } - - public SurfaceHolder getSurfaceHolder() { - SurfaceView surface = findViewById(R.id.surface); - return surface.getHolder(); - } -} diff --git a/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/PlayerTestRule.java b/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/PlayerTestRule.java index df6963c2fc1..a995bfdb5ba 100644 --- a/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/PlayerTestRule.java +++ b/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/PlayerTestRule.java @@ -53,14 +53,6 @@ public interface DataSourceInstrumentation { @Override protected void before() { - // Workaround limitation in androidx.media2.session:1.0.3 which session can only be instantiated - // on thread with prepared Looper. - // TODO: Remove when androidx.media2.session:1.1.0 is released without the limitation - // [Internal: b/146536708] - if (Looper.myLooper() == null) { - Looper.prepare(); - } - context = ApplicationProvider.getApplicationContext(); executor = Executors.newFixedThreadPool(1); diff --git a/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/SessionCallbackBuilderTest.java b/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/SessionCallbackBuilderTest.java index c578b0ba8c5..64af343455a 100644 --- a/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/SessionCallbackBuilderTest.java +++ b/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/SessionCallbackBuilderTest.java @@ -24,7 +24,6 @@ import android.net.Uri; import android.os.Bundle; import android.text.TextUtils; -import androidx.annotation.NonNull; import androidx.annotation.Nullable; import androidx.core.content.ContextCompat; import androidx.media2.common.MediaItem; @@ -41,9 +40,6 @@ import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import androidx.test.filters.LargeTest; -import androidx.test.platform.app.InstrumentationRegistry; -import androidx.test.rule.ActivityTestRule; -import com.google.android.exoplayer2.SimpleExoPlayer; import com.google.android.exoplayer2.ext.media2.test.R; import com.google.android.exoplayer2.upstream.RawResourceDataSource; import java.util.ArrayList; @@ -60,9 +56,6 @@ /** Tests {@link SessionCallbackBuilder}. */ @RunWith(AndroidJUnit4.class) public class SessionCallbackBuilderTest { - @Rule - public final ActivityTestRule activityRule = - new ActivityTestRule<>(MediaStubActivity.class); @Rule public final PlayerTestRule playerTestRule = new PlayerTestRule(); @@ -80,16 +73,6 @@ public void setUp() { context = ApplicationProvider.getApplicationContext(); executor = playerTestRule.getExecutor(); sessionPlayerConnector = playerTestRule.getSessionPlayerConnector(); - - // Sets the surface to the player for manual check. - InstrumentationRegistry.getInstrumentation() - .runOnMainSync( - () -> { - SimpleExoPlayer exoPlayer = playerTestRule.getSimpleExoPlayer(); - exoPlayer - .getVideoComponent() - .setVideoSurfaceHolder(activityRule.getActivity().getSurfaceHolder()); - }); } @Test @@ -469,8 +452,7 @@ public void setMediaItemProvider_withMediaItemProvider_receivesOnCreateMediaItem executor, new SessionPlayer.PlayerCallback() { @Override - public void onCurrentMediaItemChanged( - @NonNull SessionPlayer player, @NonNull MediaItem item) { + public void onCurrentMediaItemChanged(SessionPlayer player, MediaItem item) { MediaMetadata metadata = item.getMetadata(); assertThat(metadata.getString(MediaMetadata.METADATA_KEY_MEDIA_ID)) .isEqualTo(testMediaUri.toString()); @@ -613,15 +595,14 @@ private MediaController createConnectedController( new MediaController.ControllerCallback() { @Override public void onAllowedCommandsChanged( - @NonNull MediaController controller, @NonNull SessionCommandGroup commands) { + MediaController controller, SessionCommandGroup commands) { if (onAllowedCommandsChangedListener != null) { onAllowedCommandsChangedListener.onAllowedCommandsChanged(controller, commands); } } @Override - public void onConnected( - @NonNull MediaController controller, @NonNull SessionCommandGroup allowedCommands) { + public void onConnected(MediaController controller, SessionCommandGroup allowedCommands) { if (onConnectedListener != null) { onConnectedListener.onConnected(controller, allowedCommands); } diff --git a/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/SessionPlayerConnectorTest.java b/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/SessionPlayerConnectorTest.java index 1747b3aed74..edabd558122 100644 --- a/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/SessionPlayerConnectorTest.java +++ b/extensions/media2/src/androidTest/java/com/google/android/exoplayer2/ext/media2/SessionPlayerConnectorTest.java @@ -31,7 +31,6 @@ import android.os.Build; import android.os.Build.VERSION_CODES; import android.os.Looper; -import androidx.annotation.NonNull; import androidx.annotation.Nullable; import androidx.core.util.ObjectsCompat; import androidx.media.AudioAttributesCompat; @@ -47,7 +46,6 @@ import androidx.test.filters.SdkSuppress; import androidx.test.filters.SmallTest; import androidx.test.platform.app.InstrumentationRegistry; -import androidx.test.rule.ActivityTestRule; import com.google.android.exoplayer2.ControlDispatcher; import com.google.android.exoplayer2.DefaultControlDispatcher; import com.google.android.exoplayer2.Player; @@ -65,6 +63,7 @@ import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicReference; import org.junit.Before; +import org.junit.Ignore; import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; @@ -73,9 +72,6 @@ @SuppressWarnings("FutureReturnValueIgnored") @RunWith(AndroidJUnit4.class) public class SessionPlayerConnectorTest { - @Rule - public final ActivityTestRule activityRule = - new ActivityTestRule<>(MediaStubActivity.class); @Rule public final PlayerTestRule playerTestRule = new PlayerTestRule(); @@ -93,16 +89,6 @@ public void setUp() { context = ApplicationProvider.getApplicationContext(); executor = playerTestRule.getExecutor(); sessionPlayerConnector = playerTestRule.getSessionPlayerConnector(); - - // Sets the surface to the player for manual check. - InstrumentationRegistry.getInstrumentation() - .runOnMainSync( - () -> { - SimpleExoPlayer exoPlayer = playerTestRule.getSimpleExoPlayer(); - exoPlayer - .getVideoComponent() - .setVideoSurfaceHolder(activityRule.getActivity().getSurfaceHolder()); - }); } @Test @@ -120,7 +106,7 @@ public void play_onceWithAudioResource_changesPlayerStateToPlaying() throws Exce executor, new SessionPlayer.PlayerCallback() { @Override - public void onPlayerStateChanged(@NonNull SessionPlayer player, int playerState) { + public void onPlayerStateChanged(SessionPlayer player, int playerState) { if (playerState == PLAYER_STATE_PLAYING) { onPlayingLatch.countDown(); } @@ -157,8 +143,7 @@ public void play_onceWithAudioResourceOnMainThread_notifiesOnPlayerStateChanged( executor, new SessionPlayer.PlayerCallback() { @Override - public void onPlayerStateChanged( - @NonNull SessionPlayer player, int playerState) { + public void onPlayerStateChanged(SessionPlayer player, int playerState) { if (playerState == PLAYER_STATE_PLAYING) { onPlayerStatePlayingLatch.countDown(); } @@ -218,7 +203,7 @@ public void setMediaItem_withAudioResource_notifiesOnPlaybackCompleted() throws executor, new SessionPlayer.PlayerCallback() { @Override - public void onPlaybackCompleted(@NonNull SessionPlayer player) { + public void onPlaybackCompleted(SessionPlayer player) { onPlaybackCompletedLatch.countDown(); } }); @@ -242,7 +227,7 @@ public void setMediaItem_withVideoResource_notifiesOnPlaybackCompleted() throws executor, new SessionPlayer.PlayerCallback() { @Override - public void onPlaybackCompleted(@NonNull SessionPlayer player) { + public void onPlaybackCompleted(SessionPlayer player) { onPlaybackCompletedLatch.countDown(); } }); @@ -359,7 +344,7 @@ public void setPlaybackSpeed_afterPlayback_remainsSame() throws Exception { SessionPlayer.PlayerCallback callback = new SessionPlayer.PlayerCallback() { @Override - public void onPlaybackCompleted(@NonNull SessionPlayer player) { + public void onPlaybackCompleted(SessionPlayer player) { onPlaybackCompletedLatch.countDown(); } }; @@ -420,7 +405,7 @@ public void seekTo_skipsUnnecessarySeek() throws Exception { executor, new SessionPlayer.PlayerCallback() { @Override - public void onSeekCompleted(@NonNull SessionPlayer player, long position) { + public void onSeekCompleted(SessionPlayer player, long position) { // Do not assert here, because onSeekCompleted() can be called after the player is // closed. positionChanges.add(position); @@ -483,7 +468,7 @@ public void seekTo_byUnderlyingPlayer_notifiesOnSeekCompleted() throws Exception executor, new SessionPlayer.PlayerCallback() { @Override - public void onSeekCompleted(@NonNull SessionPlayer player, long position) { + public void onSeekCompleted(SessionPlayer player, long position) { // Do not assert here, because onSeekCompleted() can be called after the player is // closed. seekPosition.set(position); @@ -553,7 +538,7 @@ public void prepare_notifiesOnPlayerStateChanged() throws Throwable { SessionPlayer.PlayerCallback callback = new SessionPlayer.PlayerCallback() { @Override - public void onPlayerStateChanged(@NonNull SessionPlayer player, int state) { + public void onPlayerStateChanged(SessionPlayer player, int state) { if (state == SessionPlayer.PLAYER_STATE_PAUSED) { onPlayerStatePaused.countDown(); } @@ -577,7 +562,7 @@ public void prepare_notifiesBufferingCompletedOnce() throws Throwable { new SessionPlayer.PlayerCallback() { @Override public void onBufferingStateChanged( - @NonNull SessionPlayer player, MediaItem item, int buffState) { + SessionPlayer player, @Nullable MediaItem item, int buffState) { bufferingStateChanges.add(buffState); if (buffState == SessionPlayer.BUFFERING_STATE_COMPLETE) { onBufferingCompletedLatch.countDown(); @@ -613,7 +598,7 @@ public void seekTo_whenPrepared_notifiesOnSeekCompleted() throws Throwable { SessionPlayer.PlayerCallback callback = new SessionPlayer.PlayerCallback() { @Override - public void onSeekCompleted(@NonNull SessionPlayer player, long position) { + public void onSeekCompleted(SessionPlayer player, long position) { onSeekCompletedLatch.countDown(); } }; @@ -636,7 +621,7 @@ public void setPlaybackSpeed_whenPrepared_notifiesOnPlaybackSpeedChanged() throw SessionPlayer.PlayerCallback callback = new SessionPlayer.PlayerCallback() { @Override - public void onPlaybackSpeedChanged(@NonNull SessionPlayer player, float speed) { + public void onPlaybackSpeedChanged(SessionPlayer player, float speed) { assertThat(speed).isWithin(FLOAT_TOLERANCE).of(0.5f); onPlaybackSpeedChangedLatch.countDown(); } @@ -736,7 +721,6 @@ public void cancelReturnedFuture_withSeekTo_cancelsPendingCommand() throws Excep @SmallTest @SdkSuppress(minSdkVersion = Build.VERSION_CODES.KITKAT) public void setPlaylist_withNullPlaylist_throwsException() throws Exception { - List playlist = TestUtils.createPlaylist(10); try { sessionPlayerConnector.setPlaylist(null, null); assertWithMessage("null playlist shouldn't be allowed").fail(); @@ -783,7 +767,7 @@ public void setPlaylistAndRemoveAllPlaylistItem_playerStateBecomesIdle() throws PlayerCallbackForPlaylist callback = new PlayerCallbackForPlaylist(playlist, 2) { @Override - public void onPlayerStateChanged(@NonNull SessionPlayer player, int playerState) { + public void onPlayerStateChanged(SessionPlayer player, int playerState) { countDown(); } }; @@ -811,7 +795,7 @@ public void setPlaylist_calledOnlyOnce_notifiesPlaylistChangeOnlyOnce() throws E new SessionPlayer.PlayerCallback() { @Override public void onPlaylistChanged( - @NonNull SessionPlayer player, + SessionPlayer player, @Nullable List list, @Nullable MediaMetadata metadata) { assertThat(list).isEqualTo(playlist); @@ -830,7 +814,6 @@ public void onPlaylistChanged( @SdkSuppress(minSdkVersion = Build.VERSION_CODES.KITKAT) public void setPlaylist_byUnderlyingPlayerBeforePrepare_notifiesOnPlaylistChanged() throws Exception { - List playlistToSessionPlayer = TestUtils.createPlaylist(2); List playlistToExoPlayer = TestUtils.createPlaylist(4); DefaultMediaItemConverter converter = new DefaultMediaItemConverter(); List exoMediaItems = new ArrayList<>(); @@ -844,7 +827,7 @@ public void setPlaylist_byUnderlyingPlayerBeforePrepare_notifiesOnPlaylistChange new SessionPlayer.PlayerCallback() { @Override public void onPlaylistChanged( - @NonNull SessionPlayer player, + SessionPlayer player, @Nullable List list, @Nullable MediaMetadata metadata) { if (ObjectsCompat.equals(list, playlistToExoPlayer)) { @@ -876,7 +859,7 @@ public void setPlaylist_byUnderlyingPlayerAfterPrepare_notifiesOnPlaylistChanged new SessionPlayer.PlayerCallback() { @Override public void onPlaylistChanged( - @NonNull SessionPlayer player, + SessionPlayer player, @Nullable List list, @Nullable MediaMetadata metadata) { if (ObjectsCompat.equals(list, playlistToExoPlayer)) { @@ -908,7 +891,7 @@ public void addPlaylistItem_calledOnlyOnce_notifiesPlaylistChangeOnlyOnce() thro new SessionPlayer.PlayerCallback() { @Override public void onPlaylistChanged( - @NonNull SessionPlayer player, + SessionPlayer player, @Nullable List list, @Nullable MediaMetadata metadata) { assertThat(list).isEqualTo(playlist); @@ -936,7 +919,7 @@ public void removePlaylistItem_calledOnlyOnce_notifiesPlaylistChangeOnlyOnce() t new SessionPlayer.PlayerCallback() { @Override public void onPlaylistChanged( - @NonNull SessionPlayer player, + SessionPlayer player, @Nullable List list, @Nullable MediaMetadata metadata) { assertThat(list).isEqualTo(playlist); @@ -948,6 +931,40 @@ public void onPlaylistChanged( assertThat(onPlaylistChangedLatch.getCount()).isEqualTo(1); } + @Test + @LargeTest + @SdkSuppress(minSdkVersion = Build.VERSION_CODES.KITKAT) + public void movePlaylistItem_calledOnlyOnce_notifiesPlaylistChangeOnlyOnce() throws Exception { + List playlist = new ArrayList<>(); + playlist.add(TestUtils.createMediaItem(R.raw.video_1)); + playlist.add(TestUtils.createMediaItem(R.raw.video_2)); + playlist.add(TestUtils.createMediaItem(R.raw.video_3)); + assertPlayerResultSuccess(sessionPlayerConnector.setPlaylist(playlist, /* metadata= */ null)); + assertPlayerResultSuccess(sessionPlayerConnector.prepare()); + + CountDownLatch onPlaylistChangedLatch = new CountDownLatch(2); + int moveFromIndex = 0; + int moveToIndex = 2; + playlist.add(moveToIndex, playlist.remove(moveFromIndex)); + sessionPlayerConnector.registerPlayerCallback( + executor, + new SessionPlayer.PlayerCallback() { + @Override + public void onPlaylistChanged( + SessionPlayer player, + @Nullable List list, + @Nullable MediaMetadata metadata) { + assertThat(list).isEqualTo(playlist); + onPlaylistChangedLatch.countDown(); + } + }); + sessionPlayerConnector.movePlaylistItem(moveFromIndex, moveToIndex); + assertThat(onPlaylistChangedLatch.await(PLAYLIST_CHANGE_WAIT_TIME_MS, MILLISECONDS)).isFalse(); + assertThat(onPlaylistChangedLatch.getCount()).isEqualTo(1); + } + + // TODO(b/168860979): De-flake and re-enable. + @Ignore @Test @LargeTest @SdkSuppress(minSdkVersion = Build.VERSION_CODES.KITKAT) @@ -965,7 +982,7 @@ public void replacePlaylistItem_calledOnlyOnce_notifiesPlaylistChangeOnlyOnce() new SessionPlayer.PlayerCallback() { @Override public void onPlaylistChanged( - @NonNull SessionPlayer player, + SessionPlayer player, @Nullable List list, @Nullable MediaMetadata metadata) { assertThat(list).isEqualTo(playlist); @@ -1019,8 +1036,7 @@ public void play_withPlaylist_notifiesOnCurrentMediaItemChangedAndOnPlaybackComp int currentMediaItemChangedCount = 0; @Override - public void onCurrentMediaItemChanged( - @NonNull SessionPlayer player, @NonNull MediaItem item) { + public void onCurrentMediaItemChanged(SessionPlayer player, MediaItem item) { assertThat(item).isEqualTo(player.getCurrentMediaItem()); int expectedCurrentIndex = currentMediaItemChangedCount++; @@ -1029,7 +1045,7 @@ public void onCurrentMediaItemChanged( } @Override - public void onPlaybackCompleted(@NonNull SessionPlayer player) { + public void onPlaybackCompleted(SessionPlayer player) { onPlaybackCompletedLatch.countDown(); } }); @@ -1054,7 +1070,7 @@ public void play_byUnderlyingPlayer_notifiesOnPlayerStateChanges() throws Except executor, new SessionPlayer.PlayerCallback() { @Override - public void onPlayerStateChanged(@NonNull SessionPlayer player, int playerState) { + public void onPlayerStateChanged(SessionPlayer player, int playerState) { if (playerState == PLAYER_STATE_PLAYING) { onPlayingLatch.countDown(); } @@ -1093,7 +1109,7 @@ public void pause_byUnderlyingPlayer_notifiesOnPlayerStateChanges() throws Excep executor, new SessionPlayer.PlayerCallback() { @Override - public void onPlayerStateChanged(@NonNull SessionPlayer player, int playerState) { + public void onPlayerStateChanged(SessionPlayer player, int playerState) { if (playerState == PLAYER_STATE_PAUSED) { onPausedLatch.countDown(); } @@ -1119,7 +1135,7 @@ public void pause_byUnderlyingPlayerInListener_changesToPlayerStatePaused() thro executor, new SessionPlayer.PlayerCallback() { @Override - public void onPlayerStateChanged(@NonNull SessionPlayer player, int playerState) { + public void onPlayerStateChanged(SessionPlayer player, int playerState) { playerStateChanges.add(playerState); playerStateChangesLatch.countDown(); } @@ -1170,8 +1186,7 @@ public void skipToNextAndPrevious_calledInARow_notifiesOnCurrentMediaItemChanged SessionPlayer.PlayerCallback skipToNextTestCallback = new SessionPlayer.PlayerCallback() { @Override - public void onCurrentMediaItemChanged( - @NonNull SessionPlayer player, @NonNull MediaItem item) { + public void onCurrentMediaItemChanged(SessionPlayer player, MediaItem item) { super.onCurrentMediaItemChanged(player, item); assertThat(player.getCurrentMediaItemIndex()).isEqualTo(1); assertThat(item).isEqualTo(player.getCurrentMediaItem()); @@ -1189,8 +1204,7 @@ public void onCurrentMediaItemChanged( SessionPlayer.PlayerCallback skipToPreviousTestCallback = new SessionPlayer.PlayerCallback() { @Override - public void onCurrentMediaItemChanged( - @NonNull SessionPlayer player, @NonNull MediaItem item) { + public void onCurrentMediaItemChanged(SessionPlayer player, MediaItem item) { super.onCurrentMediaItemChanged(player, item); assertThat(player.getCurrentMediaItemIndex()).isEqualTo(0); assertThat(item).isEqualTo(player.getCurrentMediaItem()); @@ -1221,15 +1235,14 @@ public void setRepeatMode_withRepeatAll_continuesToPlayPlaylistWithoutBeingCompl PlayerCallbackForPlaylist callback = new PlayerCallbackForPlaylist(playlist, listSize + 2) { @Override - public void onCurrentMediaItemChanged( - @NonNull SessionPlayer player, @NonNull MediaItem item) { + public void onCurrentMediaItemChanged(SessionPlayer player, MediaItem item) { super.onCurrentMediaItemChanged(player, item); currentMediaItemChanges.add(item); countDown(); } @Override - public void onPlaybackCompleted(@NonNull SessionPlayer player) { + public void onPlaybackCompleted(SessionPlayer player) { assertWithMessage( "Playback shouldn't be completed, Actual changes were %s", currentMediaItemChanges) @@ -1305,7 +1318,7 @@ public void getPlaylist_returnsPlaylistInUnderlyingPlayer() { } private class PlayerCallbackForPlaylist extends SessionPlayer.PlayerCallback { - private List playlist; + private final List playlist; private CountDownLatch onCurrentMediaItemChangedLatch; PlayerCallbackForPlaylist(List playlist, int count) { @@ -1314,7 +1327,7 @@ private class PlayerCallbackForPlaylist extends SessionPlayer.PlayerCallback { } @Override - public void onCurrentMediaItemChanged(@NonNull SessionPlayer player, @NonNull MediaItem item) { + public void onCurrentMediaItemChanged(SessionPlayer player, MediaItem item) { int currentIndex = playlist.indexOf(item); assertThat(sessionPlayerConnector.getCurrentMediaItemIndex()).isEqualTo(currentIndex); onCurrentMediaItemChangedLatch.countDown(); diff --git a/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/PlayerCommandQueue.java b/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/PlayerCommandQueue.java index fc80c85856f..a1d4941f50e 100644 --- a/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/PlayerCommandQueue.java +++ b/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/PlayerCommandQueue.java @@ -97,6 +97,9 @@ /** Command code for {@link SessionPlayer#removePlaylistItem(int)} */ public static final int COMMAND_CODE_PLAYER_REMOVE_PLAYLIST_ITEM = 16; + /** Command code for {@link SessionPlayer#movePlaylistItem(int, int)} */ + public static final int COMMAND_CODE_PLAYER_MOVE_PLAYLIST_ITEM = 17; + /** List of session commands whose result would be set after the command is finished. */ @Documented @Retention(RetentionPolicy.SOURCE) @@ -119,6 +122,7 @@ COMMAND_CODE_PLAYER_SET_PLAYLIST, COMMAND_CODE_PLAYER_ADD_PLAYLIST_ITEM, COMMAND_CODE_PLAYER_REMOVE_PLAYLIST_ITEM, + COMMAND_CODE_PLAYER_MOVE_PLAYLIST_ITEM, }) public @interface CommandCode {} @@ -381,8 +385,24 @@ private static boolean isAsyncCommand(@CommandCode int commandCode) { case COMMAND_CODE_PLAYER_PAUSE: case COMMAND_CODE_PLAYER_PREPARE: return true; + case COMMAND_CODE_PLAYER_ADD_PLAYLIST_ITEM: + case COMMAND_CODE_PLAYER_MOVE_PLAYLIST_ITEM: + case COMMAND_CODE_PLAYER_REMOVE_PLAYLIST_ITEM: + case COMMAND_CODE_PLAYER_REPLACE_PLAYLIST_ITEM: + case COMMAND_CODE_PLAYER_SEEK_TO: + case COMMAND_CODE_PLAYER_SET_AUDIO_ATTRIBUTES: + case COMMAND_CODE_PLAYER_SET_MEDIA_ITEM: + case COMMAND_CODE_PLAYER_SET_PLAYLIST: + case COMMAND_CODE_PLAYER_SET_REPEAT_MODE: + case COMMAND_CODE_PLAYER_SET_SHUFFLE_MODE: + case COMMAND_CODE_PLAYER_SET_SPEED: + case COMMAND_CODE_PLAYER_SKIP_TO_NEXT_PLAYLIST_ITEM: + case COMMAND_CODE_PLAYER_SKIP_TO_PLAYLIST_ITEM: + case COMMAND_CODE_PLAYER_SKIP_TO_PREVIOUS_PLAYLIST_ITEM: + case COMMAND_CODE_PLAYER_UPDATE_LIST_METADATA: + default: + return false; } - return false; } private static final class AsyncPlayerCommandResult { diff --git a/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/PlayerWrapper.java b/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/PlayerWrapper.java index 21659637ab7..74d7bd110e6 100644 --- a/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/PlayerWrapper.java +++ b/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/PlayerWrapper.java @@ -223,6 +223,19 @@ public boolean replacePlaylistItem(int index, androidx.media2.common.MediaItem m return true; } + public boolean movePlaylistItem( + @IntRange(from = 0) int fromIndex, @IntRange(from = 0) int toIndex) { + int itemCount = player.getMediaItemCount(); + if (!(fromIndex < itemCount && toIndex < itemCount)) { + return false; + } + if (fromIndex == toIndex) { + return true; + } + player.moveMediaItem(fromIndex, toIndex); + return true; + } + public boolean skipToPreviousPlaylistItem() { Timeline timeline = player.getCurrentTimeline(); Assertions.checkState(!timeline.isEmpty()); diff --git a/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/SessionCallback.java b/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/SessionCallback.java index 1f60db947ed..4b161a73454 100644 --- a/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/SessionCallback.java +++ b/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/SessionCallback.java @@ -227,7 +227,8 @@ private SessionCommandGroup buildAllowedCommands( } build.addAllPredefinedCommands(SessionCommand.COMMAND_VERSION_1); - // TODO: Use removeCommand(int) when it's added [Internal: b/142848015]. + build.addCommand(new SessionCommand(SessionCommand.COMMAND_CODE_PLAYER_MOVE_PLAYLIST_ITEM)); + // TODO(internal b/142848015): Use removeCommand(int) when it's added. if (mediaItemProvider == null) { build.removeCommand(new SessionCommand(SessionCommand.COMMAND_CODE_PLAYER_SET_MEDIA_ITEM)); build.removeCommand(new SessionCommand(SessionCommand.COMMAND_CODE_PLAYER_SET_PLAYLIST)); @@ -348,7 +349,10 @@ public void onShuffleModeChanged(SessionPlayer player, int shuffleMode) { updateAllowedCommands(); } + // TODO(internal b/160846312): Remove warning suppression and mark item @Nullable once we depend + // on media2 1.2.0. @Override + @SuppressWarnings("nullness:override.param.invalid") public void onCurrentMediaItemChanged(SessionPlayer player, MediaItem item) { currentMediaItemBuffered = isBufferedState(player.getBufferingState()); updateAllowedCommands(); diff --git a/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/SessionPlayerConnector.java b/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/SessionPlayerConnector.java index 0415a5cf384..9a3dc09b071 100644 --- a/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/SessionPlayerConnector.java +++ b/extensions/media2/src/main/java/com/google/android/exoplayer2/ext/media2/SessionPlayerConnector.java @@ -324,6 +324,17 @@ public ListenableFuture replacePlaylistItem(int index, MediaItem i return result; } + @Override + public ListenableFuture movePlaylistItem(int fromIndex, int toIndex) { + Assertions.checkArgument(fromIndex >= 0); + Assertions.checkArgument(toIndex >= 0); + ListenableFuture result = + playerCommandQueue.addCommand( + PlayerCommandQueue.COMMAND_CODE_PLAYER_MOVE_PLAYLIST_ITEM, + /* command= */ () -> player.movePlaylistItem(fromIndex, toIndex)); + return result; + } + @Override public ListenableFuture skipToPreviousPlaylistItem() { ListenableFuture result = diff --git a/extensions/mediasession/src/main/java/com/google/android/exoplayer2/ext/mediasession/MediaSessionConnector.java b/extensions/mediasession/src/main/java/com/google/android/exoplayer2/ext/mediasession/MediaSessionConnector.java index e78c55b2afb..179a8a3f11a 100644 --- a/extensions/mediasession/src/main/java/com/google/android/exoplayer2/ext/mediasession/MediaSessionConnector.java +++ b/extensions/mediasession/src/main/java/com/google/android/exoplayer2/ext/mediasession/MediaSessionConnector.java @@ -15,6 +15,13 @@ */ package com.google.android.exoplayer2.ext.mediasession; +import static com.google.android.exoplayer2.Player.EVENT_IS_PLAYING_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAYBACK_PARAMETERS_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAYBACK_STATE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAY_WHEN_READY_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_REPEAT_MODE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED; + import android.content.Intent; import android.graphics.Bitmap; import android.net.Uri; @@ -38,7 +45,6 @@ import com.google.android.exoplayer2.DefaultControlDispatcher; import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.ExoPlayerLibraryInfo; -import com.google.android.exoplayer2.PlaybackParameters; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.util.Assertions; @@ -95,6 +101,10 @@ public final class MediaSessionConnector { ExoPlayerLibraryInfo.registerModule("goog.exo.mediasession"); } + /** Indicates this session supports the set playback speed command. */ + // TODO(b/174297519) Replace with PlaybackStateCompat.ACTION_SET_PLAYBACK_SPEED when released. + public static final long ACTION_SET_PLAYBACK_SPEED = 1 << 22; + /** Playback actions supported by the connector. */ @LongDef( flag = true, @@ -107,7 +117,8 @@ public final class MediaSessionConnector { PlaybackStateCompat.ACTION_REWIND, PlaybackStateCompat.ACTION_STOP, PlaybackStateCompat.ACTION_SET_REPEAT_MODE, - PlaybackStateCompat.ACTION_SET_SHUFFLE_MODE + PlaybackStateCompat.ACTION_SET_SHUFFLE_MODE, + ACTION_SET_PLAYBACK_SPEED }) @Retention(RetentionPolicy.SOURCE) public @interface PlaybackActions {} @@ -122,10 +133,13 @@ public final class MediaSessionConnector { | PlaybackStateCompat.ACTION_REWIND | PlaybackStateCompat.ACTION_STOP | PlaybackStateCompat.ACTION_SET_REPEAT_MODE - | PlaybackStateCompat.ACTION_SET_SHUFFLE_MODE; + | PlaybackStateCompat.ACTION_SET_SHUFFLE_MODE + | ACTION_SET_PLAYBACK_SPEED; /** The default playback actions. */ - @PlaybackActions public static final long DEFAULT_PLAYBACK_ACTIONS = ALL_PLAYBACK_ACTIONS; + @PlaybackActions + public static final long DEFAULT_PLAYBACK_ACTIONS = + ALL_PLAYBACK_ACTIONS - ACTION_SET_PLAYBACK_SPEED; /** * The name of the {@link PlaybackStateCompat} float extra with the value of {@code @@ -139,7 +153,8 @@ public final class MediaSessionConnector { | PlaybackStateCompat.ACTION_PAUSE | PlaybackStateCompat.ACTION_STOP | PlaybackStateCompat.ACTION_SET_SHUFFLE_MODE - | PlaybackStateCompat.ACTION_SET_REPEAT_MODE; + | PlaybackStateCompat.ACTION_SET_REPEAT_MODE + | ACTION_SET_PLAYBACK_SPEED; private static final int BASE_MEDIA_SESSION_FLAGS = MediaSessionCompat.FLAG_HANDLES_MEDIA_BUTTONS | MediaSessionCompat.FLAG_HANDLES_TRANSPORT_CONTROLS; @@ -1074,69 +1089,59 @@ private class ComponentListener extends MediaSessionCompat.Callback // Player.EventListener implementation. @Override - public void onTimelineChanged(Timeline timeline, @Player.TimelineChangeReason int reason) { - Player player = Assertions.checkNotNull(MediaSessionConnector.this.player); - int windowCount = player.getCurrentTimeline().getWindowCount(); - int windowIndex = player.getCurrentWindowIndex(); - if (queueNavigator != null) { - queueNavigator.onTimelineChanged(player); - invalidateMediaSessionPlaybackState(); - } else if (currentWindowCount != windowCount || currentWindowIndex != windowIndex) { - // active queue item and queue navigation actions may need to be updated - invalidateMediaSessionPlaybackState(); + public void onEvents(Player player, Player.Events events) { + boolean invalidatePlaybackState = false; + boolean invalidateMetadata = false; + if (events.contains(Player.EVENT_POSITION_DISCONTINUITY)) { + if (currentWindowIndex != player.getCurrentWindowIndex()) { + if (queueNavigator != null) { + queueNavigator.onCurrentWindowIndexChanged(player); + } + invalidateMetadata = true; + } + invalidatePlaybackState = true; } - currentWindowCount = windowCount; - currentWindowIndex = windowIndex; - invalidateMediaSessionMetadata(); - } - @Override - public void onPlaybackStateChanged(@Player.State int playbackState) { - invalidateMediaSessionPlaybackState(); - } - - @Override - public void onPlayWhenReadyChanged( - boolean playWhenReady, @Player.PlayWhenReadyChangeReason int reason) { - invalidateMediaSessionPlaybackState(); - } - - @Override - public void onIsPlayingChanged(boolean isPlaying) { - invalidateMediaSessionPlaybackState(); - } + if (events.contains(Player.EVENT_TIMELINE_CHANGED)) { + int windowCount = player.getCurrentTimeline().getWindowCount(); + int windowIndex = player.getCurrentWindowIndex(); + if (queueNavigator != null) { + queueNavigator.onTimelineChanged(player); + invalidatePlaybackState = true; + } else if (currentWindowCount != windowCount || currentWindowIndex != windowIndex) { + // active queue item and queue navigation actions may need to be updated + invalidatePlaybackState = true; + } + currentWindowCount = windowCount; + invalidateMetadata = true; + } - @Override - public void onRepeatModeChanged(@Player.RepeatMode int repeatMode) { - invalidateMediaSessionPlaybackState(); - } + // Update currentWindowIndex after comparisons above. + currentWindowIndex = player.getCurrentWindowIndex(); - @Override - public void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) { - invalidateMediaSessionPlaybackState(); - invalidateMediaSessionQueue(); - } + if (events.containsAny( + EVENT_PLAYBACK_STATE_CHANGED, + EVENT_PLAY_WHEN_READY_CHANGED, + EVENT_IS_PLAYING_CHANGED, + EVENT_REPEAT_MODE_CHANGED, + EVENT_PLAYBACK_PARAMETERS_CHANGED)) { + invalidatePlaybackState = true; + } - @Override - public void onPositionDiscontinuity(@Player.DiscontinuityReason int reason) { - Player player = Assertions.checkNotNull(MediaSessionConnector.this.player); - if (currentWindowIndex != player.getCurrentWindowIndex()) { - if (queueNavigator != null) { - queueNavigator.onCurrentWindowIndexChanged(player); - } - currentWindowIndex = player.getCurrentWindowIndex(); - // Update playback state after queueNavigator.onCurrentWindowIndexChanged has been called - // and before updating metadata. + // The queue needs to be updated by the queue navigator first. The queue navigator also + // delivers the active queue item that is used to update the playback state. + if (events.containsAny(EVENT_SHUFFLE_MODE_ENABLED_CHANGED)) { + invalidateMediaSessionQueue(); + invalidatePlaybackState = true; + } + // Invalidate the playback state before invalidating metadata because the active queue item of + // the session playback state needs to be updated before the MediaMetadataProvider uses it. + if (invalidatePlaybackState) { invalidateMediaSessionPlaybackState(); + } + if (invalidateMetadata) { invalidateMediaSessionMetadata(); - return; } - invalidateMediaSessionPlaybackState(); - } - - @Override - public void onPlaybackParametersChanged(PlaybackParameters playbackParameters) { - invalidateMediaSessionPlaybackState(); } // MediaSessionCompat.Callback implementation. @@ -1234,6 +1239,14 @@ public void onSetRepeatMode(@PlaybackStateCompat.RepeatMode int mediaSessionRepe } } + @Override + public void onSetPlaybackSpeed(float speed) { + if (canDispatchPlaybackAction(ACTION_SET_PLAYBACK_SPEED) && speed > 0) { + controlDispatcher.dispatchSetPlaybackParameters( + player, player.getPlaybackParameters().withSpeed(speed)); + } + } + @Override public void onSkipToNext() { if (canDispatchToQueueNavigator(PlaybackStateCompat.ACTION_SKIP_TO_NEXT)) { diff --git a/extensions/okhttp/build.gradle b/extensions/okhttp/build.gradle index 032fb0fded2..758eb646f61 100644 --- a/extensions/okhttp/build.gradle +++ b/extensions/okhttp/build.gradle @@ -14,7 +14,7 @@ apply from: "$gradle.ext.exoplayerSettingsDir/common_library_config.gradle" dependencies { - implementation project(modulePrefix + 'library-core') + implementation project(modulePrefix + 'library-common') implementation 'androidx.annotation:annotation:' + androidxAnnotationVersion compileOnly 'org.checkerframework:checker-qual:' + checkerframeworkVersion compileOnly 'org.jetbrains.kotlin:kotlin-annotations-jvm:' + kotlinAnnotationsVersion diff --git a/extensions/okhttp/src/main/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSource.java b/extensions/okhttp/src/main/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSource.java index 85d3530e2d9..d23dd22574b 100644 --- a/extensions/okhttp/src/main/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSource.java +++ b/extensions/okhttp/src/main/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSource.java @@ -23,9 +23,11 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.upstream.BaseDataSource; +import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DataSourceException; import com.google.android.exoplayer2.upstream.DataSpec; import com.google.android.exoplayer2.upstream.HttpDataSource; +import com.google.android.exoplayer2.upstream.TransferListener; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Util; import com.google.common.base.Predicate; @@ -41,6 +43,7 @@ import okhttp3.Call; import okhttp3.HttpUrl; import okhttp3.MediaType; +import okhttp3.OkHttpClient; import okhttp3.Request; import okhttp3.RequestBody; import okhttp3.Response; @@ -59,6 +62,112 @@ public class OkHttpDataSource extends BaseDataSource implements HttpDataSource { ExoPlayerLibraryInfo.registerModule("goog.exo.okhttp"); } + /** {@link DataSource.Factory} for {@link OkHttpDataSource} instances. */ + public static final class Factory implements HttpDataSource.Factory { + + private final RequestProperties defaultRequestProperties; + private final Call.Factory callFactory; + + @Nullable private String userAgent; + @Nullable private TransferListener transferListener; + @Nullable private CacheControl cacheControl; + @Nullable private Predicate contentTypePredicate; + + /** + * Creates an instance. + * + * @param callFactory A {@link Call.Factory} (typically an {@link OkHttpClient}) for use by the + * sources created by the factory. + */ + public Factory(Call.Factory callFactory) { + this.callFactory = callFactory; + defaultRequestProperties = new RequestProperties(); + } + + /** @deprecated Use {@link #setDefaultRequestProperties(Map)} instead. */ + @Deprecated + @Override + public final RequestProperties getDefaultRequestProperties() { + return defaultRequestProperties; + } + + @Override + public final Factory setDefaultRequestProperties(Map defaultRequestProperties) { + this.defaultRequestProperties.clearAndSet(defaultRequestProperties); + return this; + } + + /** + * Sets the user agent that will be used. + * + *

The default is {@code null}, which causes the default user agent of the underlying {@link + * OkHttpClient} to be used. + * + * @param userAgent The user agent that will be used, or {@code null} to use the default user + * agent of the underlying {@link OkHttpClient}. + * @return This factory. + */ + public Factory setUserAgent(@Nullable String userAgent) { + this.userAgent = userAgent; + return this; + } + + /** + * Sets the {@link CacheControl} that will be used. + * + *

The default is {@code null}. + * + * @param cacheControl The cache control that will be used. + * @return This factory. + */ + public Factory setCacheControl(@Nullable CacheControl cacheControl) { + this.cacheControl = cacheControl; + return this; + } + + /** + * Sets a content type {@link Predicate}. If a content type is rejected by the predicate then a + * {@link HttpDataSource.InvalidContentTypeException} is thrown from {@link + * OkHttpDataSource#open(DataSpec)}. + * + *

The default is {@code null}. + * + * @param contentTypePredicate The content type {@link Predicate}, or {@code null} to clear a + * predicate that was previously set. + * @return This factory. + */ + public Factory setContentTypePredicate(@Nullable Predicate contentTypePredicate) { + this.contentTypePredicate = contentTypePredicate; + return this; + } + + /** + * Sets the {@link TransferListener} that will be used. + * + *

The default is {@code null}. + * + *

See {@link DataSource#addTransferListener(TransferListener)}. + * + * @param transferListener The listener that will be used. + * @return This factory. + */ + public Factory setTransferListener(@Nullable TransferListener transferListener) { + this.transferListener = transferListener; + return this; + } + + @Override + public OkHttpDataSource createDataSource() { + OkHttpDataSource dataSource = + new OkHttpDataSource( + callFactory, userAgent, cacheControl, defaultRequestProperties, contentTypePredicate); + if (transferListener != null) { + dataSource.addTransferListener(transferListener); + } + return dataSource; + } + } + private static final byte[] SKIP_BUFFER = new byte[4096]; private final Call.Factory callFactory; @@ -80,114 +189,54 @@ public class OkHttpDataSource extends BaseDataSource implements HttpDataSource { private long bytesSkipped; private long bytesRead; - /** - * Creates an instance. - * - * @param callFactory A {@link Call.Factory} (typically an {@link okhttp3.OkHttpClient}) for use - * by the source. - */ + /** @deprecated Use {@link OkHttpDataSource.Factory} instead. */ + @SuppressWarnings("deprecation") + @Deprecated public OkHttpDataSource(Call.Factory callFactory) { - this(callFactory, ExoPlayerLibraryInfo.DEFAULT_USER_AGENT); + this(callFactory, /* userAgent= */ null); } - /** - * Creates an instance. - * - * @param callFactory A {@link Call.Factory} (typically an {@link okhttp3.OkHttpClient}) for use - * by the source. - * @param userAgent An optional User-Agent string. - */ + /** @deprecated Use {@link OkHttpDataSource.Factory} instead. */ + @SuppressWarnings("deprecation") + @Deprecated public OkHttpDataSource(Call.Factory callFactory, @Nullable String userAgent) { this(callFactory, userAgent, /* cacheControl= */ null, /* defaultRequestProperties= */ null); } - /** - * Creates an instance. - * - * @param callFactory A {@link Call.Factory} (typically an {@link okhttp3.OkHttpClient}) for use - * by the source. - * @param userAgent An optional User-Agent string. - * @param cacheControl An optional {@link CacheControl} for setting the Cache-Control header. - * @param defaultRequestProperties Optional default {@link RequestProperties} to be sent to the - * server as HTTP headers on every request. - */ + /** @deprecated Use {@link OkHttpDataSource.Factory} instead. */ + @Deprecated public OkHttpDataSource( Call.Factory callFactory, @Nullable String userAgent, @Nullable CacheControl cacheControl, @Nullable RequestProperties defaultRequestProperties) { - super(/* isNetwork= */ true); - this.callFactory = Assertions.checkNotNull(callFactory); - this.userAgent = userAgent; - this.cacheControl = cacheControl; - this.defaultRequestProperties = defaultRequestProperties; - this.requestProperties = new RequestProperties(); - } - - /** - * Creates an instance. - * - * @param callFactory A {@link Call.Factory} (typically an {@link okhttp3.OkHttpClient}) for use - * by the source. - * @param userAgent An optional User-Agent string. - * @param contentTypePredicate An optional {@link Predicate}. If a content type is rejected by the - * predicate then a {@link InvalidContentTypeException} is thrown from {@link - * #open(DataSpec)}. - * @deprecated Use {@link #OkHttpDataSource(Call.Factory, String)} and {@link - * #setContentTypePredicate(Predicate)}. - */ - @SuppressWarnings("deprecation") - @Deprecated - public OkHttpDataSource( - Call.Factory callFactory, - @Nullable String userAgent, - @Nullable Predicate contentTypePredicate) { this( callFactory, userAgent, - contentTypePredicate, - /* cacheControl= */ null, - /* defaultRequestProperties= */ null); + cacheControl, + defaultRequestProperties, + /* contentTypePredicate= */ null); } - /** - * Creates an instance. - * - * @param callFactory A {@link Call.Factory} (typically an {@link okhttp3.OkHttpClient}) for use - * by the source. - * @param userAgent An optional User-Agent string. - * @param contentTypePredicate An optional {@link Predicate}. If a content type is rejected by the - * predicate then a {@link InvalidContentTypeException} is thrown from {@link - * #open(DataSpec)}. - * @param cacheControl An optional {@link CacheControl} for setting the Cache-Control header. - * @param defaultRequestProperties Optional default {@link RequestProperties} to be sent to the - * server as HTTP headers on every request. - * @deprecated Use {@link #OkHttpDataSource(Call.Factory, String, CacheControl, - * RequestProperties)} and {@link #setContentTypePredicate(Predicate)}. - */ - @Deprecated - public OkHttpDataSource( + private OkHttpDataSource( Call.Factory callFactory, @Nullable String userAgent, - @Nullable Predicate contentTypePredicate, @Nullable CacheControl cacheControl, - @Nullable RequestProperties defaultRequestProperties) { + @Nullable RequestProperties defaultRequestProperties, + @Nullable Predicate contentTypePredicate) { super(/* isNetwork= */ true); this.callFactory = Assertions.checkNotNull(callFactory); this.userAgent = userAgent; - this.contentTypePredicate = contentTypePredicate; this.cacheControl = cacheControl; this.defaultRequestProperties = defaultRequestProperties; + this.contentTypePredicate = contentTypePredicate; this.requestProperties = new RequestProperties(); } /** - * Sets a content type {@link Predicate}. If a content type is rejected by the predicate then a - * {@link HttpDataSource.InvalidContentTypeException} is thrown from {@link #open(DataSpec)}. - * - * @param contentTypePredicate The content type {@link Predicate}, or {@code null} to clear a - * predicate that was previously set. + * @deprecated Use {@link OkHttpDataSource.Factory#setContentTypePredicate(Predicate)} instead. */ + @Deprecated public void setContentTypePredicate(@Nullable Predicate contentTypePredicate) { this.contentTypePredicate = contentTypePredicate; } @@ -259,8 +308,7 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { try { errorResponseBody = Util.toByteArray(Assertions.checkNotNull(responseByteStream)); } catch (IOException e) { - throw new HttpDataSourceException( - "Error reading non-2xx response body", e, dataSpec, HttpDataSourceException.TYPE_OPEN); + errorResponseBody = Util.EMPTY_BYTE_ARRAY; } Map> headers = response.headers().toMultimap(); closeConnectionQuietly(); @@ -274,7 +322,7 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { } // Check for a valid content type. - MediaType mediaType = responseBody.contentType(); + @Nullable MediaType mediaType = responseBody.contentType(); String contentType = mediaType != null ? mediaType.toString() : ""; if (contentTypePredicate != null && !contentTypePredicate.apply(contentType)) { closeConnectionQuietly(); @@ -357,7 +405,7 @@ private Request makeRequest(DataSpec dataSpec) throws HttpDataSourceException { long position = dataSpec.position; long length = dataSpec.length; - HttpUrl url = HttpUrl.parse(dataSpec.uri.toString()); + @Nullable HttpUrl url = HttpUrl.parse(dataSpec.uri.toString()); if (url == null) { throw new HttpDataSourceException( "Malformed URL", dataSpec, HttpDataSourceException.TYPE_OPEN); @@ -394,7 +442,7 @@ private Request makeRequest(DataSpec dataSpec) throws HttpDataSourceException { builder.addHeader("Accept-Encoding", "identity"); } - RequestBody requestBody = null; + @Nullable RequestBody requestBody = null; if (dataSpec.httpBody != null) { requestBody = RequestBody.create(null, dataSpec.httpBody); } else if (dataSpec.httpMethod == DataSpec.HTTP_METHOD_POST) { diff --git a/extensions/okhttp/src/main/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceFactory.java b/extensions/okhttp/src/main/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceFactory.java index 728428c8118..08e337f52bb 100644 --- a/extensions/okhttp/src/main/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceFactory.java +++ b/extensions/okhttp/src/main/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceFactory.java @@ -15,19 +15,16 @@ */ package com.google.android.exoplayer2.ext.okhttp; -import static com.google.android.exoplayer2.ExoPlayerLibraryInfo.DEFAULT_USER_AGENT; import androidx.annotation.Nullable; import com.google.android.exoplayer2.upstream.HttpDataSource; import com.google.android.exoplayer2.upstream.HttpDataSource.BaseFactory; -import com.google.android.exoplayer2.upstream.HttpDataSource.Factory; import com.google.android.exoplayer2.upstream.TransferListener; import okhttp3.CacheControl; import okhttp3.Call; -/** - * A {@link Factory} that produces {@link OkHttpDataSource}. - */ +/** @deprecated Use {@link OkHttpDataSource.Factory} instead. */ +@Deprecated public final class OkHttpDataSourceFactory extends BaseFactory { private final Call.Factory callFactory; @@ -42,7 +39,7 @@ public final class OkHttpDataSourceFactory extends BaseFactory { * by the sources created by the factory. */ public OkHttpDataSourceFactory(Call.Factory callFactory) { - this(callFactory, DEFAULT_USER_AGENT, /* listener= */ null, /* cacheControl= */ null); + this(callFactory, /* userAgent= */ null, /* listener= */ null, /* cacheControl= */ null); } /** @@ -102,6 +99,8 @@ public OkHttpDataSourceFactory( this.cacheControl = cacheControl; } + // Calls deprecated constructor. + @SuppressWarnings("deprecation") @Override protected OkHttpDataSource createDataSourceInternal( HttpDataSource.RequestProperties defaultRequestProperties) { diff --git a/extensions/okhttp/src/test/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceContractTest.java b/extensions/okhttp/src/test/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceContractTest.java new file mode 100644 index 00000000000..8bf572d70d5 --- /dev/null +++ b/extensions/okhttp/src/test/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceContractTest.java @@ -0,0 +1,48 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.ext.okhttp; + +import android.net.Uri; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.HttpDataSourceTestEnv; +import com.google.android.exoplayer2.upstream.DataSource; +import com.google.common.collect.ImmutableList; +import okhttp3.OkHttpClient; +import org.junit.Rule; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link OkHttpDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class OkHttpDataSourceContractTest extends DataSourceContractTest { + + @Rule public HttpDataSourceTestEnv httpDataSourceTestEnv = new HttpDataSourceTestEnv(); + + @Override + protected DataSource createDataSource() { + return new OkHttpDataSource.Factory(new OkHttpClient()).createDataSource(); + } + + @Override + protected ImmutableList getTestResources() { + return httpDataSourceTestEnv.getServedResources(); + } + + @Override + protected Uri getNotFoundUri() { + return Uri.parse(httpDataSourceTestEnv.getNonexistentUrl()); + } +} diff --git a/extensions/okhttp/src/test/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceTest.java b/extensions/okhttp/src/test/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceTest.java index 73e9909a8db..889d99cc2e6 100644 --- a/extensions/okhttp/src/test/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceTest.java +++ b/extensions/okhttp/src/test/java/com/google/android/exoplayer2/ext/okhttp/OkHttpDataSourceTest.java @@ -59,15 +59,16 @@ public void open_setsCorrectHeaders() throws Exception { MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); - String propertyFromConstructor = "fromConstructor"; - HttpDataSource.RequestProperties constructorProperties = new HttpDataSource.RequestProperties(); - constructorProperties.set("0", propertyFromConstructor); - constructorProperties.set("1", propertyFromConstructor); - constructorProperties.set("2", propertyFromConstructor); - constructorProperties.set("4", propertyFromConstructor); - OkHttpDataSource dataSource = - new OkHttpDataSource( - new OkHttpClient(), "testAgent", /* cacheControl= */ null, constructorProperties); + String propertyFromFactory = "fromFactory"; + Map defaultRequestProperties = new HashMap<>(); + defaultRequestProperties.put("0", propertyFromFactory); + defaultRequestProperties.put("1", propertyFromFactory); + defaultRequestProperties.put("2", propertyFromFactory); + defaultRequestProperties.put("4", propertyFromFactory); + HttpDataSource dataSource = + new OkHttpDataSource.Factory(new OkHttpClient()) + .setDefaultRequestProperties(defaultRequestProperties) + .createDataSource(); String propertyFromSetter = "fromSetter"; dataSource.setRequestProperty("1", propertyFromSetter); @@ -91,7 +92,7 @@ public void open_setsCorrectHeaders() throws Exception { dataSource.open(dataSpec); Headers headers = mockWebServer.takeRequest(10, SECONDS).getHeaders(); - assertThat(headers.get("0")).isEqualTo(propertyFromConstructor); + assertThat(headers.get("0")).isEqualTo(propertyFromFactory); assertThat(headers.get("1")).isEqualTo(propertyFromSetter); assertThat(headers.get("2")).isEqualTo(propertyFromDataSpec); assertThat(headers.get("3")).isEqualTo(propertyFromDataSpec); @@ -101,16 +102,13 @@ public void open_setsCorrectHeaders() throws Exception { } @Test - public void open_invalidResponseCode() throws Exception { + public void open_invalidResponseCode() { MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(404).setBody("failure msg")); - OkHttpDataSource okHttpDataSource = - new OkHttpDataSource( - new OkHttpClient(), - "testAgent", - /* cacheControl= */ null, - /* defaultRequestProperties= */ null); + HttpDataSource okHttpDataSource = + new OkHttpDataSource.Factory(new OkHttpClient()).createDataSource(); + DataSpec dataSpec = new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build(); @@ -122,4 +120,22 @@ public void open_invalidResponseCode() throws Exception { assertThat(exception.responseCode).isEqualTo(404); assertThat(exception.responseBody).isEqualTo("failure msg".getBytes(Charsets.UTF_8)); } + + @Test + public void factory_setRequestPropertyAfterCreation_setsCorrectHeaders() throws Exception { + MockWebServer mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse()); + DataSpec dataSpec = + new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build(); + OkHttpDataSource.Factory factory = new OkHttpDataSource.Factory(new OkHttpClient()); + OkHttpDataSource dataSource = factory.createDataSource(); + + Map defaultRequestProperties = new HashMap<>(); + defaultRequestProperties.put("0", "afterCreation"); + factory.setDefaultRequestProperties(defaultRequestProperties); + dataSource.open(dataSpec); + + Headers headers = mockWebServer.takeRequest(10, SECONDS).getHeaders(); + assertThat(headers.get("0")).isEqualTo("afterCreation"); + } } diff --git a/extensions/opus/README.md b/extensions/opus/README.md index d3691b07bdf..b683dae0bf8 100644 --- a/extensions/opus/README.md +++ b/extensions/opus/README.md @@ -39,7 +39,7 @@ NDK_PATH="" ``` cd "${OPUS_EXT_PATH}/jni" && \ -git clone https://git.xiph.org/opus.git libopus +git clone https://gitlab.xiph.org/xiph/opus.git libopus ``` * Run the script to convert arm assembly to NDK compatible format: diff --git a/extensions/opus/src/main/java/com/google/android/exoplayer2/ext/opus/LibopusAudioRenderer.java b/extensions/opus/src/main/java/com/google/android/exoplayer2/ext/opus/LibopusAudioRenderer.java index 603241486c2..55100198784 100644 --- a/extensions/opus/src/main/java/com/google/android/exoplayer2/ext/opus/LibopusAudioRenderer.java +++ b/extensions/opus/src/main/java/com/google/android/exoplayer2/ext/opus/LibopusAudioRenderer.java @@ -79,21 +79,21 @@ public String getName() { } @Override - @FormatSupport + @C.FormatSupport protected int supportsFormatInternal(Format format) { boolean drmIsSupported = format.exoMediaCryptoType == null || OpusLibrary.matchesExpectedExoMediaCryptoType(format.exoMediaCryptoType); if (!OpusLibrary.isAvailable() || !MimeTypes.AUDIO_OPUS.equalsIgnoreCase(format.sampleMimeType)) { - return FORMAT_UNSUPPORTED_TYPE; + return C.FORMAT_UNSUPPORTED_TYPE; } else if (!sinkSupportsFormat( Util.getPcmFormat(C.ENCODING_PCM_16BIT, format.channelCount, format.sampleRate))) { - return FORMAT_UNSUPPORTED_SUBTYPE; + return C.FORMAT_UNSUPPORTED_SUBTYPE; } else if (!drmIsSupported) { - return FORMAT_UNSUPPORTED_DRM; + return C.FORMAT_UNSUPPORTED_DRM; } else { - return FORMAT_HANDLED; + return C.FORMAT_HANDLED; } } diff --git a/extensions/opus/src/main/java/com/google/android/exoplayer2/ext/opus/OpusDecoder.java b/extensions/opus/src/main/java/com/google/android/exoplayer2/ext/opus/OpusDecoder.java index 6b96cc5e49f..f6da031a7af 100644 --- a/extensions/opus/src/main/java/com/google/android/exoplayer2/ext/opus/OpusDecoder.java +++ b/extensions/opus/src/main/java/com/google/android/exoplayer2/ext/opus/OpusDecoder.java @@ -15,7 +15,10 @@ */ package com.google.android.exoplayer2.ext.opus; +import static androidx.annotation.VisibleForTesting.PACKAGE_PRIVATE; + import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.audio.OpusUtil; import com.google.android.exoplayer2.decoder.CryptoInfo; @@ -30,7 +33,8 @@ import java.util.List; /** Opus decoder. */ -/* package */ final class OpusDecoder +@VisibleForTesting(otherwise = PACKAGE_PRIVATE) +public final class OpusDecoder extends SimpleDecoder { private static final int NO_ERROR = 0; diff --git a/extensions/opus/src/main/jni/convert_android_asm.sh b/extensions/opus/src/main/jni/convert_android_asm.sh index e8aa255842b..9c79738439b 100755 --- a/extensions/opus/src/main/jni/convert_android_asm.sh +++ b/extensions/opus/src/main/jni/convert_android_asm.sh @@ -34,7 +34,7 @@ while read file; do perl -pi -e "s/-gnu\.S/_gnu\.s/g" "${gnu_file}" rm -f "${file}" fi -done < <(find . -iname '*.s') +done < <(find -L . -iname '*.s') # Generate armopts.s from armopts.s.in sed \ diff --git a/extensions/rtmp/build.gradle b/extensions/rtmp/build.gradle index 3d912bebf69..7a373965688 100644 --- a/extensions/rtmp/build.gradle +++ b/extensions/rtmp/build.gradle @@ -14,10 +14,11 @@ apply from: "$gradle.ext.exoplayerSettingsDir/common_library_config.gradle" dependencies { - implementation project(modulePrefix + 'library-core') + implementation project(modulePrefix + 'library-common') implementation 'net.butterflytv.utils:rtmp-client:3.1.0' implementation 'androidx.annotation:annotation:' + androidxAnnotationVersion compileOnly 'org.jetbrains.kotlin:kotlin-annotations-jvm:' + kotlinAnnotationsVersion + testImplementation project(modulePrefix + 'library-core') testImplementation project(modulePrefix + 'testutils') testImplementation 'org.robolectric:robolectric:' + robolectricVersion } diff --git a/extensions/vp9/README.md b/extensions/vp9/README.md index 765cdbca3bd..1c1d91eb035 100644 --- a/extensions/vp9/README.md +++ b/extensions/vp9/README.md @@ -35,25 +35,23 @@ VP9_EXT_PATH="${EXOPLAYER_ROOT}/extensions/vp9/src/main" NDK_PATH="" ``` -* Fetch libvpx: - -``` -cd "${VP9_EXT_PATH}/jni" && \ -git clone https://chromium.googlesource.com/webm/libvpx libvpx -``` - -* Checkout an appropriate branch of libvpx. We cannot guarantee compatibility +* Fetch an appropriate branch of libvpx. We cannot guarantee compatibility with all versions of libvpx. We currently recommend version 1.8.0: ``` -cd "${VP9_EXT_PATH}/jni/libvpx" && \ -git checkout tags/v1.8.0 -b v1.8.0 +cd "" && \ +git clone https://chromium.googlesource.com/webm/libvpx && \ +cd libvpx && \ +git checkout tags/v1.8.0 -b v1.8.0 && \ +LIBVPX_PATH="$(pwd)" ``` -* Run a script that generates necessary configuration files for libvpx: +* Add a link to the libvpx source code in the vp9 extension `jni` directory and + run a script that generates necessary configuration files for libvpx: ``` cd ${VP9_EXT_PATH}/jni && \ +ln -s "$LIBVPX_PATH" libvpx && \ ./generate_libvpx_android_configs.sh ``` @@ -80,8 +78,8 @@ should be possible to follow the Linux instructions in [Windows PowerShell][]. * Android config scripts should be re-generated by running `generate_libvpx_android_configs.sh` * Clean and re-build the project. -* If you want to use your own version of libvpx, place it in - `${VP9_EXT_PATH}/jni/libvpx`. Please note that +* If you want to use your own version of libvpx, point to it with the + `${VP9_EXT_PATH}/jni/libvpx` symlink. Please note that `generate_libvpx_android_configs.sh` and the makefiles may need to be modified to work with arbitrary versions of libvpx. diff --git a/extensions/vp9/proguard-rules.txt b/extensions/vp9/proguard-rules.txt index d64773da2f8..6e473eb21e1 100644 --- a/extensions/vp9/proguard-rules.txt +++ b/extensions/vp9/proguard-rules.txt @@ -6,6 +6,12 @@ } # Some members of this class are being accessed from native methods. Keep them unobfuscated. +-keep class com.google.android.exoplayer2.video.VideoDecoderOutputBuffer { + *; +} + +# The deprecated VpxOutputBuffer might be used by old binary versions. Remove +# once VpxOutputBuffer is removed. -keep class com.google.android.exoplayer2.ext.vp9.VpxOutputBuffer { *; } diff --git a/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/LibvpxVideoRenderer.java b/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/LibvpxVideoRenderer.java index 61ebc8b0d9d..610f2f84e45 100644 --- a/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/LibvpxVideoRenderer.java +++ b/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/LibvpxVideoRenderer.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.ext.vp9; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITHOUT_RECONFIGURATION; import static java.lang.Runtime.getRuntime; import android.os.Handler; @@ -23,6 +24,7 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.RendererCapabilities; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.drm.ExoMediaCrypto; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.TraceUtil; @@ -125,15 +127,16 @@ public String getName() { @Capabilities public final int supportsFormat(Format format) { if (!VpxLibrary.isAvailable() || !MimeTypes.VIDEO_VP9.equalsIgnoreCase(format.sampleMimeType)) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } boolean drmIsSupported = format.exoMediaCryptoType == null || VpxLibrary.matchesExpectedExoMediaCryptoType(format.exoMediaCryptoType); if (!drmIsSupported) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_DRM); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_DRM); } - return RendererCapabilities.create(FORMAT_HANDLED, ADAPTIVE_SEAMLESS, TUNNELING_NOT_SUPPORTED); + return RendererCapabilities.create( + C.FORMAT_HANDLED, ADAPTIVE_SEAMLESS, TUNNELING_NOT_SUPPORTED); } @Override @@ -169,7 +172,13 @@ protected void setDecoderOutputMode(@C.VideoOutputMode int outputMode) { } @Override - protected boolean canKeepCodec(Format oldFormat, Format newFormat) { - return true; + protected DecoderReuseEvaluation canReuseDecoder( + String decoderName, Format oldFormat, Format newFormat) { + return new DecoderReuseEvaluation( + decoderName, + oldFormat, + newFormat, + REUSE_RESULT_YES_WITHOUT_RECONFIGURATION, + /* discardReasons= */ 0); } } diff --git a/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/VpxDecoder.java b/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/VpxDecoder.java index ce0873ad404..021ce3a9460 100644 --- a/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/VpxDecoder.java +++ b/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/VpxDecoder.java @@ -15,8 +15,11 @@ */ package com.google.android.exoplayer2.ext.vp9; +import static androidx.annotation.VisibleForTesting.PACKAGE_PRIVATE; + import android.view.Surface; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.decoder.CryptoInfo; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; @@ -30,7 +33,8 @@ import java.nio.ByteBuffer; /** Vpx decoder. */ -/* package */ final class VpxDecoder +@VisibleForTesting(otherwise = PACKAGE_PRIVATE) +public final class VpxDecoder extends SimpleDecoder { // These constants should match the codes returned from vpxDecode and vpxSecureDecode functions in diff --git a/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/VpxOutputBuffer.java b/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/VpxOutputBuffer.java index 99f35217fc1..6d36164d1de 100644 --- a/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/VpxOutputBuffer.java +++ b/extensions/vp9/src/main/java/com/google/android/exoplayer2/ext/vp9/VpxOutputBuffer.java @@ -18,7 +18,8 @@ import com.google.android.exoplayer2.video.VideoDecoderOutputBuffer; // TODO(b/139174707): Delete this class once binaries in WVVp9OpusPlaybackTest are updated to depend -// on VideoDecoderOutputBuffer. Also mark VideoDecoderOutputBuffer as final. +// on VideoDecoderOutputBuffer. Also mark VideoDecoderOutputBuffer as final and remove proguard +// config for VpxOutputBuffer. /** * Video output buffer, populated by {@link VpxDecoder}. * diff --git a/library/all/build.gradle b/library/all/build.gradle index fa3491bb5d7..e18d856c838 100644 --- a/library/all/build.gradle +++ b/library/all/build.gradle @@ -18,6 +18,7 @@ dependencies { api project(modulePrefix + 'library-dash') api project(modulePrefix + 'library-hls') api project(modulePrefix + 'library-smoothstreaming') + api project(modulePrefix + 'library-transformer') api project(modulePrefix + 'library-ui') } diff --git a/library/common/build.gradle b/library/common/build.gradle index de0df42506d..d1d0d86f422 100644 --- a/library/common/build.gradle +++ b/library/common/build.gradle @@ -35,7 +35,9 @@ dependencies { testImplementation 'androidx.test.ext:junit:' + androidxTestJUnitVersion testImplementation 'junit:junit:' + junitVersion testImplementation 'com.google.truth:truth:' + truthVersion + testImplementation 'com.squareup.okhttp3:mockwebserver:' + mockWebServerVersion testImplementation 'org.robolectric:robolectric:' + robolectricVersion + testImplementation project(modulePrefix + 'testutils') } ext { diff --git a/library/common/proguard-rules.txt b/library/common/proguard-rules.txt index 8de310a867f..2eb313c45d5 100644 --- a/library/common/proguard-rules.txt +++ b/library/common/proguard-rules.txt @@ -16,3 +16,7 @@ -dontwarn com.google.errorprone.annotations.** -dontwarn com.google.j2objc.annotations.** -dontwarn org.codehaus.mojo.animal_sniffer.IgnoreJRERequirement + +# Workaround for https://issuetracker.google.com/issues/112297269 +# This is needed for ProGuard but not R8. +-keepclassmembernames class com.google.common.base.Function { *; } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/BasePlayer.java b/library/common/src/main/java/com/google/android/exoplayer2/BasePlayer.java similarity index 82% rename from library/core/src/main/java/com/google/android/exoplayer2/BasePlayer.java rename to library/common/src/main/java/com/google/android/exoplayer2/BasePlayer.java index 4f89925121f..d99a320e32b 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/BasePlayer.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/BasePlayer.java @@ -214,7 +214,7 @@ public final boolean isCurrentWindowDynamic() { @Override public final boolean isCurrentWindowLive() { Timeline timeline = getCurrentTimeline(); - return !timeline.isEmpty() && timeline.getWindow(getCurrentWindowIndex(), window).isLive; + return !timeline.isEmpty() && timeline.getWindow(getCurrentWindowIndex(), window).isLive(); } @Override @@ -249,58 +249,4 @@ private int getRepeatModeForNavigation() { @RepeatMode int repeatMode = getRepeatMode(); return repeatMode == REPEAT_MODE_ONE ? REPEAT_MODE_OFF : repeatMode; } - - /** Holds a listener reference. */ - protected static final class ListenerHolder { - - /** - * The listener on which {link #invoke} will execute {@link ListenerInvocation listener - * invocations}. - */ - public final Player.EventListener listener; - - private boolean released; - - public ListenerHolder(Player.EventListener listener) { - this.listener = listener; - } - - /** Prevents any further {@link ListenerInvocation} to be executed on {@link #listener}. */ - public void release() { - released = true; - } - - /** - * Executes the given {@link ListenerInvocation} on {@link #listener}. Does nothing if {@link - * #release} has been called on this instance. - */ - public void invoke(ListenerInvocation listenerInvocation) { - if (!released) { - listenerInvocation.invokeListener(listener); - } - } - - @Override - public boolean equals(@Nullable Object other) { - if (this == other) { - return true; - } - if (other == null || getClass() != other.getClass()) { - return false; - } - return listener.equals(((ListenerHolder) other).listener); - } - - @Override - public int hashCode() { - return listener.hashCode(); - } - } - - /** Parameterized invocation of a {@link Player.EventListener} method. */ - protected interface ListenerInvocation { - - /** Executes the invocation on the given {@link Player.EventListener}. */ - void invokeListener(Player.EventListener listener); - } } diff --git a/library/common/src/main/java/com/google/android/exoplayer2/C.java b/library/common/src/main/java/com/google/android/exoplayer2/C.java index b4208c5282a..1c2cc923624 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/C.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/C.java @@ -24,6 +24,7 @@ import androidx.annotation.IntDef; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.Util; import java.lang.annotation.Documented; import java.lang.annotation.Retention; @@ -60,9 +61,10 @@ private C() {} */ public static final int POSITION_UNSET = -1; - /** - * Represents an unset or unknown length. - */ + /** Represents an unset or unknown rate. */ + public static final float RATE_UNSET = -Float.MAX_VALUE; + + /** Represents an unset or unknown length. */ public static final int LENGTH_UNSET = -1; /** Represents an unset or unknown percentage. */ @@ -555,24 +557,21 @@ private C() {} // ../../../../../../../../../extensions/vp9/src/main/jni/vpx_jni.cc // ) - /** @deprecated Use {@code Renderer.VideoScalingMode}. */ - @SuppressWarnings("deprecation") + /** + * Video scaling modes for {@link MediaCodec}-based renderers. One of {@link + * #VIDEO_SCALING_MODE_SCALE_TO_FIT} or {@link #VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING}. + */ @Documented @Retention(RetentionPolicy.SOURCE) @IntDef(value = {VIDEO_SCALING_MODE_SCALE_TO_FIT, VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING}) - @Deprecated public @interface VideoScalingMode {} - /** @deprecated Use {@code Renderer.VIDEO_SCALING_MODE_SCALE_TO_FIT}. */ - @Deprecated + /** See {@link MediaCodec#VIDEO_SCALING_MODE_SCALE_TO_FIT}. */ public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT = MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT; - /** @deprecated Use {@code Renderer.VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING}. */ - @Deprecated + /** See {@link MediaCodec#VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING}. */ public static final int VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING = MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING; - /** @deprecated Use {@code Renderer.VIDEO_SCALING_MODE_DEFAULT}. */ - @SuppressWarnings("deprecation") - @Deprecated + /** A default video scaling mode for {@link MediaCodec}-based renderers. */ public static final int VIDEO_SCALING_MODE_DEFAULT = VIDEO_SCALING_MODE_SCALE_TO_FIT; /** @@ -681,12 +680,14 @@ private C() {} public static final int TRACK_TYPE_VIDEO = 2; /** A type constant for text tracks. */ public static final int TRACK_TYPE_TEXT = 3; + /** A type constant for image tracks. */ + public static final int TRACK_TYPE_IMAGE = 4; /** A type constant for metadata tracks. */ - public static final int TRACK_TYPE_METADATA = 4; + public static final int TRACK_TYPE_METADATA = 5; /** A type constant for camera motion tracks. */ - public static final int TRACK_TYPE_CAMERA_MOTION = 5; + public static final int TRACK_TYPE_CAMERA_MOTION = 6; /** A type constant for a fake or empty track. */ - public static final int TRACK_TYPE_NONE = 6; + public static final int TRACK_TYPE_NONE = 7; /** * Applications or extensions may define custom {@code TRACK_TYPE_*} constants greater than or * equal to this value. @@ -1080,8 +1081,63 @@ private C() {} public static final int ROLE_FLAG_TRICK_PLAY = 1 << 14; /** - * Converts a time in microseconds to the corresponding time in milliseconds, preserving - * {@link #TIME_UNSET} and {@link #TIME_END_OF_SOURCE} values. + * Level of renderer support for a format. One of {@link #FORMAT_HANDLED}, {@link + * #FORMAT_EXCEEDS_CAPABILITIES}, {@link #FORMAT_UNSUPPORTED_DRM}, {@link + * #FORMAT_UNSUPPORTED_SUBTYPE} or {@link #FORMAT_UNSUPPORTED_TYPE}. + */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + FORMAT_HANDLED, + FORMAT_EXCEEDS_CAPABILITIES, + FORMAT_UNSUPPORTED_DRM, + FORMAT_UNSUPPORTED_SUBTYPE, + FORMAT_UNSUPPORTED_TYPE + }) + public static @interface FormatSupport {} + // TODO(b/172315872) Renderer was a link. Link to equivalent concept or remove @code. + /** The {@code Renderer} is capable of rendering the format. */ + public static final int FORMAT_HANDLED = 0b100; + /** + * The {@code Renderer} is capable of rendering formats with the same MIME type, but the + * properties of the format exceed the renderer's capabilities. There is a chance the renderer + * will be able to play the format in practice because some renderers report their capabilities + * conservatively, but the expected outcome is that playback will fail. + * + *

Example: The {@code Renderer} is capable of rendering H264 and the format's MIME type is + * {@code MimeTypes#VIDEO_H264}, but the format's resolution exceeds the maximum limit supported + * by the underlying H264 decoder. + */ + public static final int FORMAT_EXCEEDS_CAPABILITIES = 0b011; + /** + * The {@code Renderer} is capable of rendering formats with the same MIME type, but is not + * capable of rendering the format because the format's drm protection is not supported. + * + *

Example: The {@code Renderer} is capable of rendering H264 and the format's MIME type is + * {@link MimeTypes#VIDEO_H264}, but the format indicates PlayReady drm protection whereas the + * renderer only supports Widevine. + */ + public static final int FORMAT_UNSUPPORTED_DRM = 0b010; + /** + * The {@code Renderer} is a general purpose renderer for formats of the same top-level type, but + * is not capable of rendering the format or any other format with the same MIME type because the + * sub-type is not supported. + * + *

Example: The {@code Renderer} is a general purpose audio renderer and the format's MIME type + * matches audio/[subtype], but there does not exist a suitable decoder for [subtype]. + */ + public static final int FORMAT_UNSUPPORTED_SUBTYPE = 0b001; + /** + * The {@code Renderer} is not capable of rendering the format, either because it does not support + * the format's top-level type, or because it's a specialized renderer for a different MIME type. + * + *

Example: The {@code Renderer} is a general purpose video renderer, but the format has an + * audio MIME type. + */ + public static final int FORMAT_UNSUPPORTED_TYPE = 0b000; + /** + * Converts a time in microseconds to the corresponding time in milliseconds, preserving {@link + * #TIME_UNSET} and {@link #TIME_END_OF_SOURCE} values. * * @param timeUs The time in microseconds. * @return The corresponding time in milliseconds. @@ -1114,4 +1170,26 @@ public static int generateAudioSessionIdV21(Context context) { return audioManager == null ? AudioManager.ERROR : audioManager.generateAudioSessionId(); } + /** + * Returns string representation of a {@link FormatSupport} flag. + * + * @param formatSupport A {@link FormatSupport} flag. + * @return A string representation of the flag. + */ + public static String getFormatSupportString(@FormatSupport int formatSupport) { + switch (formatSupport) { + case FORMAT_HANDLED: + return "YES"; + case FORMAT_EXCEEDS_CAPABILITIES: + return "NO_EXCEEDS_CAPABILITIES"; + case FORMAT_UNSUPPORTED_DRM: + return "NO_UNSUPPORTED_DRM"; + case FORMAT_UNSUPPORTED_SUBTYPE: + return "NO_UNSUPPORTED_TYPE"; + case FORMAT_UNSUPPORTED_TYPE: + return "NO"; + default: + throw new IllegalStateException(); + } + } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlaybackException.java b/library/common/src/main/java/com/google/android/exoplayer2/ExoPlaybackException.java similarity index 64% rename from library/core/src/main/java/com/google/android/exoplayer2/ExoPlaybackException.java rename to library/common/src/main/java/com/google/android/exoplayer2/ExoPlaybackException.java index 93fb4b01186..95edfdf6f4a 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlaybackException.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/ExoPlaybackException.java @@ -20,48 +20,39 @@ import androidx.annotation.CheckResult; import androidx.annotation.IntDef; import androidx.annotation.Nullable; -import com.google.android.exoplayer2.RendererCapabilities.FormatSupport; -import com.google.android.exoplayer2.source.MediaSource; +import com.google.android.exoplayer2.C.FormatSupport; +import com.google.android.exoplayer2.source.MediaPeriodId; import com.google.android.exoplayer2.util.Assertions; import java.io.IOException; import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; -import java.util.concurrent.TimeoutException; -/** - * Thrown when a non-recoverable playback failure occurs. - */ +/** Thrown when a non locally recoverable playback failure occurs. */ public final class ExoPlaybackException extends Exception { /** * The type of source that produced the error. One of {@link #TYPE_SOURCE}, {@link #TYPE_RENDERER} - * {@link #TYPE_UNEXPECTED}, {@link #TYPE_REMOTE}, {@link #TYPE_OUT_OF_MEMORY} or {@link - * #TYPE_TIMEOUT}. Note that new types may be added in the future and error handling should handle - * unknown type values. + * {@link #TYPE_UNEXPECTED} or {@link #TYPE_REMOTE}. Note that new types may be added in the + * future and error handling should handle unknown type values. */ @Documented @Retention(RetentionPolicy.SOURCE) - @IntDef({ - TYPE_SOURCE, - TYPE_RENDERER, - TYPE_UNEXPECTED, - TYPE_REMOTE, - TYPE_OUT_OF_MEMORY, - TYPE_TIMEOUT - }) + @IntDef({TYPE_SOURCE, TYPE_RENDERER, TYPE_UNEXPECTED, TYPE_REMOTE}) public @interface Type {} /** - * The error occurred loading data from a {@link MediaSource}. - *

- * Call {@link #getSourceException()} to retrieve the underlying cause. + * The error occurred loading data from a {@code MediaSource}. + * + *

Call {@link #getSourceException()} to retrieve the underlying cause. */ + // TODO(b/172315872) MediaSource was a link. Link to equivalent concept or remove @code. public static final int TYPE_SOURCE = 0; /** - * The error occurred in a {@link Renderer}. - *

- * Call {@link #getRendererException()} to retrieve the underlying cause. + * The error occurred in a {@code Renderer}. + * + *

Call {@link #getRendererException()} to retrieve the underlying cause. */ + // TODO(b/172315872) Renderer was a link. Link to equivalent concept or remove @code. public static final int TYPE_RENDERER = 1; /** * The error was an unexpected {@link RuntimeException}. @@ -75,40 +66,20 @@ public final class ExoPlaybackException extends Exception { *

Call {@link #getMessage()} to retrieve the message associated with the error. */ public static final int TYPE_REMOTE = 3; - /** The error was an {@link OutOfMemoryError}. */ - public static final int TYPE_OUT_OF_MEMORY = 4; - /** The error was a {@link TimeoutException}. */ - public static final int TYPE_TIMEOUT = 5; /** The {@link Type} of the playback failure. */ @Type public final int type; /** - * The operation which produced the timeout error. One of {@link #TIMEOUT_OPERATION_RELEASE}, - * {@link #TIMEOUT_OPERATION_SET_FOREGROUND_MODE} or {@link #TIMEOUT_OPERATION_UNDEFINED}. Note - * that new operations may be added in the future and error handling should handle unknown - * operation values. + * If {@link #type} is {@link #TYPE_RENDERER}, this is the name of the renderer, or null if + * unknown. */ - @Documented - @Retention(RetentionPolicy.SOURCE) - @IntDef({ - TIMEOUT_OPERATION_UNDEFINED, - TIMEOUT_OPERATION_RELEASE, - TIMEOUT_OPERATION_SET_FOREGROUND_MODE - }) - public @interface TimeoutOperation {} - - /** The operation where this error occurred is not defined. */ - public static final int TIMEOUT_OPERATION_UNDEFINED = 0; - /** The error occurred in {@link ExoPlayer#release}. */ - public static final int TIMEOUT_OPERATION_RELEASE = 1; - /** The error occurred in {@link ExoPlayer#setForegroundMode}. */ - public static final int TIMEOUT_OPERATION_SET_FOREGROUND_MODE = 2; - - /** If {@link #type} is {@link #TYPE_RENDERER}, this is the name of the renderer. */ @Nullable public final String rendererName; - /** If {@link #type} is {@link #TYPE_RENDERER}, this is the index of the renderer. */ + /** + * If {@link #type} is {@link #TYPE_RENDERER}, this is the index of the renderer, or {@link + * C#INDEX_UNSET} if unknown. + */ public final int rendererIndex; /** @@ -120,23 +91,26 @@ public final class ExoPlaybackException extends Exception { /** * If {@link #type} is {@link #TYPE_RENDERER}, this is the level of {@link FormatSupport} of the * renderer for {@link #rendererFormat}. If {@link #rendererFormat} is null, this is {@link - * RendererCapabilities#FORMAT_HANDLED}. + * C#FORMAT_HANDLED}. */ @FormatSupport public final int rendererFormatSupport; - /** - * If {@link #type} is {@link #TYPE_TIMEOUT}, this is the operation where the timeout happened. - */ - @TimeoutOperation public final int timeoutOperation; - /** The value of {@link SystemClock#elapsedRealtime()} when this exception was created. */ public final long timestampMs; + /** The {@link MediaPeriodId} of the media associated with this error, or null if undetermined. */ + @Nullable public final MediaPeriodId mediaPeriodId; + /** - * The {@link MediaSource.MediaPeriodId} of the media associated with this error, or null if - * undetermined. + * Whether the error may be recoverable. + * + *

This is only used internally by ExoPlayer to try to recover from some errors and should not + * be used by apps. + * + *

If the {@link #type} is {@link #TYPE_RENDERER}, it may be possible to recover from the error + * by disabling and re-enabling the renderers. */ - @Nullable public final MediaSource.MediaPeriodId mediaPeriodId; + /* package */ final boolean isRecoverable; @Nullable private final Throwable cause; @@ -150,6 +124,24 @@ public static ExoPlaybackException createForSource(IOException cause) { return new ExoPlaybackException(TYPE_SOURCE, cause); } + /** + * Creates an instance of type {@link #TYPE_RENDERER} for an unknown renderer. + * + * @param cause The cause of the failure. + * @return The created instance. + */ + public static ExoPlaybackException createForRenderer(Exception cause) { + return new ExoPlaybackException( + TYPE_RENDERER, + cause, + /* customMessage= */ null, + /* rendererName */ null, + /* rendererIndex= */ C.INDEX_UNSET, + /* rendererFormat= */ null, + /* rendererFormatSupport= */ C.FORMAT_HANDLED, + /* isRecoverable= */ false); + } + /** * Creates an instance of type {@link #TYPE_RENDERER}. * @@ -162,11 +154,39 @@ public static ExoPlaybackException createForSource(IOException cause) { * @return The created instance. */ public static ExoPlaybackException createForRenderer( - Exception cause, + Throwable cause, String rendererName, int rendererIndex, @Nullable Format rendererFormat, @FormatSupport int rendererFormatSupport) { + return createForRenderer( + cause, + rendererName, + rendererIndex, + rendererFormat, + rendererFormatSupport, + /* isRecoverable= */ false); + } + + /** + * Creates an instance of type {@link #TYPE_RENDERER}. + * + * @param cause The cause of the failure. + * @param rendererIndex The index of the renderer in which the failure occurred. + * @param rendererFormat The {@link Format} the renderer was using at the time of the exception, + * or null if the renderer wasn't using a {@link Format}. + * @param rendererFormatSupport The {@link FormatSupport} of the renderer for {@code + * rendererFormat}. Ignored if {@code rendererFormat} is null. + * @param isRecoverable If the failure can be recovered by disabling and re-enabling the renderer. + * @return The created instance. + */ + public static ExoPlaybackException createForRenderer( + Throwable cause, + String rendererName, + int rendererIndex, + @Nullable Format rendererFormat, + @FormatSupport int rendererFormatSupport, + boolean isRecoverable) { return new ExoPlaybackException( TYPE_RENDERER, cause, @@ -174,8 +194,8 @@ public static ExoPlaybackException createForRenderer( rendererName, rendererIndex, rendererFormat, - rendererFormat == null ? RendererCapabilities.FORMAT_HANDLED : rendererFormatSupport, - TIMEOUT_OPERATION_UNDEFINED); + rendererFormat == null ? C.FORMAT_HANDLED : rendererFormatSupport, + isRecoverable); } /** @@ -198,36 +218,6 @@ public static ExoPlaybackException createForRemote(String message) { return new ExoPlaybackException(TYPE_REMOTE, message); } - /** - * Creates an instance of type {@link #TYPE_OUT_OF_MEMORY}. - * - * @param cause The cause of the failure. - * @return The created instance. - */ - public static ExoPlaybackException createForOutOfMemory(OutOfMemoryError cause) { - return new ExoPlaybackException(TYPE_OUT_OF_MEMORY, cause); - } - - /** - * Creates an instance of type {@link #TYPE_TIMEOUT}. - * - * @param cause The cause of the failure. - * @param timeoutOperation The operation that caused this timeout. - * @return The created instance. - */ - public static ExoPlaybackException createForTimeout( - TimeoutException cause, @TimeoutOperation int timeoutOperation) { - return new ExoPlaybackException( - TYPE_TIMEOUT, - cause, - /* customMessage= */ null, - /* rendererName= */ null, - /* rendererIndex= */ C.INDEX_UNSET, - /* rendererFormat= */ null, - /* rendererFormatSupport= */ RendererCapabilities.FORMAT_HANDLED, - timeoutOperation); - } - private ExoPlaybackException(@Type int type, Throwable cause) { this( type, @@ -236,8 +226,8 @@ private ExoPlaybackException(@Type int type, Throwable cause) { /* rendererName= */ null, /* rendererIndex= */ C.INDEX_UNSET, /* rendererFormat= */ null, - /* rendererFormatSupport= */ RendererCapabilities.FORMAT_HANDLED, - TIMEOUT_OPERATION_UNDEFINED); + /* rendererFormatSupport= */ C.FORMAT_HANDLED, + /* isRecoverable= */ false); } private ExoPlaybackException(@Type int type, String message) { @@ -248,8 +238,8 @@ private ExoPlaybackException(@Type int type, String message) { /* rendererName= */ null, /* rendererIndex= */ C.INDEX_UNSET, /* rendererFormat= */ null, - /* rendererFormatSupport= */ RendererCapabilities.FORMAT_HANDLED, - /* timeoutOperation= */ TIMEOUT_OPERATION_UNDEFINED); + /* rendererFormatSupport= */ C.FORMAT_HANDLED, + /* isRecoverable= */ false); } private ExoPlaybackException( @@ -260,7 +250,7 @@ private ExoPlaybackException( int rendererIndex, @Nullable Format rendererFormat, @FormatSupport int rendererFormatSupport, - @TimeoutOperation int timeoutOperation) { + boolean isRecoverable) { this( deriveMessage( type, @@ -276,8 +266,8 @@ private ExoPlaybackException( rendererFormat, rendererFormatSupport, /* mediaPeriodId= */ null, - timeoutOperation, - /* timestampMs= */ SystemClock.elapsedRealtime()); + /* timestampMs= */ SystemClock.elapsedRealtime(), + isRecoverable); } private ExoPlaybackException( @@ -288,9 +278,9 @@ private ExoPlaybackException( int rendererIndex, @Nullable Format rendererFormat, @FormatSupport int rendererFormatSupport, - @Nullable MediaSource.MediaPeriodId mediaPeriodId, - @TimeoutOperation int timeoutOperation, - long timestampMs) { + @Nullable MediaPeriodId mediaPeriodId, + long timestampMs, + boolean isRecoverable) { super(message, cause); this.type = type; this.cause = cause; @@ -299,8 +289,8 @@ private ExoPlaybackException( this.rendererFormat = rendererFormat; this.rendererFormatSupport = rendererFormatSupport; this.mediaPeriodId = mediaPeriodId; - this.timeoutOperation = timeoutOperation; this.timestampMs = timestampMs; + this.isRecoverable = isRecoverable; } /** @@ -334,34 +324,13 @@ public RuntimeException getUnexpectedException() { } /** - * Retrieves the underlying error when {@link #type} is {@link #TYPE_OUT_OF_MEMORY}. + * Returns a copy of this exception with the provided {@link MediaPeriodId}. * - * @throws IllegalStateException If {@link #type} is not {@link #TYPE_OUT_OF_MEMORY}. - */ - public OutOfMemoryError getOutOfMemoryError() { - Assertions.checkState(type == TYPE_OUT_OF_MEMORY); - return (OutOfMemoryError) Assertions.checkNotNull(cause); - } - - /** - * Retrieves the underlying error when {@link #type} is {@link #TYPE_TIMEOUT}. - * - * @throws IllegalStateException If {@link #type} is not {@link #TYPE_TIMEOUT}. - */ - public TimeoutException getTimeoutException() { - Assertions.checkState(type == TYPE_TIMEOUT); - return (TimeoutException) Assertions.checkNotNull(cause); - } - - /** - * Returns a copy of this exception with the provided {@link MediaSource.MediaPeriodId}. - * - * @param mediaPeriodId The {@link MediaSource.MediaPeriodId}. + * @param mediaPeriodId The {@link MediaPeriodId}. * @return The copied exception. */ @CheckResult - /* package= */ ExoPlaybackException copyWithMediaPeriodId( - @Nullable MediaSource.MediaPeriodId mediaPeriodId) { + /* package */ ExoPlaybackException copyWithMediaPeriodId(@Nullable MediaPeriodId mediaPeriodId) { return new ExoPlaybackException( getMessage(), cause, @@ -371,8 +340,8 @@ public TimeoutException getTimeoutException() { rendererFormat, rendererFormatSupport, mediaPeriodId, - timeoutOperation, - timestampMs); + timestampMs, + isRecoverable); } @Nullable @@ -397,17 +366,11 @@ private static String deriveMessage( + ", format=" + rendererFormat + ", format_supported=" - + RendererCapabilities.getFormatSupportString(rendererFormatSupport); + + C.getFormatSupportString(rendererFormatSupport); break; case TYPE_REMOTE: message = "Remote error"; break; - case TYPE_OUT_OF_MEMORY: - message = "Out of memory error"; - break; - case TYPE_TIMEOUT: - message = "Timeout error"; - break; case TYPE_UNEXPECTED: default: message = "Unexpected runtime error"; diff --git a/library/common/src/main/java/com/google/android/exoplayer2/ExoPlayerLibraryInfo.java b/library/common/src/main/java/com/google/android/exoplayer2/ExoPlayerLibraryInfo.java index aadf3bd0a4b..21f352590cc 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/ExoPlayerLibraryInfo.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/ExoPlayerLibraryInfo.java @@ -30,11 +30,11 @@ public final class ExoPlayerLibraryInfo { /** The version of the library expressed as a string, for example "1.2.3". */ // Intentionally hardcoded. Do not derive from other constants (e.g. VERSION_INT) or vice versa. - public static final String VERSION = "2.12.3"; + public static final String VERSION = "2.13.0"; /** The version of the library expressed as {@code "ExoPlayerLib/" + VERSION}. */ // Intentionally hardcoded. Do not derive from other constants (e.g. VERSION) or vice versa. - public static final String VERSION_SLASHY = "ExoPlayerLib/2.12.3"; + public static final String VERSION_SLASHY = "ExoPlayerLib/2.13.0"; /** * The version of the library expressed as an integer, for example 1002003. @@ -44,11 +44,16 @@ public final class ExoPlayerLibraryInfo { * integer version 123045006 (123-045-006). */ // Intentionally hardcoded. Do not derive from other constants (e.g. VERSION) or vice versa. - public static final int VERSION_INT = 2012003; + public static final int VERSION_INT = 2013000; - /** The default user agent for requests made by the library. */ + /** + * The default user agent for requests made by the library. + * + * @deprecated ExoPlayer now uses the user agent of the underlying network stack by default. + */ + @Deprecated public static final String DEFAULT_USER_AGENT = - VERSION_SLASHY + " (Linux;Android " + Build.VERSION.RELEASE + ") " + VERSION_SLASHY; + VERSION_SLASHY + " (Linux; Android " + Build.VERSION.RELEASE + ") " + VERSION_SLASHY; /** * Whether the library was compiled with {@link com.google.android.exoplayer2.util.Assertions} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/IllegalSeekPositionException.java b/library/common/src/main/java/com/google/android/exoplayer2/IllegalSeekPositionException.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/IllegalSeekPositionException.java rename to library/common/src/main/java/com/google/android/exoplayer2/IllegalSeekPositionException.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/MediaItem.java b/library/common/src/main/java/com/google/android/exoplayer2/MediaItem.java index 0c25f2d43ea..184c065e8a6 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/MediaItem.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/MediaItem.java @@ -15,6 +15,9 @@ */ package com.google.android.exoplayer2; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; + import android.net.Uri; import androidx.annotation.Nullable; import com.google.android.exoplayer2.offline.StreamKey; @@ -74,8 +77,14 @@ public static final class Builder { @Nullable private String customCacheKey; private List subtitles; @Nullable private Uri adTagUri; + @Nullable private Object adsId; @Nullable private Object tag; @Nullable private MediaMetadata mediaMetadata; + private long liveTargetOffsetMs; + private long liveMinOffsetMs; + private long liveMaxOffsetMs; + private float liveMinPlaybackSpeed; + private float liveMaxPlaybackSpeed; /** Creates a builder. */ public Builder() { @@ -84,6 +93,11 @@ public Builder() { drmLicenseRequestHeaders = Collections.emptyMap(); streamKeys = Collections.emptyList(); subtitles = Collections.emptyList(); + liveTargetOffsetMs = C.TIME_UNSET; + liveMinOffsetMs = C.TIME_UNSET; + liveMaxOffsetMs = C.TIME_UNSET; + liveMinPlaybackSpeed = C.RATE_UNSET; + liveMaxPlaybackSpeed = C.RATE_UNSET; } private Builder(MediaItem mediaItem) { @@ -95,9 +109,13 @@ private Builder(MediaItem mediaItem) { clipStartsAtKeyFrame = mediaItem.clippingProperties.startsAtKeyFrame; mediaId = mediaItem.mediaId; mediaMetadata = mediaItem.mediaMetadata; + liveTargetOffsetMs = mediaItem.liveConfiguration.targetOffsetMs; + liveMinOffsetMs = mediaItem.liveConfiguration.minOffsetMs; + liveMaxOffsetMs = mediaItem.liveConfiguration.maxOffsetMs; + liveMinPlaybackSpeed = mediaItem.liveConfiguration.minPlaybackSpeed; + liveMaxPlaybackSpeed = mediaItem.liveConfiguration.maxPlaybackSpeed; @Nullable PlaybackProperties playbackProperties = mediaItem.playbackProperties; if (playbackProperties != null) { - adTagUri = playbackProperties.adTagUri; customCacheKey = playbackProperties.customCacheKey; mimeType = playbackProperties.mimeType; uri = playbackProperties.uri; @@ -115,6 +133,11 @@ private Builder(MediaItem mediaItem) { drmUuid = drmConfiguration.uuid; drmKeySetId = drmConfiguration.getKeySetId(); } + @Nullable AdsConfiguration adsConfiguration = playbackProperties.adsConfiguration; + if (adsConfiguration != null) { + adTagUri = adsConfiguration.adTagUri; + adsId = adsConfiguration.adsId; + } } } @@ -393,14 +416,19 @@ public Builder setSubtitles(@Nullable List subtitles) { } /** - * Sets the optional ad tag URI. + * Sets the optional ad tag {@link Uri}. * *

If {@link #setUri} is passed a non-null {@code uri}, the ad tag URI is used to create a * {@link PlaybackProperties} object. Otherwise it will be ignored. + * + *

Media items in the playlist with the same ad tag URI, media ID and ads loader will share + * the same ad playback state. To resume ad playback when recreating the playlist on returning + * from the background, pass media items with the same ad tag URIs and media IDs to the player. + * + * @param adTagUri The ad tag URI to load. */ public Builder setAdTagUri(@Nullable String adTagUri) { - this.adTagUri = adTagUri != null ? Uri.parse(adTagUri) : null; - return this; + return setAdTagUri(adTagUri != null ? Uri.parse(adTagUri) : null); } /** @@ -408,9 +436,102 @@ public Builder setAdTagUri(@Nullable String adTagUri) { * *

If {@link #setUri} is passed a non-null {@code uri}, the ad tag URI is used to create a * {@link PlaybackProperties} object. Otherwise it will be ignored. + * + *

Media items in the playlist with the same ad tag URI, media ID and ads loader will share + * the same ad playback state. To resume ad playback when recreating the playlist on returning + * from the background, pass media items with the same ad tag URIs and media IDs to the player. + * + * @param adTagUri The ad tag URI to load. */ public Builder setAdTagUri(@Nullable Uri adTagUri) { + return setAdTagUri(adTagUri, /* adsId= */ null); + } + + /** + * Sets the optional ad tag {@link Uri} and ads identifier. + * + *

If {@link #setUri} is passed a non-null {@code uri}, the ad tag URI is used to create a + * {@link PlaybackProperties} object. Otherwise it will be ignored. + * + *

Media items in the playlist that have the same ads identifier and ads loader share the + * same ad playback state. To resume ad playback when recreating the playlist on returning from + * the background, pass the same ads IDs to the player. + * + * @param adTagUri The ad tag URI to load. + * @param adsId An opaque identifier for ad playback state associated with this item. Ad loading + * and playback state is shared among all media items that have the same ads ID (by {@link + * Object#equals(Object) equality}) and ads loader, so it is important to pass the same + * identifiers when constructing playlist items each time the player returns to the + * foreground. + */ + public Builder setAdTagUri(@Nullable Uri adTagUri, @Nullable Object adsId) { this.adTagUri = adTagUri; + this.adsId = adsId; + return this; + } + + /** + * Sets the optional target offset from the live edge for live streams, in milliseconds. + * + *

See {@code Player#getCurrentLiveOffset()}. + * + * @param liveTargetOffsetMs The target offset, in milliseconds, or {@link C#TIME_UNSET} to use + * the media-defined default. + */ + public Builder setLiveTargetOffsetMs(long liveTargetOffsetMs) { + this.liveTargetOffsetMs = liveTargetOffsetMs; + return this; + } + + /** + * Sets the optional minimum offset from the live edge for live streams, in milliseconds. + * + *

See {@code Player#getCurrentLiveOffset()}. + * + * @param liveMinOffsetMs The minimum allowed offset, in milliseconds, or {@link C#TIME_UNSET} + * to use the media-defined default. + */ + public Builder setLiveMinOffsetMs(long liveMinOffsetMs) { + this.liveMinOffsetMs = liveMinOffsetMs; + return this; + } + + /** + * Sets the optional maximum offset from the live edge for live streams, in milliseconds. + * + *

See {@code Player#getCurrentLiveOffset()}. + * + * @param liveMaxOffsetMs The maximum allowed offset, in milliseconds, or {@link C#TIME_UNSET} + * to use the media-defined default. + */ + public Builder setLiveMaxOffsetMs(long liveMaxOffsetMs) { + this.liveMaxOffsetMs = liveMaxOffsetMs; + return this; + } + + /** + * Sets the optional minimum playback speed for live stream speed adjustment. + * + *

This value is ignored for other stream types. + * + * @param minPlaybackSpeed The minimum factor by which playback can be sped up for live streams, + * or {@link C#RATE_UNSET} to use the media-defined default. + */ + public Builder setLiveMinPlaybackSpeed(float minPlaybackSpeed) { + this.liveMinPlaybackSpeed = minPlaybackSpeed; + return this; + } + + /** + * Sets the optional maximum playback speed for live stream speed adjustment. + * + *

This value is ignored for other stream types. + * + * @param maxPlaybackSpeed The maximum factor by which playback can be sped up for live streams, + * or {@link C#RATE_UNSET} to use the media-defined default. + */ + public Builder setLiveMaxPlaybackSpeed(float maxPlaybackSpeed) { + this.liveMaxPlaybackSpeed = maxPlaybackSpeed; return this; } @@ -437,8 +558,9 @@ public Builder setMediaMetadata(MediaMetadata mediaMetadata) { * Returns a new {@link MediaItem} instance with the current builder values. */ public MediaItem build() { - Assertions.checkState(drmLicenseUri == null || drmUuid != null); + checkState(drmLicenseUri == null || drmUuid != null); @Nullable PlaybackProperties playbackProperties = null; + @Nullable Uri uri = this.uri; if (uri != null) { playbackProperties = new PlaybackProperties( @@ -455,15 +577,15 @@ public MediaItem build() { drmSessionForClearTypes, drmKeySetId) : null, + adTagUri != null ? new AdsConfiguration(adTagUri, adsId) : null, streamKeys, customCacheKey, subtitles, - adTagUri, tag); mediaId = mediaId != null ? mediaId : uri.toString(); } return new MediaItem( - Assertions.checkNotNull(mediaId), + checkNotNull(mediaId), new ClippingProperties( clipStartPositionMs, clipEndPositionMs, @@ -471,6 +593,12 @@ public MediaItem build() { clipRelativeToDefaultPosition, clipStartsAtKeyFrame), playbackProperties, + new LiveConfiguration( + liveTargetOffsetMs, + liveMinOffsetMs, + liveMaxOffsetMs, + liveMinPlaybackSpeed, + liveMaxPlaybackSpeed), mediaMetadata != null ? mediaMetadata : new MediaMetadata.Builder().build()); } } @@ -570,6 +698,54 @@ public int hashCode() { } } + /** Configuration for playing back linear ads with a media item. */ + public static final class AdsConfiguration { + + /** The ad tag URI to load. */ + public final Uri adTagUri; + /** + * An opaque identifier for ad playback state associated with this item, or {@code null} if the + * combination of the {@link MediaItem.Builder#setMediaId(String) media ID} and {@link #adTagUri + * ad tag URI} should be used as the ads identifier. + */ + @Nullable public final Object adsId; + + /** + * Creates an ads configuration with the given ad tag URI and ads identifier. + * + * @param adTagUri The ad tag URI to load. + * @param adsId An opaque identifier for ad playback state associated with this item. Ad loading + * and playback state is shared among all media items that have the same ads ID (by {@link + * Object#equals(Object) equality}) and ads loader, so it is important to pass the same + * identifiers when constructing playlist items each time the player returns to the + * foreground. + */ + private AdsConfiguration(Uri adTagUri, @Nullable Object adsId) { + this.adTagUri = adTagUri; + this.adsId = adsId; + } + + @Override + public boolean equals(@Nullable Object obj) { + if (this == obj) { + return true; + } + if (!(obj instanceof AdsConfiguration)) { + return false; + } + + AdsConfiguration other = (AdsConfiguration) obj; + return adTagUri.equals(other.adTagUri) && Util.areEqual(adsId, other.adsId); + } + + @Override + public int hashCode() { + int result = adTagUri.hashCode(); + result = 31 * result + (adsId != null ? adsId.hashCode() : 0); + return result; + } + } + /** Properties for local playback. */ public static final class PlaybackProperties { @@ -587,6 +763,9 @@ public static final class PlaybackProperties { /** Optional {@link DrmConfiguration} for the media. */ @Nullable public final DrmConfiguration drmConfiguration; + /** Optional ads configuration. */ + @Nullable public final AdsConfiguration adsConfiguration; + /** Optional stream keys by which the manifest is filtered. */ public final List streamKeys; @@ -596,9 +775,6 @@ public static final class PlaybackProperties { /** Optional subtitles to be sideloaded. */ public final List subtitles; - /** Optional ad tag {@link Uri}. */ - @Nullable public final Uri adTagUri; - /** * Optional tag for custom attributes. The tag for the media source which will be published in * the {@code com.google.android.exoplayer2.Timeline} of the source as {@code @@ -610,18 +786,18 @@ private PlaybackProperties( Uri uri, @Nullable String mimeType, @Nullable DrmConfiguration drmConfiguration, + @Nullable AdsConfiguration adsConfiguration, List streamKeys, @Nullable String customCacheKey, List subtitles, - @Nullable Uri adTagUri, @Nullable Object tag) { this.uri = uri; this.mimeType = mimeType; this.drmConfiguration = drmConfiguration; + this.adsConfiguration = adsConfiguration; this.streamKeys = streamKeys; this.customCacheKey = customCacheKey; this.subtitles = subtitles; - this.adTagUri = adTagUri; this.tag = tag; } @@ -638,10 +814,10 @@ public boolean equals(@Nullable Object obj) { return uri.equals(other.uri) && Util.areEqual(mimeType, other.mimeType) && Util.areEqual(drmConfiguration, other.drmConfiguration) + && Util.areEqual(adsConfiguration, other.adsConfiguration) && streamKeys.equals(other.streamKeys) && Util.areEqual(customCacheKey, other.customCacheKey) && subtitles.equals(other.subtitles) - && Util.areEqual(adTagUri, other.adTagUri) && Util.areEqual(tag, other.tag); } @@ -650,15 +826,112 @@ public int hashCode() { int result = uri.hashCode(); result = 31 * result + (mimeType == null ? 0 : mimeType.hashCode()); result = 31 * result + (drmConfiguration == null ? 0 : drmConfiguration.hashCode()); + result = 31 * result + (adsConfiguration == null ? 0 : adsConfiguration.hashCode()); result = 31 * result + streamKeys.hashCode(); result = 31 * result + (customCacheKey == null ? 0 : customCacheKey.hashCode()); result = 31 * result + subtitles.hashCode(); - result = 31 * result + (adTagUri == null ? 0 : adTagUri.hashCode()); result = 31 * result + (tag == null ? 0 : tag.hashCode()); return result; } } + /** Live playback configuration. */ + public static final class LiveConfiguration { + + /** A live playback configuration with unset values. */ + public static final LiveConfiguration UNSET = + new LiveConfiguration( + /* targetLiveOffsetMs= */ C.TIME_UNSET, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET); + + /** + * Target offset from the live edge, in milliseconds, or {@link C#TIME_UNSET} to use the + * media-defined default. + */ + public final long targetOffsetMs; + + /** + * The minimum allowed offset from the live edge, in milliseconds, or {@link C#TIME_UNSET} to + * use the media-defined default. + */ + public final long minOffsetMs; + + /** + * The maximum allowed offset from the live edge, in milliseconds, or {@link C#TIME_UNSET} to + * use the media-defined default. + */ + public final long maxOffsetMs; + + /** + * Minimum factor by which playback can be sped up, or {@link C#RATE_UNSET} to use the + * media-defined default. + */ + public final float minPlaybackSpeed; + + /** + * Maximum factor by which playback can be sped up, or {@link C#RATE_UNSET} to use the + * media-defined default. + */ + public final float maxPlaybackSpeed; + + /** + * Creates a live playback configuration. + * + * @param targetOffsetMs Target live offset, in milliseconds, or {@link C#TIME_UNSET} to use the + * media-defined default. + * @param minOffsetMs The minimum allowed live offset, in milliseconds, or {@link C#TIME_UNSET} + * to use the media-defined default. + * @param maxOffsetMs The maximum allowed live offset, in milliseconds, or {@link C#TIME_UNSET} + * to use the media-defined default. + * @param minPlaybackSpeed Minimum playback speed, or {@link C#RATE_UNSET} to use the + * media-defined default. + * @param maxPlaybackSpeed Maximum playback speed, or {@link C#RATE_UNSET} to use the + * media-defined default. + */ + public LiveConfiguration( + long targetOffsetMs, + long minOffsetMs, + long maxOffsetMs, + float minPlaybackSpeed, + float maxPlaybackSpeed) { + this.targetOffsetMs = targetOffsetMs; + this.minOffsetMs = minOffsetMs; + this.maxOffsetMs = maxOffsetMs; + this.minPlaybackSpeed = minPlaybackSpeed; + this.maxPlaybackSpeed = maxPlaybackSpeed; + } + + @Override + public boolean equals(@Nullable Object obj) { + if (this == obj) { + return true; + } + if (!(obj instanceof LiveConfiguration)) { + return false; + } + LiveConfiguration other = (LiveConfiguration) obj; + + return targetOffsetMs == other.targetOffsetMs + && minOffsetMs == other.minOffsetMs + && maxOffsetMs == other.maxOffsetMs + && minPlaybackSpeed == other.minPlaybackSpeed + && maxPlaybackSpeed == other.maxPlaybackSpeed; + } + + @Override + public int hashCode() { + int result = (int) (targetOffsetMs ^ (targetOffsetMs >>> 32)); + result = 31 * result + (int) (minOffsetMs ^ (minOffsetMs >>> 32)); + result = 31 * result + (int) (maxOffsetMs ^ (maxOffsetMs >>> 32)); + result = 31 * result + (minPlaybackSpeed != 0 ? Float.floatToIntBits(minPlaybackSpeed) : 0); + result = 31 * result + (maxPlaybackSpeed != 0 ? Float.floatToIntBits(maxPlaybackSpeed) : 0); + return result; + } + } + /** Properties for a text track. */ public static final class Subtitle { @@ -815,8 +1088,8 @@ public boolean equals(@Nullable Object obj) { @Override public int hashCode() { - int result = Long.valueOf(startPositionMs).hashCode(); - result = 31 * result + Long.valueOf(endPositionMs).hashCode(); + int result = (int) (startPositionMs ^ (startPositionMs >>> 32)); + result = 31 * result + (int) (endPositionMs ^ (endPositionMs >>> 32)); result = 31 * result + (relativeToLiveWindow ? 1 : 0); result = 31 * result + (relativeToDefaultPosition ? 1 : 0); result = 31 * result + (startsAtKeyFrame ? 1 : 0); @@ -827,9 +1100,12 @@ public int hashCode() { /** Identifies the media item. */ public final String mediaId; - /** Optional playback properties. Maybe be {@code null} if shared over process boundaries. */ + /** Optional playback properties. May be {@code null} if shared over process boundaries. */ @Nullable public final PlaybackProperties playbackProperties; + /** The live playback configuration. */ + public final LiveConfiguration liveConfiguration; + /** The media metadata. */ public final MediaMetadata mediaMetadata; @@ -840,9 +1116,11 @@ private MediaItem( String mediaId, ClippingProperties clippingProperties, @Nullable PlaybackProperties playbackProperties, + LiveConfiguration liveConfiguration, MediaMetadata mediaMetadata) { this.mediaId = mediaId; this.playbackProperties = playbackProperties; + this.liveConfiguration = liveConfiguration; this.mediaMetadata = mediaMetadata; this.clippingProperties = clippingProperties; } @@ -866,6 +1144,7 @@ public boolean equals(@Nullable Object obj) { return Util.areEqual(mediaId, other.mediaId) && clippingProperties.equals(other.clippingProperties) && Util.areEqual(playbackProperties, other.playbackProperties) + && Util.areEqual(liveConfiguration, other.liveConfiguration) && Util.areEqual(mediaMetadata, other.mediaMetadata); } @@ -873,6 +1152,7 @@ public boolean equals(@Nullable Object obj) { public int hashCode() { int result = mediaId.hashCode(); result = 31 * result + (playbackProperties != null ? playbackProperties.hashCode() : 0); + result = 31 * result + liveConfiguration.hashCode(); result = 31 * result + clippingProperties.hashCode(); result = 31 * result + mediaMetadata.hashCode(); return result; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/PlaybackParameters.java b/library/common/src/main/java/com/google/android/exoplayer2/PlaybackParameters.java similarity index 91% rename from library/core/src/main/java/com/google/android/exoplayer2/PlaybackParameters.java rename to library/common/src/main/java/com/google/android/exoplayer2/PlaybackParameters.java index 7dcd6f80aa0..ff4f262812c 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/PlaybackParameters.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/PlaybackParameters.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2; +import androidx.annotation.CheckResult; import androidx.annotation.Nullable; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Util; @@ -70,6 +71,17 @@ public long getMediaTimeUsForPlayoutTimeMs(long timeMs) { return timeMs * scaledUsPerMs; } + /** + * Returns a copy with the given speed. + * + * @param speed The new speed. + * @return The copied playback parameters. + */ + @CheckResult + public PlaybackParameters withSpeed(float speed) { + return new PlaybackParameters(speed, pitch); + } + @Override public boolean equals(@Nullable Object obj) { if (this == obj) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/Player.java b/library/common/src/main/java/com/google/android/exoplayer2/Player.java similarity index 81% rename from library/core/src/main/java/com/google/android/exoplayer2/Player.java rename to library/common/src/main/java/com/google/android/exoplayer2/Player.java index 3d1c135a6be..2a92964649d 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/Player.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/Player.java @@ -28,14 +28,14 @@ import com.google.android.exoplayer2.audio.AuxEffectInfo; import com.google.android.exoplayer2.device.DeviceInfo; import com.google.android.exoplayer2.device.DeviceListener; +import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.metadata.MetadataOutput; import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.text.Cue; import com.google.android.exoplayer2.text.TextOutput; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; -import com.google.android.exoplayer2.trackselection.TrackSelector; +import com.google.android.exoplayer2.util.MutableFlags; import com.google.android.exoplayer2.util.Util; -import com.google.android.exoplayer2.video.VideoDecoderOutputBufferRenderer; import com.google.android.exoplayer2.video.VideoFrameMetadataListener; import com.google.android.exoplayer2.video.VideoListener; import com.google.android.exoplayer2.video.spherical.CameraMotionListener; @@ -82,26 +82,6 @@ interface AudioComponent { */ void removeAudioListener(AudioListener listener); - /** - * Sets the attributes for audio playback, used by the underlying audio track. If not set, the - * default audio attributes will be used. They are suitable for general media playback. - * - *

Setting the audio attributes during playback may introduce a short gap in audio output as - * the audio track is recreated. A new audio session id will also be generated. - * - *

If tunneling is enabled by the track selector, the specified audio attributes will be - * ignored, but they will take effect if audio is later played without tunneling. - * - *

If the device is running a build before platform API version 21, audio attributes cannot - * be set directly on the underlying audio track. In this case, the usage will be mapped onto an - * equivalent stream type using {@link Util#getStreamTypeForAudioUsage(int)}. - * - * @param audioAttributes The attributes to use for audio playback. - * @deprecated Use {@link AudioComponent#setAudioAttributes(AudioAttributes, boolean)}. - */ - @Deprecated - void setAudioAttributes(AudioAttributes audioAttributes); - /** * Sets the attributes for audio playback, used by the underlying audio track. If not set, the * default audio attributes will be used. They are suitable for general media playback. @@ -174,14 +154,14 @@ interface AudioComponent { interface VideoComponent { /** - * Sets the {@link Renderer.VideoScalingMode}. + * Sets the {@link C.VideoScalingMode}. * - * @param videoScalingMode The {@link Renderer.VideoScalingMode}. + * @param videoScalingMode The {@link C.VideoScalingMode}. */ - void setVideoScalingMode(@Renderer.VideoScalingMode int videoScalingMode); + void setVideoScalingMode(@C.VideoScalingMode int videoScalingMode); - /** Returns the {@link Renderer.VideoScalingMode}. */ - @Renderer.VideoScalingMode + /** Returns the {@link C.VideoScalingMode}. */ + @C.VideoScalingMode int getVideoScalingMode(); /** @@ -308,30 +288,6 @@ interface VideoComponent { * @param textureView The texture view to clear. */ void clearVideoTextureView(@Nullable TextureView textureView); - - /** - * Sets the video decoder output buffer renderer. This is intended for use only with extension - * renderers that accept {@link Renderer#MSG_SET_VIDEO_DECODER_OUTPUT_BUFFER_RENDERER}. For most - * use cases, an output surface or view should be passed via {@link #setVideoSurface(Surface)} - * or {@link #setVideoSurfaceView(SurfaceView)} instead. - * - * @param videoDecoderOutputBufferRenderer The video decoder output buffer renderer, or {@code - * null} to clear the output buffer renderer. - */ - void setVideoDecoderOutputBufferRenderer( - @Nullable VideoDecoderOutputBufferRenderer videoDecoderOutputBufferRenderer); - - /** Clears the video decoder output buffer renderer. */ - void clearVideoDecoderOutputBufferRenderer(); - - /** - * Clears the video decoder output buffer renderer if it matches the one passed. Else does - * nothing. - * - * @param videoDecoderOutputBufferRenderer The video decoder output buffer renderer to clear. - */ - void clearVideoDecoderOutputBufferRenderer( - @Nullable VideoDecoderOutputBufferRenderer videoDecoderOutputBufferRenderer); } /** The text component of a {@link Player}. */ @@ -420,8 +376,13 @@ interface DeviceComponent { } /** - * Listener of changes in player state. All methods have no-op default implementations to allow - * selective overrides. + * Listener of changes in player state. + * + *

All methods have no-op default implementations to allow selective overrides. + * + *

Listeners can choose to implement individual events (e.g. {@link + * #onIsPlayingChanged(boolean)}) or {@link #onEvents(Player, Events)}, which is called after one + * or more events occurred together. */ interface EventListener { @@ -433,6 +394,9 @@ interface EventListener { * added or removed from the timeline. This will not be reported via a separate call to * {@link #onPositionDiscontinuity(int)}. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param timeline The latest timeline. Never null, but may be empty. * @param reason The {@link TimelineChangeReason} responsible for this timeline change. */ @@ -475,6 +439,9 @@ default void onTimelineChanged( *

Note that this callback is also called when the playlist becomes non-empty or empty as a * consequence of a playlist change. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param mediaItem The {@link MediaItem}. May be null if the playlist becomes empty. * @param reason The reason for the transition. */ @@ -484,6 +451,9 @@ default void onMediaItemTransition( /** * Called when the available or selected tracks change. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param trackGroups The available tracks. Never null, but may be of length zero. * @param trackSelections The track selections for each renderer. Never null and always of * length {@link #getRendererCount()}, but may contain null elements. @@ -491,9 +461,31 @@ default void onMediaItemTransition( default void onTracksChanged( TrackGroupArray trackGroups, TrackSelectionArray trackSelections) {} + /** + * Called when the static metadata changes. + * + *

The provided {@code metadataList} is an immutable list of {@link Metadata} instances, + * where the elements correspond to the {@link #getCurrentTrackSelections() current track + * selections}, or an empty list if there are no track selections or the selected tracks contain + * no static metadata. + * + *

The metadata is considered static in the sense that it comes from the tracks' declared + * Formats, rather than being timed (or dynamic) metadata, which is represented within a + * metadata track. + * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * + * @param metadataList The static metadata. + */ + default void onStaticMetadataChanged(List metadataList) {} + /** * Called when the player starts or stops loading the source. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param isLoading Whether the source is currently being loaded. */ @SuppressWarnings("deprecation") @@ -515,6 +507,9 @@ default void onPlayerStateChanged(boolean playWhenReady, @State int playbackStat /** * Called when the value returned from {@link #getPlaybackState()} changes. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param state The new playback {@link State state}. */ default void onPlaybackStateChanged(@State int state) {} @@ -522,6 +517,9 @@ default void onPlaybackStateChanged(@State int state) {} /** * Called when the value returned from {@link #getPlayWhenReady()} changes. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param playWhenReady Whether playback will proceed when ready. * @param reason The {@link PlayWhenReadyChangeReason reason} for the change. */ @@ -531,6 +529,9 @@ default void onPlayWhenReadyChanged( /** * Called when the value returned from {@link #getPlaybackSuppressionReason()} changes. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param playbackSuppressionReason The current {@link PlaybackSuppressionReason}. */ default void onPlaybackSuppressionReasonChanged( @@ -539,6 +540,9 @@ default void onPlaybackSuppressionReasonChanged( /** * Called when the value of {@link #isPlaying()} changes. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param isPlaying Whether the player is playing. */ default void onIsPlayingChanged(boolean isPlaying) {} @@ -546,6 +550,9 @@ default void onIsPlayingChanged(boolean isPlaying) {} /** * Called when the value of {@link #getRepeatMode()} changes. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param repeatMode The {@link RepeatMode} used for playback. */ default void onRepeatModeChanged(@RepeatMode int repeatMode) {} @@ -553,6 +560,9 @@ default void onRepeatModeChanged(@RepeatMode int repeatMode) {} /** * Called when the value of {@link #getShuffleModeEnabled()} changes. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param shuffleModeEnabled Whether shuffling of windows is enabled. */ default void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) {} @@ -562,6 +572,9 @@ default void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) {} * immediately after this method is called. The player instance can still be used, and {@link * #release()} must still be called on the player should it no longer be required. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param error The error. */ default void onPlayerError(ExoPlaybackException error) {} @@ -576,6 +589,9 @@ default void onPlayerError(ExoPlaybackException error) {} *

When a position discontinuity occurs as a result of a change to the timeline this method * is not called. {@link #onTimelineChanged(Timeline, int)} is called in this case. * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param reason The {@link DiscontinuityReason} responsible for the discontinuity. */ default void onPositionDiscontinuity(@DiscontinuityReason int reason) {} @@ -586,6 +602,9 @@ default void onPositionDiscontinuity(@DiscontinuityReason int reason) {} * them (for example, if audio playback switches to passthrough or offload mode, where speed * adjustment is no longer possible). * + *

{@link #onEvents(Player, Events)} will also be called to report this event along with + * other events that happen in the same {@link Looper} message queue iteration. + * * @param playbackParameters The playback parameters. */ default void onPlaybackParametersChanged(PlaybackParameters playbackParameters) {} @@ -598,12 +617,52 @@ default void onPlaybackParametersChanged(PlaybackParameters playbackParameters) default void onSeekProcessed() {} /** - * Called when the player has started or stopped offload scheduling after a call to {@link + * Called when the player has started or stopped offload scheduling. + * + *

If using ExoPlayer, this is done by calling {@code * ExoPlayer#experimentalSetOffloadSchedulingEnabled(boolean)}. * *

This method is experimental, and will be renamed or removed in a future release. */ + // TODO(b/172315872) Move this method in a new ExoPlayer.EventListener. default void onExperimentalOffloadSchedulingEnabledChanged(boolean offloadSchedulingEnabled) {} + + /** + * Called when the player has started or finished sleeping for offload. + * + *

This method is experimental, and will be renamed or removed in a future release. + */ + default void onExperimentalSleepingForOffloadChanged(boolean sleepingForOffload) {} + + /** + * Called when one or more player states changed. + * + *

State changes and events that happen within one {@link Looper} message queue iteration are + * reported together and only after all individual callbacks were triggered. + * + *

Listeners should prefer this method over individual callbacks in the following cases: + * + *

+ * + * @param player The {@link Player} whose state changed. Use the getters to obtain the latest + * states. + * @param events The {@link Events} that happened in this iteration, indicating which player + * states changed. + */ + default void onEvents(Player player, Events events) {} } /** @@ -626,17 +685,52 @@ public void onTimelineChanged(Timeline timeline, @TimelineChangeReason int reaso } @Override - @SuppressWarnings("deprecation") public void onTimelineChanged( Timeline timeline, @Nullable Object manifest, @TimelineChangeReason int reason) { - // Call deprecated version. Otherwise, do nothing. - onTimelineChanged(timeline, manifest); + // Do nothing. } + } - /** @deprecated Use {@link EventListener#onTimelineChanged(Timeline, int)} instead. */ - @Deprecated - public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { - // Do nothing. + /** A set of {@link EventFlags}. */ + final class Events extends MutableFlags { + /** + * Returns whether the given event occurred. + * + * @param event The {@link EventFlags event}. + * @return Whether the event occurred. + */ + @Override + public boolean contains(@EventFlags int event) { + // Overridden to add IntDef compiler enforcement and new JavaDoc. + return super.contains(event); + } + + /** + * Returns whether any of the given events occurred. + * + * @param events The {@link EventFlags events}. + * @return Whether any of the events occurred. + */ + @Override + public boolean containsAny(@EventFlags int... events) { + // Overridden to add IntDef compiler enforcement and new JavaDoc. + return super.containsAny(events); + } + + /** + * Returns the {@link EventFlags event} at the given index. + * + *

Although index-based access is possible, it doesn't imply a particular order of these + * events. + * + * @param index The index. Must be between 0 (inclusive) and {@link #size()} (exclusive). + * @return The {@link EventFlags event} at the given index. + */ + @Override + @EventFlags + public int get(int index) { + // Overridden to add IntDef compiler enforcement and new JavaDoc. + return super.get(index); } } @@ -648,9 +742,7 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { @Retention(RetentionPolicy.SOURCE) @IntDef({STATE_IDLE, STATE_BUFFERING, STATE_READY, STATE_ENDED}) @interface State {} - /** - * The player does not have any media to play. - */ + /** The player does not have any media to play. */ int STATE_IDLE = 1; /** * The player is not able to immediately play from its current position. This state typically @@ -662,9 +754,7 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { * {@link #getPlayWhenReady()} is true, and paused otherwise. */ int STATE_READY = 3; - /** - * The player has finished playing the media. - */ + /** The player has finished playing the media. */ int STATE_ENDED = 4; /** @@ -785,7 +875,11 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { /** Timeline changed as a result of a dynamic update introduced by the played media. */ int TIMELINE_CHANGE_REASON_SOURCE_UPDATE = 1; - /** Reasons for media item transitions. */ + /** + * Reasons for media item transitions. One of {@link #MEDIA_ITEM_TRANSITION_REASON_REPEAT}, {@link + * #MEDIA_ITEM_TRANSITION_REASON_AUTO}, {@link #MEDIA_ITEM_TRANSITION_REASON_SEEK} or {@link + * #MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED}. + */ @Documented @Retention(RetentionPolicy.SOURCE) @IntDef({ @@ -808,6 +902,59 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { */ int MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED = 3; + /** + * Events that can be reported via {@link EventListener#onEvents(Player, Events)}. + * + *

One of the {@link Player}{@code .EVENT_*} flags. + */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + EVENT_TIMELINE_CHANGED, + EVENT_MEDIA_ITEM_TRANSITION, + EVENT_TRACKS_CHANGED, + EVENT_STATIC_METADATA_CHANGED, + EVENT_IS_LOADING_CHANGED, + EVENT_PLAYBACK_STATE_CHANGED, + EVENT_PLAY_WHEN_READY_CHANGED, + EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED, + EVENT_IS_PLAYING_CHANGED, + EVENT_REPEAT_MODE_CHANGED, + EVENT_SHUFFLE_MODE_ENABLED_CHANGED, + EVENT_PLAYER_ERROR, + EVENT_POSITION_DISCONTINUITY, + EVENT_PLAYBACK_PARAMETERS_CHANGED + }) + @interface EventFlags {} + /** {@link #getCurrentTimeline()} changed. */ + int EVENT_TIMELINE_CHANGED = 0; + /** {@link #getCurrentMediaItem()} changed or the player started repeating the current item. */ + int EVENT_MEDIA_ITEM_TRANSITION = 1; + /** {@link #getCurrentTrackGroups()} or {@link #getCurrentTrackSelections()} changed. */ + int EVENT_TRACKS_CHANGED = 2; + /** {@link #getCurrentStaticMetadata()} changed. */ + int EVENT_STATIC_METADATA_CHANGED = 3; + /** {@link #isLoading()} ()} changed. */ + int EVENT_IS_LOADING_CHANGED = 4; + /** {@link #getPlaybackState()} changed. */ + int EVENT_PLAYBACK_STATE_CHANGED = 5; + /** {@link #getPlayWhenReady()} changed. */ + int EVENT_PLAY_WHEN_READY_CHANGED = 6; + /** {@link #getPlaybackSuppressionReason()} changed. */ + int EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED = 7; + /** {@link #isPlaying()} changed. */ + int EVENT_IS_PLAYING_CHANGED = 8; + /** {@link #getRepeatMode()} changed. */ + int EVENT_REPEAT_MODE_CHANGED = 9; + /** {@link #getShuffleModeEnabled()} changed. */ + int EVENT_SHUFFLE_MODE_ENABLED_CHANGED = 10; + /** {@link #getPlayerError()} changed. */ + int EVENT_PLAYER_ERROR = 11; + /** A position discontinuity occurred. See {@link EventListener#onPositionDiscontinuity(int)}. */ + int EVENT_POSITION_DISCONTINUITY = 12; + /** {@link #getPlaybackParameters()} changed. */ + int EVENT_PLAYBACK_PARAMETERS_CHANGED = 13; + /** Returns the component of this player for audio output, or null if audio is not supported. */ @Nullable AudioComponent getAudioComponent(); @@ -1017,8 +1164,7 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { /** * Returns the error that caused playback to fail. This is the same error that will have been * reported via {@link Player.EventListener#onPlayerError(ExoPlaybackException)} at the time of - * failure. It can be queried using this method until {@code stop(true)} is called or the player - * is re-prepared. + * failure. It can be queried using this method until the player is re-prepared. * *

Note that this method will always return {@code null} if {@link #getPlaybackState()} is not * {@link #STATE_IDLE}. @@ -1070,7 +1216,8 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { * * @return The current repeat mode. */ - @RepeatMode int getRepeatMode(); + @RepeatMode + int getRepeatMode(); /** * Sets whether shuffling of windows is enabled. @@ -1079,9 +1226,7 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { */ void setShuffleModeEnabled(boolean shuffleModeEnabled); - /** - * Returns whether shuffling of windows is enabled. - */ + /** Returns whether shuffling of windows is enabled. */ boolean getShuffleModeEnabled(); /** @@ -1203,16 +1348,11 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { void stop(); /** - * Stops playback and optionally clears the playlist and resets the position and playback error. - * Use {@link #pause()} rather than this method if the intention is to pause playback. - * - *

Calling this method will cause the playback state to transition to {@link #STATE_IDLE}. The - * player instance can still be used, and {@link #release()} must still be called on the player if - * it's no longer required. - * - * @param reset Whether the playlist should be cleared and whether the playback position and - * playback error should be reset. + * @deprecated Use {@link #stop()} and {@link #clearMediaItems()} (if {@code reset} is true) or + * just {@link #stop()} (if {@code reset} is false). Any player error will be cleared when + * {@link #prepare() re-preparing} the player. */ + @Deprecated void stop(boolean reset); /** @@ -1221,35 +1361,38 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { */ void release(); - /** - * Returns the number of renderers. - */ + /** Returns the number of renderers. */ int getRendererCount(); /** * Returns the track type that the renderer at a given index handles. * - * @see Renderer#getTrackType() + *

For example, a video renderer will return {@link C#TRACK_TYPE_VIDEO}, an audio renderer will + * return {@link C#TRACK_TYPE_AUDIO} and a text renderer will return {@link C#TRACK_TYPE_TEXT}. + * * @param index The index of the renderer. * @return One of the {@code TRACK_TYPE_*} constants defined in {@link C}. */ int getRendererType(int index); - /** - * Returns the track selector that this player uses, or null if track selection is not supported. - */ - @Nullable - TrackSelector getTrackSelector(); - - /** - * Returns the available track groups. - */ + /** Returns the available track groups. */ TrackGroupArray getCurrentTrackGroups(); + /** Returns the current track selections for each renderer. */ + TrackSelectionArray getCurrentTrackSelections(); + /** - * Returns the current track selections for each renderer. + * Returns the current static metadata for the track selections. + * + *

The returned {@code metadataList} is an immutable list of {@link Metadata} instances, where + * the elements correspond to the {@link #getCurrentTrackSelections() current track selections}, + * or an empty list if there are no track selections or the selected tracks contain no static + * metadata. + * + *

This metadata is considered static in that it comes from the tracks' declared Formats, + * rather than being timed (or dynamic) metadata, which is represented within a metadata track. */ - TrackSelectionArray getCurrentTrackSelections(); + List getCurrentStaticMetadata(); /** * Returns the current manifest. The type depends on the type of media being played. May be null. @@ -1257,14 +1400,10 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { @Nullable Object getCurrentManifest(); - /** - * Returns the current {@link Timeline}. Never null, but may be empty. - */ + /** Returns the current {@link Timeline}. Never null, but may be empty. */ Timeline getCurrentTimeline(); - /** - * Returns the index of the period currently being played. - */ + /** Returns the index of the period currently being played. */ int getCurrentPeriodIndex(); /** @@ -1359,7 +1498,7 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { /** * Returns whether the current window is live, or {@code false} if the {@link Timeline} is empty. * - * @see Timeline.Window#isLive + * @see Timeline.Window#isLive() */ boolean isCurrentWindowLive(); @@ -1384,9 +1523,7 @@ public void onTimelineChanged(Timeline timeline, @Nullable Object manifest) { */ boolean isCurrentWindowSeekable(); - /** - * Returns whether the player is currently playing an ad. - */ + /** Returns whether the player is currently playing an ad. */ boolean isPlayingAd(); /** diff --git a/library/core/src/main/java/com/google/android/exoplayer2/Timeline.java b/library/common/src/main/java/com/google/android/exoplayer2/Timeline.java similarity index 94% rename from library/core/src/main/java/com/google/android/exoplayer2/Timeline.java rename to library/common/src/main/java/com/google/android/exoplayer2/Timeline.java index 19be5bf9d0e..d7e1e955dbf 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/Timeline.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/Timeline.java @@ -15,6 +15,8 @@ */ package com.google.android.exoplayer2; +import static com.google.android.exoplayer2.util.Assertions.checkState; + import android.net.Uri; import android.os.SystemClock; import android.util.Pair; @@ -74,10 +76,10 @@ *

A timeline for a live stream consists of a period whose duration is unknown, since it's * continually extending as more content is broadcast. If content only remains available for a * limited period of time then the window may start at a non-zero position, defining the region of - * content that can still be played. The window will have {@link Window#isLive} set to true to - * indicate it's a live stream and {@link Window#isDynamic} set to true as long as we expect changes - * to the live window. Its default position is typically near to the live edge (indicated by the - * black dot in the figure above). + * content that can still be played. The window will return true from {@link Window#isLive()} to + * indicate it's a live stream and {@link Window#isDynamic} will be set to true as long as we expect + * changes to the live window. Its default position is typically near to the live edge (indicated by + * the black dot in the figure above). * *

Live stream with indefinite availability

* @@ -191,12 +193,14 @@ public static final class Window { /** Whether this window may change when the timeline is updated. */ public boolean isDynamic; + /** @deprecated Use {@link #isLive()} instead. */ + @Deprecated public boolean isLive; + /** - * Whether the media in this window is live. For informational purposes only. - * - *

Check {@link #isDynamic} to know whether this window may still change. + * The {@link MediaItem.LiveConfiguration} that is used or null if {@link #isLive()} returns + * false. */ - public boolean isLive; + @Nullable public MediaItem.LiveConfiguration liveConfiguration; /** * Whether this window contains placeholder information because the real information has yet to @@ -248,7 +252,7 @@ public Window set( long elapsedRealtimeEpochOffsetMs, boolean isSeekable, boolean isDynamic, - boolean isLive, + @Nullable MediaItem.LiveConfiguration liveConfiguration, long defaultPositionUs, long durationUs, int firstPeriodIndex, @@ -266,7 +270,8 @@ public Window set( this.elapsedRealtimeEpochOffsetMs = elapsedRealtimeEpochOffsetMs; this.isSeekable = isSeekable; this.isDynamic = isDynamic; - this.isLive = isLive; + this.isLive = liveConfiguration != null; + this.liveConfiguration = liveConfiguration; this.defaultPositionUs = defaultPositionUs; this.durationUs = durationUs; this.firstPeriodIndex = firstPeriodIndex; @@ -336,6 +341,14 @@ public long getCurrentUnixTimeMs() { return Util.getNowUnixTimeMs(elapsedRealtimeEpochOffsetMs); } + /** Returns whether this is a live stream. */ + // Verifies whether the deprecated isLive member field is in a correct state. + @SuppressWarnings("deprecation") + public boolean isLive() { + checkState(isLive == (liveConfiguration != null)); + return liveConfiguration != null; + } + // Provide backward compatibility for tag. @Override public boolean equals(@Nullable Object obj) { @@ -349,12 +362,12 @@ public boolean equals(@Nullable Object obj) { return Util.areEqual(uid, that.uid) && Util.areEqual(mediaItem, that.mediaItem) && Util.areEqual(manifest, that.manifest) + && Util.areEqual(liveConfiguration, that.liveConfiguration) && presentationStartTimeMs == that.presentationStartTimeMs && windowStartTimeMs == that.windowStartTimeMs && elapsedRealtimeEpochOffsetMs == that.elapsedRealtimeEpochOffsetMs && isSeekable == that.isSeekable && isDynamic == that.isDynamic - && isLive == that.isLive && isPlaceholder == that.isPlaceholder && defaultPositionUs == that.defaultPositionUs && durationUs == that.durationUs @@ -370,6 +383,7 @@ public int hashCode() { result = 31 * result + uid.hashCode(); result = 31 * result + mediaItem.hashCode(); result = 31 * result + (manifest == null ? 0 : manifest.hashCode()); + result = 31 * result + (liveConfiguration == null ? 0 : liveConfiguration.hashCode()); result = 31 * result + (int) (presentationStartTimeMs ^ (presentationStartTimeMs >>> 32)); result = 31 * result + (int) (windowStartTimeMs ^ (windowStartTimeMs >>> 32)); result = @@ -377,7 +391,6 @@ public int hashCode() { + (int) (elapsedRealtimeEpochOffsetMs ^ (elapsedRealtimeEpochOffsetMs >>> 32)); result = 31 * result + (isSeekable ? 1 : 0); result = 31 * result + (isDynamic ? 1 : 0); - result = 31 * result + (isLive ? 1 : 0); result = 31 * result + (isPlaceholder ? 1 : 0); result = 31 * result + (int) (defaultPositionUs ^ (defaultPositionUs >>> 32)); result = 31 * result + (int) (durationUs ^ (durationUs >>> 32)); @@ -519,9 +532,13 @@ public long getPositionInWindowUs() { return positionInWindowUs; } - /** - * Returns the number of ad groups in the period. - */ + /** Returns the opaque identifier for ads played with this period, or {@code null} if unset. */ + @Nullable + public Object getAdsId() { + return adPlaybackState.adsId; + } + + /** Returns the number of ad groups in the period. */ public int getAdGroupCount() { return adPlaybackState.adGroupCount; } @@ -716,8 +733,8 @@ public final boolean isEmpty() { * @param shuffleModeEnabled Whether shuffling is enabled. * @return The index of the next window, or {@link C#INDEX_UNSET} if this is the last window. */ - public int getNextWindowIndex(int windowIndex, @Player.RepeatMode int repeatMode, - boolean shuffleModeEnabled) { + public int getNextWindowIndex( + int windowIndex, @Player.RepeatMode int repeatMode, boolean shuffleModeEnabled) { switch (repeatMode) { case Player.REPEAT_MODE_OFF: return windowIndex == getLastWindowIndex(shuffleModeEnabled) ? C.INDEX_UNSET @@ -741,8 +758,8 @@ public int getNextWindowIndex(int windowIndex, @Player.RepeatMode int repeatMode * @param shuffleModeEnabled Whether shuffling is enabled. * @return The index of the previous window, or {@link C#INDEX_UNSET} if this is the first window. */ - public int getPreviousWindowIndex(int windowIndex, @Player.RepeatMode int repeatMode, - boolean shuffleModeEnabled) { + public int getPreviousWindowIndex( + int windowIndex, @Player.RepeatMode int repeatMode, boolean shuffleModeEnabled) { switch (repeatMode) { case Player.REPEAT_MODE_OFF: return windowIndex == getFirstWindowIndex(shuffleModeEnabled) ? C.INDEX_UNSET @@ -826,8 +843,12 @@ public abstract Window getWindow( * @param shuffleModeEnabled Whether shuffling is enabled. * @return The index of the next period, or {@link C#INDEX_UNSET} if this is the last period. */ - public final int getNextPeriodIndex(int periodIndex, Period period, Window window, - @Player.RepeatMode int repeatMode, boolean shuffleModeEnabled) { + public final int getNextPeriodIndex( + int periodIndex, + Period period, + Window window, + @Player.RepeatMode int repeatMode, + boolean shuffleModeEnabled) { int windowIndex = getPeriod(periodIndex, period).windowIndex; if (getWindow(windowIndex, window).lastPeriodIndex == periodIndex) { int nextWindowIndex = getNextWindowIndex(windowIndex, repeatMode, shuffleModeEnabled); @@ -840,8 +861,8 @@ public final int getNextPeriodIndex(int periodIndex, Period period, Window windo } /** - * Returns whether the given period is the last period of the timeline depending on the - * {@code repeatMode} and whether shuffling is enabled. + * Returns whether the given period is the last period of the timeline depending on the {@code + * repeatMode} and whether shuffling is enabled. * * @param periodIndex A period index. * @param period A {@link Period} to be used internally. Must not be null. @@ -850,8 +871,12 @@ public final int getNextPeriodIndex(int periodIndex, Period period, Window windo * @param shuffleModeEnabled Whether shuffling is enabled. * @return Whether the period of the given index is the last period of the timeline. */ - public final boolean isLastPeriod(int periodIndex, Period period, Window window, - @Player.RepeatMode int repeatMode, boolean shuffleModeEnabled) { + public final boolean isLastPeriod( + int periodIndex, + Period period, + Window window, + @Player.RepeatMode int repeatMode, + boolean shuffleModeEnabled) { return getNextPeriodIndex(periodIndex, period, window, repeatMode, shuffleModeEnabled) == C.INDEX_UNSET; } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioListener.java b/library/common/src/main/java/com/google/android/exoplayer2/audio/AudioListener.java similarity index 89% rename from library/core/src/main/java/com/google/android/exoplayer2/audio/AudioListener.java rename to library/common/src/main/java/com/google/android/exoplayer2/audio/AudioListener.java index f208f602e1e..1abe6b2f3c1 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioListener.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/audio/AudioListener.java @@ -19,11 +19,11 @@ public interface AudioListener { /** - * Called when the audio session is set. + * Called when the audio session ID changes. * - * @param audioSessionId The audio session id. + * @param audioSessionId The audio session ID. */ - default void onAudioSessionId(int audioSessionId) {} + default void onAudioSessionIdChanged(int audioSessionId) {} /** * Called when the audio attributes change. diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/AuxEffectInfo.java b/library/common/src/main/java/com/google/android/exoplayer2/audio/AuxEffectInfo.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/audio/AuxEffectInfo.java rename to library/common/src/main/java/com/google/android/exoplayer2/audio/AuxEffectInfo.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/decoder/DecoderInputBuffer.java b/library/common/src/main/java/com/google/android/exoplayer2/decoder/DecoderInputBuffer.java index 0ae8ce31f9b..4d00ee77481 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/decoder/DecoderInputBuffer.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/decoder/DecoderInputBuffer.java @@ -29,6 +29,31 @@ */ public class DecoderInputBuffer extends Buffer { + /** + * Thrown when an attempt is made to write into a {@link DecoderInputBuffer} whose {@link + * #bufferReplacementMode} is {@link #BUFFER_REPLACEMENT_MODE_DISABLED} and who {@link #data} + * capacity is smaller than required. + */ + public static final class InsufficientCapacityException extends IllegalStateException { + + /** The current capacity of the buffer. */ + public final int currentCapacity; + /** The required capacity of the buffer. */ + public final int requiredCapacity; + + /** + * Creates an instance. + * + * @param currentCapacity The current capacity of the buffer. + * @param requiredCapacity The required capacity of the buffer. + */ + public InsufficientCapacityException(int currentCapacity, int requiredCapacity) { + super("Buffer too small (" + currentCapacity + " < " + requiredCapacity + ")"); + this.currentCapacity = currentCapacity; + this.requiredCapacity = requiredCapacity; + } + } + /** * The buffer replacement mode. This controls how {@link #ensureSpaceForWrite} generates * replacement buffers when the capacity of the existing buffer is insufficient. One of {@link @@ -144,8 +169,8 @@ public void resetSupplementalData(int length) { * whose capacity is sufficient. Data up to the current position is copied to the new buffer. * * @param length The length of the write that must be accommodated, in bytes. - * @throws IllegalStateException If there is insufficient capacity to accommodate the write and - * the buffer replacement mode of the holder is {@link #BUFFER_REPLACEMENT_MODE_DISABLED}. + * @throws InsufficientCapacityException If there is insufficient capacity to accommodate the + * write and {@link #bufferReplacementMode} is {@link #BUFFER_REPLACEMENT_MODE_DISABLED}. */ @EnsuresNonNull("data") public void ensureSpaceForWrite(int length) { @@ -223,8 +248,7 @@ private ByteBuffer createReplacementByteBuffer(int requiredCapacity) { return ByteBuffer.allocateDirect(requiredCapacity); } else { int currentCapacity = data == null ? 0 : data.capacity(); - throw new IllegalStateException("Buffer too small (" + currentCapacity + " < " - + requiredCapacity + ")"); + throw new InsufficientCapacityException(currentCapacity, requiredCapacity); } } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/device/DeviceListener.java b/library/common/src/main/java/com/google/android/exoplayer2/device/DeviceListener.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/device/DeviceListener.java rename to library/common/src/main/java/com/google/android/exoplayer2/device/DeviceListener.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/drm/DrmInitData.java b/library/common/src/main/java/com/google/android/exoplayer2/drm/DrmInitData.java index ee09838b0ad..bc2b8bba868 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/drm/DrmInitData.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/drm/DrmInitData.java @@ -137,31 +137,12 @@ private DrmInitData(@Nullable String schemeType, boolean cloneSchemeDatas, Arrays.sort(this.schemeDatas, this); } - /* package */ - DrmInitData(Parcel in) { + /* package */ DrmInitData(Parcel in) { schemeType = in.readString(); schemeDatas = Util.castNonNull(in.createTypedArray(SchemeData.CREATOR)); schemeDataCount = schemeDatas.length; } - /** - * Retrieves data for a given DRM scheme, specified by its UUID. - * - * @deprecated Use {@link #get(int)} and {@link SchemeData#matches(UUID)} instead. - * @param uuid The DRM scheme's UUID. - * @return The initialization data for the scheme, or null if the scheme is not supported. - */ - @Deprecated - @Nullable - public SchemeData get(UUID uuid) { - for (SchemeData schemeData : schemeDatas) { - if (schemeData.matches(uuid)) { - return schemeData; - } - } - return null; - } - /** * Retrieves the {@link SchemeData} at a given index. * diff --git a/library/common/src/main/java/com/google/android/exoplayer2/drm/ExoMediaCrypto.java b/library/common/src/main/java/com/google/android/exoplayer2/drm/ExoMediaCrypto.java index feba7eaaf40..27730c14b39 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/drm/ExoMediaCrypto.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/drm/ExoMediaCrypto.java @@ -15,5 +15,5 @@ */ package com.google.android.exoplayer2.drm; -/** An opaque {@link android.media.MediaCrypto} equivalent. */ +/** Enables decoding of encrypted data using keys in a DRM session. */ public interface ExoMediaCrypto {} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/metadata/MetadataOutput.java b/library/common/src/main/java/com/google/android/exoplayer2/metadata/MetadataOutput.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/metadata/MetadataOutput.java rename to library/common/src/main/java/com/google/android/exoplayer2/metadata/MetadataOutput.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/metadata/id3/ChapterTocFrame.java b/library/common/src/main/java/com/google/android/exoplayer2/metadata/id3/ChapterTocFrame.java index 5d454e84ac5..c8aa9bd9ad7 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/metadata/id3/ChapterTocFrame.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/metadata/id3/ChapterTocFrame.java @@ -45,8 +45,7 @@ public ChapterTocFrame(String elementId, boolean isRoot, boolean isOrdered, Stri this.subFrames = subFrames; } - /* package */ - ChapterTocFrame(Parcel in) { + /* package */ ChapterTocFrame(Parcel in) { super(ID); this.elementId = castNonNull(in.readString()); this.isRoot = in.readByte() != 0; diff --git a/library/common/src/main/java/com/google/android/exoplayer2/metadata/id3/MlltFrame.java b/library/common/src/main/java/com/google/android/exoplayer2/metadata/id3/MlltFrame.java index 0cdd2e038e9..c5f05427950 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/metadata/id3/MlltFrame.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/metadata/id3/MlltFrame.java @@ -45,8 +45,7 @@ public MlltFrame( this.millisecondsDeviations = millisecondsDeviations; } - /* package */ - MlltFrame(Parcel in) { + /* package */ MlltFrame(Parcel in) { super(ID); this.mpegFramesBetweenReference = in.readInt(); this.bytesBetweenReference = in.readInt(); diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/MdtaMetadataEntry.java b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/MdtaMetadataEntry.java similarity index 94% rename from library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/MdtaMetadataEntry.java rename to library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/MdtaMetadataEntry.java index 5ad2b63b4c9..5b2db4945c3 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/MdtaMetadataEntry.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/MdtaMetadataEntry.java @@ -13,7 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.google.android.exoplayer2.extractor.mp4; +package com.google.android.exoplayer2.metadata.mp4; import android.os.Parcel; import android.os.Parcelable; @@ -28,6 +28,9 @@ */ public final class MdtaMetadataEntry implements Metadata.Entry { + /** Key for the capture frame rate (in frames per second). */ + public static final String KEY_ANDROID_CAPTURE_FPS = "com.android.capture.fps"; + /** The metadata key name. */ public final String key; /** The payload. The interpretation of the value depends on {@link #typeIndicator}. */ diff --git a/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/MotionPhotoMetadata.java b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/MotionPhotoMetadata.java new file mode 100644 index 00000000000..b547ad67f6f --- /dev/null +++ b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/MotionPhotoMetadata.java @@ -0,0 +1,134 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.metadata.mp4; + +import android.os.Parcel; +import android.os.Parcelable; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.common.primitives.Longs; + +/** Metadata of a motion photo file. */ +public final class MotionPhotoMetadata implements Metadata.Entry { + + /** The start offset of the photo data, in bytes. */ + public final long photoStartPosition; + /** The size of the photo data, in bytes. */ + public final long photoSize; + /** + * The presentation timestamp of the photo, in microseconds, or {@link C#TIME_UNSET} if unknown. + */ + public final long photoPresentationTimestampUs; + /** The start offset of the video data, in bytes. */ + public final long videoStartPosition; + /** The size of the video data, in bytes. */ + public final long videoSize; + + /** Creates an instance. */ + public MotionPhotoMetadata( + long photoStartPosition, + long photoSize, + long photoPresentationTimestampUs, + long videoStartPosition, + long videoSize) { + this.photoStartPosition = photoStartPosition; + this.photoSize = photoSize; + this.photoPresentationTimestampUs = photoPresentationTimestampUs; + this.videoStartPosition = videoStartPosition; + this.videoSize = videoSize; + } + + private MotionPhotoMetadata(Parcel in) { + photoStartPosition = in.readLong(); + photoSize = in.readLong(); + photoPresentationTimestampUs = in.readLong(); + videoStartPosition = in.readLong(); + videoSize = in.readLong(); + } + + @Override + public boolean equals(@Nullable Object obj) { + if (this == obj) { + return true; + } + if (obj == null || getClass() != obj.getClass()) { + return false; + } + MotionPhotoMetadata other = (MotionPhotoMetadata) obj; + return photoStartPosition == other.photoStartPosition + && photoSize == other.photoSize + && photoPresentationTimestampUs == other.photoPresentationTimestampUs + && videoStartPosition == other.videoStartPosition + && videoSize == other.videoSize; + } + + @Override + public int hashCode() { + int result = 17; + result = 31 * result + Longs.hashCode(photoStartPosition); + result = 31 * result + Longs.hashCode(photoSize); + result = 31 * result + Longs.hashCode(photoPresentationTimestampUs); + result = 31 * result + Longs.hashCode(videoStartPosition); + result = 31 * result + Longs.hashCode(videoSize); + return result; + } + + @Override + public String toString() { + return "Motion photo metadata: photoStartPosition=" + + photoStartPosition + + ", photoSize=" + + photoSize + + ", photoPresentationTimestampUs=" + + photoPresentationTimestampUs + + ", videoStartPosition=" + + videoStartPosition + + ", videoSize=" + + videoSize; + } + + // Parcelable implementation. + + @Override + public void writeToParcel(Parcel dest, int flags) { + dest.writeLong(photoStartPosition); + dest.writeLong(photoSize); + dest.writeLong(photoPresentationTimestampUs); + dest.writeLong(videoStartPosition); + dest.writeLong(videoSize); + } + + @Override + public int describeContents() { + return 0; + } + + public static final Parcelable.Creator CREATOR = + new Parcelable.Creator() { + + @Override + public MotionPhotoMetadata createFromParcel(Parcel in) { + return new MotionPhotoMetadata(in); + } + + @Override + public MotionPhotoMetadata[] newArray(int size) { + return new MotionPhotoMetadata[size]; + } + }; +} diff --git a/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/SlowMotionData.java b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/SlowMotionData.java new file mode 100644 index 00000000000..ae8698b66af --- /dev/null +++ b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/SlowMotionData.java @@ -0,0 +1,201 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.metadata.mp4; + +import static com.google.android.exoplayer2.util.Assertions.checkArgument; + +import android.os.Parcel; +import android.os.Parcelable; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.util.Util; +import com.google.common.base.Objects; +import com.google.common.collect.ComparisonChain; +import java.util.ArrayList; +import java.util.Comparator; +import java.util.List; + +/** Holds information about the segments of slow motion playback within a track. */ +public final class SlowMotionData implements Metadata.Entry { + + /** Holds information about a single segment of slow motion playback within a track. */ + public static final class Segment implements Parcelable { + + public static final Comparator BY_START_THEN_END_THEN_DIVISOR = + (s1, s2) -> + ComparisonChain.start() + .compare(s1.startTimeMs, s2.startTimeMs) + .compare(s1.endTimeMs, s2.endTimeMs) + .compare(s1.speedDivisor, s2.speedDivisor) + .result(); + + /** The start time, in milliseconds, of the track segment that is intended to be slow motion. */ + public final long startTimeMs; + /** The end time, in milliseconds, of the track segment that is intended to be slow motion. */ + public final long endTimeMs; + /** + * The speed reduction factor. + * + *

For example, 4 would mean the segment should be played at a quarter (1/4) of the normal + * speed. + */ + public final int speedDivisor; + + /** + * Creates an instance. + * + * @param startTimeMs See {@link #startTimeMs}. Must be less than endTimeMs. + * @param endTimeMs See {@link #endTimeMs}. + * @param speedDivisor See {@link #speedDivisor}. + */ + public Segment(long startTimeMs, long endTimeMs, int speedDivisor) { + checkArgument(startTimeMs < endTimeMs); + this.startTimeMs = startTimeMs; + this.endTimeMs = endTimeMs; + this.speedDivisor = speedDivisor; + } + + @Override + public String toString() { + return Util.formatInvariant( + "Segment: startTimeMs=%d, endTimeMs=%d, speedDivisor=%d", + startTimeMs, endTimeMs, speedDivisor); + } + + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + Segment segment = (Segment) o; + return startTimeMs == segment.startTimeMs + && endTimeMs == segment.endTimeMs + && speedDivisor == segment.speedDivisor; + } + + @Override + public int hashCode() { + return Objects.hashCode(startTimeMs, endTimeMs, speedDivisor); + } + + @Override + public int describeContents() { + return 0; + } + + @Override + public void writeToParcel(Parcel dest, int flags) { + dest.writeLong(startTimeMs); + dest.writeLong(endTimeMs); + dest.writeInt(speedDivisor); + } + + public static final Creator CREATOR = + new Creator() { + + @Override + public Segment createFromParcel(Parcel in) { + long startTimeMs = in.readLong(); + long endTimeMs = in.readLong(); + int speedDivisor = in.readInt(); + return new Segment(startTimeMs, endTimeMs, speedDivisor); + } + + @Override + public Segment[] newArray(int size) { + return new Segment[size]; + } + }; + } + + public final List segments; + + /** + * Creates an instance with a list of {@link Segment}s. + * + *

The segments must not overlap, that is that the start time of a segment can not be between + * the start and end time of another segment. + */ + public SlowMotionData(List segments) { + this.segments = segments; + checkArgument(!doSegmentsOverlap(segments)); + } + + @Override + public String toString() { + return "SlowMotion: segments=" + segments; + } + + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + SlowMotionData that = (SlowMotionData) o; + return segments.equals(that.segments); + } + + @Override + public int hashCode() { + return segments.hashCode(); + } + + @Override + public int describeContents() { + return 0; + } + + @Override + public void writeToParcel(Parcel dest, int flags) { + dest.writeList(segments); + } + + public static final Creator CREATOR = + new Creator() { + @Override + public SlowMotionData createFromParcel(Parcel in) { + List slowMotionSegments = new ArrayList<>(); + in.readList(slowMotionSegments, Segment.class.getClassLoader()); + return new SlowMotionData(slowMotionSegments); + } + + @Override + public SlowMotionData[] newArray(int size) { + return new SlowMotionData[size]; + } + }; + + private static boolean doSegmentsOverlap(List segments) { + if (segments.isEmpty()) { + return false; + } + long previousEndTimeMs = segments.get(0).endTimeMs; + for (int i = 1; i < segments.size(); i++) { + if (segments.get(i).startTimeMs < previousEndTimeMs) { + return true; + } + previousEndTimeMs = segments.get(i).endTimeMs; + } + + return false; + } +} diff --git a/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/SmtaMetadataEntry.java b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/SmtaMetadataEntry.java new file mode 100644 index 00000000000..6654a9dbb61 --- /dev/null +++ b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/SmtaMetadataEntry.java @@ -0,0 +1,107 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.metadata.mp4; + +import android.os.Parcel; +import android.os.Parcelable; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.common.primitives.Floats; + +/** + * Stores metadata from the Samsung smta box. + * + *

See [Internal: b/150138465#comment76]. + */ +public final class SmtaMetadataEntry implements Metadata.Entry { + + /** + * The capture frame rate, in fps, or {@link C#RATE_UNSET} if it is unknown. + * + *

If known, the capture frame rate should always be an integer value. + */ + public final float captureFrameRate; + /** The number of layers in the SVC extended frames. */ + public final int svcTemporalLayerCount; + + /** Creates an instance. */ + public SmtaMetadataEntry(float captureFrameRate, int svcTemporalLayerCount) { + this.captureFrameRate = captureFrameRate; + this.svcTemporalLayerCount = svcTemporalLayerCount; + } + + private SmtaMetadataEntry(Parcel in) { + captureFrameRate = in.readFloat(); + svcTemporalLayerCount = in.readInt(); + } + + @Override + public boolean equals(@Nullable Object obj) { + if (this == obj) { + return true; + } + if (obj == null || getClass() != obj.getClass()) { + return false; + } + SmtaMetadataEntry other = (SmtaMetadataEntry) obj; + return captureFrameRate == other.captureFrameRate + && svcTemporalLayerCount == other.svcTemporalLayerCount; + } + + @Override + public int hashCode() { + int result = 17; + result = 31 * result + Floats.hashCode(captureFrameRate); + result = 31 * result + svcTemporalLayerCount; + return result; + } + + @Override + public String toString() { + return "smta: captureFrameRate=" + + captureFrameRate + + ", svcTemporalLayerCount=" + + svcTemporalLayerCount; + } + + // Parcelable implementation. + + @Override + public void writeToParcel(Parcel dest, int flags) { + dest.writeFloat(captureFrameRate); + dest.writeInt(svcTemporalLayerCount); + } + + @Override + public int describeContents() { + return 0; + } + + public static final Parcelable.Creator CREATOR = + new Parcelable.Creator() { + + @Override + public SmtaMetadataEntry createFromParcel(Parcel in) { + return new SmtaMetadataEntry(in); + } + + @Override + public SmtaMetadataEntry[] newArray(int size) { + return new SmtaMetadataEntry[size]; + } + }; +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/AdaptiveMediaSourceEventListener.java b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/package-info.java similarity index 62% rename from library/core/src/main/java/com/google/android/exoplayer2/source/AdaptiveMediaSourceEventListener.java rename to library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/package-info.java index ccc3beac551..8ddf4040c14 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/AdaptiveMediaSourceEventListener.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/metadata/mp4/package-info.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2016 The Android Open Source Project + * Copyright 2020 The Android Open Source Project * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -13,12 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.google.android.exoplayer2.source; +@NonNullApi +package com.google.android.exoplayer2.metadata.mp4; -/** - * Interface for callbacks to be notified of {@link MediaSource} events. - * - * @deprecated Use {@link MediaSourceEventListener}. - */ -@Deprecated -public interface AdaptiveMediaSourceEventListener extends MediaSourceEventListener {} +import com.google.android.exoplayer2.util.NonNullApi; diff --git a/library/common/src/main/java/com/google/android/exoplayer2/source/MediaPeriodId.java b/library/common/src/main/java/com/google/android/exoplayer2/source/MediaPeriodId.java new file mode 100644 index 00000000000..ad0e289ba90 --- /dev/null +++ b/library/common/src/main/java/com/google/android/exoplayer2/source/MediaPeriodId.java @@ -0,0 +1,187 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source; + +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Timeline; + +/** + * Identifies a specific playback of a {@link Timeline.Period}. + * + *

A {@link Timeline.Period} can be played multiple times, for example if it is repeated. Each + * instances of this class identifies a specific playback of a {@link Timeline.Period}. + * + *

In ExoPlayer's implementation, {@link MediaPeriodId} identifies a {@code MediaPeriod}. + */ +// TODO(b/172315872) Should be final, but subclassed in MediaSource for backward-compatibility. +public class MediaPeriodId { + + /** The unique id of the timeline period. */ + public final Object periodUid; + + /** + * If the media period is in an ad group, the index of the ad group in the period. {@link + * C#INDEX_UNSET} otherwise. + */ + public final int adGroupIndex; + + /** + * If the media period is in an ad group, the index of the ad in its ad group in the period. + * {@link C#INDEX_UNSET} otherwise. + */ + public final int adIndexInAdGroup; + + /** + * The sequence number of the window in the buffered sequence of windows this media period is part + * of. {@link C#INDEX_UNSET} if the media period id is not part of a buffered sequence of windows. + */ + public final long windowSequenceNumber; + + /** + * The index of the next ad group to which the media period's content is clipped, or {@link + * C#INDEX_UNSET} if there is no following ad group or if this media period is an ad. + */ + public final int nextAdGroupIndex; + + /** + * Creates a media period identifier for a period which is not part of a buffered sequence of + * windows. + * + * @param periodUid The unique id of the timeline period. + */ + public MediaPeriodId(Object periodUid) { + this(periodUid, /* windowSequenceNumber= */ C.INDEX_UNSET); + } + + /** + * Creates a media period identifier for the specified period in the timeline. + * + * @param periodUid The unique id of the timeline period. + * @param windowSequenceNumber The sequence number of the window in the buffered sequence of + * windows this media period is part of. + */ + public MediaPeriodId(Object periodUid, long windowSequenceNumber) { + this( + periodUid, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET, + windowSequenceNumber, + /* nextAdGroupIndex= */ C.INDEX_UNSET); + } + + /** + * Creates a media period identifier for the specified clipped period in the timeline. + * + * @param periodUid The unique id of the timeline period. + * @param windowSequenceNumber The sequence number of the window in the buffered sequence of + * windows this media period is part of. + * @param nextAdGroupIndex The index of the next ad group to which the media period's content is + * clipped. + */ + public MediaPeriodId(Object periodUid, long windowSequenceNumber, int nextAdGroupIndex) { + this( + periodUid, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET, + windowSequenceNumber, + nextAdGroupIndex); + } + + /** + * Creates a media period identifier that identifies an ad within an ad group at the specified + * timeline period. + * + * @param periodUid The unique id of the timeline period that contains the ad group. + * @param adGroupIndex The index of the ad group. + * @param adIndexInAdGroup The index of the ad in the ad group. + * @param windowSequenceNumber The sequence number of the window in the buffered sequence of + * windows this media period is part of. + */ + public MediaPeriodId( + Object periodUid, int adGroupIndex, int adIndexInAdGroup, long windowSequenceNumber) { + this( + periodUid, + adGroupIndex, + adIndexInAdGroup, + windowSequenceNumber, + /* nextAdGroupIndex= */ C.INDEX_UNSET); + } + + /** Copy constructor for inheritance. */ + // TODO(b/172315872) Delete when client have migrated from MediaSource.MediaPeriodId + protected MediaPeriodId(MediaPeriodId mediaPeriodId) { + this.periodUid = mediaPeriodId.periodUid; + this.adGroupIndex = mediaPeriodId.adGroupIndex; + this.adIndexInAdGroup = mediaPeriodId.adIndexInAdGroup; + this.windowSequenceNumber = mediaPeriodId.windowSequenceNumber; + this.nextAdGroupIndex = mediaPeriodId.nextAdGroupIndex; + } + + private MediaPeriodId( + Object periodUid, + int adGroupIndex, + int adIndexInAdGroup, + long windowSequenceNumber, + int nextAdGroupIndex) { + this.periodUid = periodUid; + this.adGroupIndex = adGroupIndex; + this.adIndexInAdGroup = adIndexInAdGroup; + this.windowSequenceNumber = windowSequenceNumber; + this.nextAdGroupIndex = nextAdGroupIndex; + } + + /** Returns a copy of this period identifier but with {@code newPeriodUid} as its period uid. */ + public MediaPeriodId copyWithPeriodUid(Object newPeriodUid) { + return periodUid.equals(newPeriodUid) + ? this + : new MediaPeriodId( + newPeriodUid, adGroupIndex, adIndexInAdGroup, windowSequenceNumber, nextAdGroupIndex); + } + + /** Returns whether this period identifier identifies an ad in an ad group in a period. */ + public boolean isAd() { + return adGroupIndex != C.INDEX_UNSET; + } + + @Override + public boolean equals(@Nullable Object obj) { + if (this == obj) { + return true; + } + if (!(obj instanceof MediaPeriodId)) { + return false; + } + + MediaPeriodId periodId = (MediaPeriodId) obj; + return periodUid.equals(periodId.periodUid) + && adGroupIndex == periodId.adGroupIndex + && adIndexInAdGroup == periodId.adIndexInAdGroup + && windowSequenceNumber == periodId.windowSequenceNumber + && nextAdGroupIndex == periodId.nextAdGroupIndex; + } + + @Override + public int hashCode() { + int result = 17; + result = 31 * result + periodUid.hashCode(); + result = 31 * result + adGroupIndex; + result = 31 * result + adIndexInAdGroup; + result = 31 * result + (int) windowSequenceNumber; + result = 31 * result + nextAdGroupIndex; + return result; + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/TrackGroup.java b/library/common/src/main/java/com/google/android/exoplayer2/source/TrackGroup.java similarity index 86% rename from library/core/src/main/java/com/google/android/exoplayer2/source/TrackGroup.java rename to library/common/src/main/java/com/google/android/exoplayer2/source/TrackGroup.java index 9e837bf05d4..607f7971038 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/TrackGroup.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/source/TrackGroup.java @@ -23,20 +23,10 @@ import com.google.android.exoplayer2.util.Assertions; import java.util.Arrays; -// TODO: Add an allowMultipleStreams boolean to indicate where the one stream per group restriction -// does not apply. -/** - * Defines a group of tracks exposed by a {@link MediaPeriod}. - * - *

A {@link MediaPeriod} is only able to provide one {@link SampleStream} corresponding to a - * group at any given time, however this {@link SampleStream} may adapt between multiple tracks - * within the group. - */ +/** Defines an immutable group of tracks identified by their format identity. */ public final class TrackGroup implements Parcelable { - /** - * The number of tracks in the group. - */ + /** The number of tracks in the group. */ public final int length; private final Format[] formats; @@ -45,7 +35,7 @@ public final class TrackGroup implements Parcelable { private int hashCode; /** - * @param formats The track formats. Must not be null, contain null elements or be of length 0. + * @param formats The track formats. At least one {@link Format} must be provided. */ public TrackGroup(Format... formats) { Assertions.checkState(formats.length > 0); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/TrackGroupArray.java b/library/common/src/main/java/com/google/android/exoplayer2/source/TrackGroupArray.java similarity index 90% rename from library/core/src/main/java/com/google/android/exoplayer2/source/TrackGroupArray.java rename to library/common/src/main/java/com/google/android/exoplayer2/source/TrackGroupArray.java index e737a5fafa4..8db7b9c3856 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/TrackGroupArray.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/source/TrackGroupArray.java @@ -21,17 +21,13 @@ import com.google.android.exoplayer2.C; import java.util.Arrays; -/** An array of {@link TrackGroup}s exposed by a {@link MediaPeriod}. */ +/** An immutable array of {@link TrackGroup}s. */ public final class TrackGroupArray implements Parcelable { - /** - * The empty array. - */ + /** The empty array. */ public static final TrackGroupArray EMPTY = new TrackGroupArray(); - /** - * The number of groups in the array. Greater than or equal to zero. - */ + /** The number of groups in the array. Greater than or equal to zero. */ public final int length; private final TrackGroup[] trackGroups; @@ -39,9 +35,7 @@ public final class TrackGroupArray implements Parcelable { // Lazily initialized hashcode. private int hashCode; - /** - * @param trackGroups The groups. Must not be null or contain null elements, but may be empty. - */ + /** @param trackGroups The groups. May be empty. */ public TrackGroupArray(TrackGroup... trackGroups) { this.trackGroups = trackGroups; this.length = trackGroups.length; @@ -83,9 +77,7 @@ public int indexOf(TrackGroup group) { return C.INDEX_UNSET; } - /** - * Returns whether this track group array is empty. - */ + /** Returns whether this track group array is empty. */ public boolean isEmpty() { return length == 0; } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdPlaybackState.java b/library/common/src/main/java/com/google/android/exoplayer2/source/ads/AdPlaybackState.java similarity index 89% rename from library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdPlaybackState.java rename to library/common/src/main/java/com/google/android/exoplayer2/source/ads/AdPlaybackState.java index 9493746669c..a50fcd7d1dc 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdPlaybackState.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/source/ads/AdPlaybackState.java @@ -258,7 +258,18 @@ private static long[] copyDurationsUsWithSpaceForAdCount(long[] durationsUs, int public static final int AD_STATE_ERROR = 4; /** Ad playback state with no ads. */ - public static final AdPlaybackState NONE = new AdPlaybackState(); + public static final AdPlaybackState NONE = + new AdPlaybackState( + /* adsId= */ null, + /* adGroupTimesUs= */ new long[0], + /* adGroups= */ null, + /* adResumePositionUs= */ 0L, + /* contentDurationUs= */ C.TIME_UNSET); + + /** + * The opaque identifier for ads with which this instance is associated, or {@code null} if unset. + */ + @Nullable public final Object adsId; /** The number of ad groups. */ public final int adGroupCount; @@ -280,29 +291,38 @@ private static long[] copyDurationsUsWithSpaceForAdCount(long[] durationsUs, int /** * Creates a new ad playback state with the specified ad group times. * + * @param adsId The opaque identifier for ads with which this instance is associated. * @param adGroupTimesUs The times of ad groups in microseconds, relative to the start of the * {@link com.google.android.exoplayer2.Timeline.Period} they belong to. A final element with * the value {@link C#TIME_END_OF_SOURCE} indicates that there is a postroll ad. */ - public AdPlaybackState(long... adGroupTimesUs) { - int count = adGroupTimesUs.length; - adGroupCount = count; - this.adGroupTimesUs = Arrays.copyOf(adGroupTimesUs, count); - this.adGroups = new AdGroup[count]; - for (int i = 0; i < count; i++) { - adGroups[i] = new AdGroup(); - } - adResumePositionUs = 0; - contentDurationUs = C.TIME_UNSET; + public AdPlaybackState(Object adsId, long... adGroupTimesUs) { + this( + adsId, + adGroupTimesUs, + /* adGroups= */ null, + /* adResumePositionUs= */ 0, + /* contentDurationUs= */ C.TIME_UNSET); } private AdPlaybackState( - long[] adGroupTimesUs, AdGroup[] adGroups, long adResumePositionUs, long contentDurationUs) { - adGroupCount = adGroups.length; + @Nullable Object adsId, + long[] adGroupTimesUs, + @Nullable AdGroup[] adGroups, + long adResumePositionUs, + long contentDurationUs) { + this.adsId = adsId; this.adGroupTimesUs = adGroupTimesUs; - this.adGroups = adGroups; this.adResumePositionUs = adResumePositionUs; this.contentDurationUs = contentDurationUs; + adGroupCount = adGroupTimesUs.length; + if (adGroups == null) { + adGroups = new AdGroup[adGroupCount]; + for (int i = 0; i < adGroupCount; i++) { + adGroups[i] = new AdGroup(); + } + } + this.adGroups = adGroups; } /** @@ -378,7 +398,8 @@ public AdPlaybackState withAdCount(int adGroupIndex, int adCount) { } AdGroup[] adGroups = Util.nullSafeArrayCopy(this.adGroups, this.adGroups.length); adGroups[adGroupIndex] = this.adGroups[adGroupIndex].withAdCount(adCount); - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } /** Returns an instance with the specified ad URI. */ @@ -386,7 +407,8 @@ public AdPlaybackState withAdCount(int adGroupIndex, int adCount) { public AdPlaybackState withAdUri(int adGroupIndex, int adIndexInAdGroup, Uri uri) { AdGroup[] adGroups = Util.nullSafeArrayCopy(this.adGroups, this.adGroups.length); adGroups[adGroupIndex] = adGroups[adGroupIndex].withAdUri(uri, adIndexInAdGroup); - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } /** Returns an instance with the specified ad marked as played. */ @@ -394,7 +416,8 @@ public AdPlaybackState withAdUri(int adGroupIndex, int adIndexInAdGroup, Uri uri public AdPlaybackState withPlayedAd(int adGroupIndex, int adIndexInAdGroup) { AdGroup[] adGroups = Util.nullSafeArrayCopy(this.adGroups, this.adGroups.length); adGroups[adGroupIndex] = adGroups[adGroupIndex].withAdState(AD_STATE_PLAYED, adIndexInAdGroup); - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } /** Returns an instance with the specified ad marked as skipped. */ @@ -402,7 +425,8 @@ public AdPlaybackState withPlayedAd(int adGroupIndex, int adIndexInAdGroup) { public AdPlaybackState withSkippedAd(int adGroupIndex, int adIndexInAdGroup) { AdGroup[] adGroups = Util.nullSafeArrayCopy(this.adGroups, this.adGroups.length); adGroups[adGroupIndex] = adGroups[adGroupIndex].withAdState(AD_STATE_SKIPPED, adIndexInAdGroup); - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } /** Returns an instance with the specified ad marked as having a load error. */ @@ -410,7 +434,8 @@ public AdPlaybackState withSkippedAd(int adGroupIndex, int adIndexInAdGroup) { public AdPlaybackState withAdLoadError(int adGroupIndex, int adIndexInAdGroup) { AdGroup[] adGroups = Util.nullSafeArrayCopy(this.adGroups, this.adGroups.length); adGroups[adGroupIndex] = adGroups[adGroupIndex].withAdState(AD_STATE_ERROR, adIndexInAdGroup); - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } /** @@ -421,7 +446,8 @@ public AdPlaybackState withAdLoadError(int adGroupIndex, int adIndexInAdGroup) { public AdPlaybackState withSkippedAdGroup(int adGroupIndex) { AdGroup[] adGroups = Util.nullSafeArrayCopy(this.adGroups, this.adGroups.length); adGroups[adGroupIndex] = adGroups[adGroupIndex].withAllAdsSkipped(); - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } /** Returns an instance with the specified ad durations, in microseconds. */ @@ -431,7 +457,8 @@ public AdPlaybackState withAdDurationsUs(long[][] adDurationUs) { for (int adGroupIndex = 0; adGroupIndex < adGroupCount; adGroupIndex++) { adGroups[adGroupIndex] = adGroups[adGroupIndex].withAdDurationsUs(adDurationUs[adGroupIndex]); } - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } /** @@ -443,7 +470,8 @@ public AdPlaybackState withAdResumePositionUs(long adResumePositionUs) { if (this.adResumePositionUs == adResumePositionUs) { return this; } else { - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } } @@ -453,7 +481,8 @@ public AdPlaybackState withContentDurationUs(long contentDurationUs) { if (this.contentDurationUs == contentDurationUs) { return this; } else { - return new AdPlaybackState(adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); + return new AdPlaybackState( + adsId, adGroupTimesUs, adGroups, adResumePositionUs, contentDurationUs); } } @@ -466,7 +495,8 @@ public boolean equals(@Nullable Object o) { return false; } AdPlaybackState that = (AdPlaybackState) o; - return adGroupCount == that.adGroupCount + return Util.areEqual(adsId, that.adsId) + && adGroupCount == that.adGroupCount && adResumePositionUs == that.adResumePositionUs && contentDurationUs == that.contentDurationUs && Arrays.equals(adGroupTimesUs, that.adGroupTimesUs) @@ -476,6 +506,7 @@ public boolean equals(@Nullable Object o) { @Override public int hashCode() { int result = adGroupCount; + result = 31 * result + (adsId == null ? 0 : adsId.hashCode()); result = 31 * result + (int) adResumePositionUs; result = 31 * result + (int) contentDurationUs; result = 31 * result + Arrays.hashCode(adGroupTimesUs); @@ -486,7 +517,9 @@ public int hashCode() { @Override public String toString() { StringBuilder sb = new StringBuilder(); - sb.append("AdPlaybackState(adResumePositionUs="); + sb.append("AdPlaybackState(adsId="); + sb.append(adsId); + sb.append(", adResumePositionUs="); sb.append(adResumePositionUs); sb.append(", adGroups=["); for (int i = 0; i < adGroups.length; i++) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/ads/package-info.java b/library/common/src/main/java/com/google/android/exoplayer2/source/ads/package-info.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/source/ads/package-info.java rename to library/common/src/main/java/com/google/android/exoplayer2/source/ads/package-info.java diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/package-info.java b/library/common/src/main/java/com/google/android/exoplayer2/source/package-info.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/source/package-info.java rename to library/common/src/main/java/com/google/android/exoplayer2/source/package-info.java diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/Cue.java b/library/common/src/main/java/com/google/android/exoplayer2/text/Cue.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/text/Cue.java rename to library/common/src/main/java/com/google/android/exoplayer2/text/Cue.java diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/TextOutput.java b/library/common/src/main/java/com/google/android/exoplayer2/text/TextOutput.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/text/TextOutput.java rename to library/common/src/main/java/com/google/android/exoplayer2/text/TextOutput.java diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/package-info.java b/library/common/src/main/java/com/google/android/exoplayer2/text/package-info.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/text/package-info.java rename to library/common/src/main/java/com/google/android/exoplayer2/text/package-info.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelection.java b/library/common/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelection.java new file mode 100644 index 00000000000..dca840790dc --- /dev/null +++ b/library/common/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelection.java @@ -0,0 +1,73 @@ +/* + * Copyright (C) 2016 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.trackselection; + +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.source.TrackGroup; + +/** + * A track selection consisting of a static subset of selected tracks belonging to a {@link + * TrackGroup}. + * + *

Tracks belonging to the subset are exposed in decreasing bandwidth order. + */ +public interface TrackSelection { + + /** Returns the {@link TrackGroup} to which the selected tracks belong. */ + TrackGroup getTrackGroup(); + + // Static subset of selected tracks. + + /** Returns the number of tracks in the selection. */ + int length(); + + /** + * Returns the format of the track at a given index in the selection. + * + * @param index The index in the selection. + * @return The format of the selected track. + */ + Format getFormat(int index); + + /** + * Returns the index in the track group of the track at a given index in the selection. + * + * @param index The index in the selection. + * @return The index of the selected track. + */ + int getIndexInTrackGroup(int index); + + /** + * Returns the index in the selection of the track with the specified format. The format is + * located by identity so, for example, {@code selection.indexOf(selection.getFormat(index)) == + * index} even if multiple selected tracks have formats that contain the same values. + * + * @param format The format. + * @return The index in the selection, or {@link C#INDEX_UNSET} if the track with the specified + * format is not part of the selection. + */ + int indexOf(Format format); + + /** + * Returns the index in the selection of the track with the specified index in the track group. + * + * @param indexInTrackGroup The index in the track group. + * @return The index in the selection, or {@link C#INDEX_UNSET} if the track with the specified + * index is not part of the selection. + */ + int indexOf(int indexInTrackGroup); +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionArray.java b/library/common/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionArray.java similarity index 99% rename from library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionArray.java rename to library/common/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionArray.java index fc20e863bac..b703998b2e9 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionArray.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionArray.java @@ -73,5 +73,4 @@ public boolean equals(@Nullable Object obj) { TrackSelectionArray other = (TrackSelectionArray) obj; return Arrays.equals(trackSelections, other.trackSelections); } - } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/package-info.java b/library/common/src/main/java/com/google/android/exoplayer2/trackselection/package-info.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/trackselection/package-info.java rename to library/common/src/main/java/com/google/android/exoplayer2/trackselection/package-info.java diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/BaseDataSource.java b/library/common/src/main/java/com/google/android/exoplayer2/upstream/BaseDataSource.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/upstream/BaseDataSource.java rename to library/common/src/main/java/com/google/android/exoplayer2/upstream/BaseDataSource.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/upstream/DataSpec.java b/library/common/src/main/java/com/google/android/exoplayer2/upstream/DataSpec.java index 75e23ae6f2a..d484779d365 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/upstream/DataSpec.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/upstream/DataSpec.java @@ -343,8 +343,7 @@ public static String getStringForHttpMethod(@HttpMethod int httpMethod) { * header directly. *

  • Other headers set at the {@link HttpDataSource} layer. I.e., headers set using {@link * HttpDataSource#setRequestProperty(String, String)}, and using {@link - * HttpDataSource.RequestProperties#set(String, String)} on the default properties obtained - * from {@link HttpDataSource.Factory#getDefaultRequestProperties()}. + * HttpDataSource.Factory#setDefaultRequestProperties(Map)}. * */ public final Map httpRequestHeaders; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSource.java b/library/common/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSource.java similarity index 77% rename from library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSource.java rename to library/common/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSource.java index 94c02c7e833..575a10b6cd5 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSource.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSource.java @@ -15,6 +15,8 @@ */ package com.google.android.exoplayer2.upstream; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Util.castNonNull; import static java.lang.Math.max; import static java.lang.Math.min; @@ -23,9 +25,7 @@ import androidx.annotation.Nullable; import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; -import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.upstream.DataSpec.HttpMethod; -import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.Util; import com.google.common.base.Predicate; @@ -43,25 +43,158 @@ import java.util.HashMap; import java.util.List; import java.util.Map; -import java.util.concurrent.atomic.AtomicReference; import java.util.regex.Matcher; import java.util.regex.Pattern; import java.util.zip.GZIPInputStream; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; /** * An {@link HttpDataSource} that uses Android's {@link HttpURLConnection}. * *

    By default this implementation will not follow cross-protocol redirects (i.e. redirects from - * HTTP to HTTPS or vice versa). Cross-protocol redirects can be enabled by using the {@link - * #DefaultHttpDataSource(String, int, int, boolean, RequestProperties)} constructor and passing - * {@code true} for the {@code allowCrossProtocolRedirects} argument. + * HTTP to HTTPS or vice versa). Cross-protocol redirects can be enabled by passing {@code true} to + * {@link DefaultHttpDataSource.Factory#setAllowCrossProtocolRedirects(boolean)}. * *

    Note: HTTP request headers will be set using all parameters passed via (in order of decreasing - * priority) the {@code dataSpec}, {@link #setRequestProperty} and the default parameters used to - * construct the instance. + * priority) the {@code dataSpec}, {@link #setRequestProperty} and the default properties that can + * be passed to {@link HttpDataSource.Factory#setDefaultRequestProperties(Map)}. */ public class DefaultHttpDataSource extends BaseDataSource implements HttpDataSource { + /** {@link DataSource.Factory} for {@link DefaultHttpDataSource} instances. */ + public static final class Factory implements HttpDataSource.Factory { + + private final RequestProperties defaultRequestProperties; + + @Nullable private TransferListener transferListener; + @Nullable private Predicate contentTypePredicate; + @Nullable private String userAgent; + private int connectTimeoutMs; + private int readTimeoutMs; + private boolean allowCrossProtocolRedirects; + + /** Creates an instance. */ + public Factory() { + defaultRequestProperties = new RequestProperties(); + connectTimeoutMs = DEFAULT_CONNECT_TIMEOUT_MILLIS; + readTimeoutMs = DEFAULT_READ_TIMEOUT_MILLIS; + } + + /** @deprecated Use {@link #setDefaultRequestProperties(Map)} instead. */ + @Deprecated + @Override + public final RequestProperties getDefaultRequestProperties() { + return defaultRequestProperties; + } + + @Override + public final Factory setDefaultRequestProperties(Map defaultRequestProperties) { + this.defaultRequestProperties.clearAndSet(defaultRequestProperties); + return this; + } + + /** + * Sets the user agent that will be used. + * + *

    The default is {@code null}, which causes the default user agent of the underlying + * platform to be used. + * + * @param userAgent The user agent that will be used, or {@code null} to use the default user + * agent of the underlying platform. + * @return This factory. + */ + public Factory setUserAgent(@Nullable String userAgent) { + this.userAgent = userAgent; + return this; + } + + /** + * Sets the connect timeout, in milliseconds. + * + *

    The default is {@link DefaultHttpDataSource#DEFAULT_CONNECT_TIMEOUT_MILLIS}. + * + * @param connectTimeoutMs The connect timeout, in milliseconds, that will be used. + * @return This factory. + */ + public Factory setConnectTimeoutMs(int connectTimeoutMs) { + this.connectTimeoutMs = connectTimeoutMs; + return this; + } + + /** + * Sets the read timeout, in milliseconds. + * + *

    The default is {@link DefaultHttpDataSource#DEFAULT_READ_TIMEOUT_MILLIS}. + * + * @param readTimeoutMs The connect timeout, in milliseconds, that will be used. + * @return This factory. + */ + public Factory setReadTimeoutMs(int readTimeoutMs) { + this.readTimeoutMs = readTimeoutMs; + return this; + } + + /** + * Sets whether to allow cross protocol redirects. + * + *

    The default is {@code false}. + * + * @param allowCrossProtocolRedirects Whether to allow cross protocol redirects. + * @return This factory. + */ + public Factory setAllowCrossProtocolRedirects(boolean allowCrossProtocolRedirects) { + this.allowCrossProtocolRedirects = allowCrossProtocolRedirects; + return this; + } + + /** + * Sets a content type {@link Predicate}. If a content type is rejected by the predicate then a + * {@link HttpDataSource.InvalidContentTypeException} is thrown from {@link + * DefaultHttpDataSource#open(DataSpec)}. + * + *

    The default is {@code null}. + * + * @param contentTypePredicate The content type {@link Predicate}, or {@code null} to clear a + * predicate that was previously set. + * @return This factory. + */ + public Factory setContentTypePredicate(@Nullable Predicate contentTypePredicate) { + this.contentTypePredicate = contentTypePredicate; + return this; + } + + /** + * Sets the {@link TransferListener} that will be used. + * + *

    The default is {@code null}. + * + *

    See {@link DataSource#addTransferListener(TransferListener)}. + * + * @param transferListener The listener that will be used. + * @return This factory. + */ + public Factory setTransferListener(@Nullable TransferListener transferListener) { + this.transferListener = transferListener; + return this; + } + + @Override + public DefaultHttpDataSource createDataSource() { + DefaultHttpDataSource dataSource = + new DefaultHttpDataSource( + userAgent, + connectTimeoutMs, + readTimeoutMs, + allowCrossProtocolRedirects, + defaultRequestProperties, + contentTypePredicate); + if (transferListener != null) { + dataSource.addTransferListener(transferListener); + } + return dataSource; + } + } + /** The default connection timeout, in milliseconds. */ public static final int DEFAULT_CONNECT_TIMEOUT_MILLIS = 8 * 1000; /** @@ -76,12 +209,11 @@ public class DefaultHttpDataSource extends BaseDataSource implements HttpDataSou private static final long MAX_BYTES_TO_DRAIN = 2048; private static final Pattern CONTENT_RANGE_HEADER = Pattern.compile("^bytes (\\d+)-(\\d+)/(\\d+)$"); - private static final AtomicReference skipBufferReference = new AtomicReference<>(); private final boolean allowCrossProtocolRedirects; private final int connectTimeoutMillis; private final int readTimeoutMillis; - private final String userAgent; + @Nullable private final String userAgent; @Nullable private final RequestProperties defaultRequestProperties; private final RequestProperties requestProperties; @@ -89,6 +221,7 @@ public class DefaultHttpDataSource extends BaseDataSource implements HttpDataSou @Nullable private DataSpec dataSpec; @Nullable private HttpURLConnection connection; @Nullable private InputStream inputStream; + private byte @MonotonicNonNull [] skipBuffer; private boolean opened; private int responseCode; @@ -98,33 +231,25 @@ public class DefaultHttpDataSource extends BaseDataSource implements HttpDataSou private long bytesSkipped; private long bytesRead; - /** Creates an instance. */ + /** @deprecated Use {@link DefaultHttpDataSource.Factory} instead. */ + @SuppressWarnings("deprecation") + @Deprecated public DefaultHttpDataSource() { - this( - ExoPlayerLibraryInfo.DEFAULT_USER_AGENT, - DEFAULT_CONNECT_TIMEOUT_MILLIS, - DEFAULT_READ_TIMEOUT_MILLIS); + this(/* userAgent= */ null, DEFAULT_CONNECT_TIMEOUT_MILLIS, DEFAULT_READ_TIMEOUT_MILLIS); } - /** - * Creates an instance. - * - * @param userAgent The User-Agent string that should be used. - */ - public DefaultHttpDataSource(String userAgent) { + /** @deprecated Use {@link DefaultHttpDataSource.Factory} instead. */ + @SuppressWarnings("deprecation") + @Deprecated + public DefaultHttpDataSource(@Nullable String userAgent) { this(userAgent, DEFAULT_CONNECT_TIMEOUT_MILLIS, DEFAULT_READ_TIMEOUT_MILLIS); } - /** - * Creates an instance. - * - * @param userAgent The User-Agent string that should be used. - * @param connectTimeoutMillis The connection timeout, in milliseconds. A timeout of zero is - * interpreted as an infinite timeout. - * @param readTimeoutMillis The read timeout, in milliseconds. A timeout of zero is interpreted as - * an infinite timeout. - */ - public DefaultHttpDataSource(String userAgent, int connectTimeoutMillis, int readTimeoutMillis) { + /** @deprecated Use {@link DefaultHttpDataSource.Factory} instead. */ + @SuppressWarnings("deprecation") + @Deprecated + public DefaultHttpDataSource( + @Nullable String userAgent, int connectTimeoutMillis, int readTimeoutMillis) { this( userAgent, connectTimeoutMillis, @@ -133,129 +258,45 @@ public DefaultHttpDataSource(String userAgent, int connectTimeoutMillis, int rea /* defaultRequestProperties= */ null); } - /** - * Creates an instance. - * - * @param userAgent The User-Agent string that should be used. - * @param connectTimeoutMillis The connection timeout, in milliseconds. A timeout of zero is - * interpreted as an infinite timeout. Pass {@link #DEFAULT_CONNECT_TIMEOUT_MILLIS} to use the - * default value. - * @param readTimeoutMillis The read timeout, in milliseconds. A timeout of zero is interpreted as - * an infinite timeout. Pass {@link #DEFAULT_READ_TIMEOUT_MILLIS} to use the default value. - * @param allowCrossProtocolRedirects Whether cross-protocol redirects (i.e. redirects from HTTP - * to HTTPS and vice versa) are enabled. - * @param defaultRequestProperties The default request properties to be sent to the server as HTTP - * headers or {@code null} if not required. - */ + /** @deprecated Use {@link DefaultHttpDataSource.Factory} instead. */ + @Deprecated public DefaultHttpDataSource( - String userAgent, + @Nullable String userAgent, int connectTimeoutMillis, int readTimeoutMillis, boolean allowCrossProtocolRedirects, @Nullable RequestProperties defaultRequestProperties) { - super(/* isNetwork= */ true); - this.userAgent = Assertions.checkNotEmpty(userAgent); - this.requestProperties = new RequestProperties(); - this.connectTimeoutMillis = connectTimeoutMillis; - this.readTimeoutMillis = readTimeoutMillis; - this.allowCrossProtocolRedirects = allowCrossProtocolRedirects; - this.defaultRequestProperties = defaultRequestProperties; - } - - /** - * Creates an instance. - * - * @param userAgent The User-Agent string that should be used. - * @param contentTypePredicate An optional {@link Predicate}. If a content type is rejected by the - * predicate then a {@link HttpDataSource.InvalidContentTypeException} is thrown from {@link - * #open(DataSpec)}. - * @deprecated Use {@link #DefaultHttpDataSource(String)} and {@link - * #setContentTypePredicate(Predicate)}. - */ - @SuppressWarnings("deprecation") - @Deprecated - public DefaultHttpDataSource(String userAgent, @Nullable Predicate contentTypePredicate) { this( userAgent, - contentTypePredicate, - DEFAULT_CONNECT_TIMEOUT_MILLIS, - DEFAULT_READ_TIMEOUT_MILLIS); - } - - /** - * Creates an instance. - * - * @param userAgent The User-Agent string that should be used. - * @param contentTypePredicate An optional {@link Predicate}. If a content type is rejected by the - * predicate then a {@link HttpDataSource.InvalidContentTypeException} is thrown from {@link - * #open(DataSpec)}. - * @param connectTimeoutMillis The connection timeout, in milliseconds. A timeout of zero is - * interpreted as an infinite timeout. - * @param readTimeoutMillis The read timeout, in milliseconds. A timeout of zero is interpreted as - * an infinite timeout. - * @deprecated Use {@link #DefaultHttpDataSource(String, int, int)} and {@link - * #setContentTypePredicate(Predicate)}. - */ - @SuppressWarnings("deprecation") - @Deprecated - public DefaultHttpDataSource( - String userAgent, - @Nullable Predicate contentTypePredicate, - int connectTimeoutMillis, - int readTimeoutMillis) { - this( - userAgent, - contentTypePredicate, connectTimeoutMillis, readTimeoutMillis, - /* allowCrossProtocolRedirects= */ false, - /* defaultRequestProperties= */ null); + allowCrossProtocolRedirects, + defaultRequestProperties, + /* contentTypePredicate= */ null); } - /** - * Creates an instance. - * - * @param userAgent The User-Agent string that should be used. - * @param contentTypePredicate An optional {@link Predicate}. If a content type is rejected by the - * predicate then a {@link HttpDataSource.InvalidContentTypeException} is thrown from {@link - * #open(DataSpec)}. - * @param connectTimeoutMillis The connection timeout, in milliseconds. A timeout of zero is - * interpreted as an infinite timeout. Pass {@link #DEFAULT_CONNECT_TIMEOUT_MILLIS} to use the - * default value. - * @param readTimeoutMillis The read timeout, in milliseconds. A timeout of zero is interpreted as - * an infinite timeout. Pass {@link #DEFAULT_READ_TIMEOUT_MILLIS} to use the default value. - * @param allowCrossProtocolRedirects Whether cross-protocol redirects (i.e. redirects from HTTP - * to HTTPS and vice versa) are enabled. - * @param defaultRequestProperties The default request properties to be sent to the server as HTTP - * headers or {@code null} if not required. - * @deprecated Use {@link #DefaultHttpDataSource(String, int, int, boolean, RequestProperties)} - * and {@link #setContentTypePredicate(Predicate)}. - */ - @Deprecated - public DefaultHttpDataSource( - String userAgent, - @Nullable Predicate contentTypePredicate, + private DefaultHttpDataSource( + @Nullable String userAgent, int connectTimeoutMillis, int readTimeoutMillis, boolean allowCrossProtocolRedirects, - @Nullable RequestProperties defaultRequestProperties) { + @Nullable RequestProperties defaultRequestProperties, + @Nullable Predicate contentTypePredicate) { super(/* isNetwork= */ true); - this.userAgent = Assertions.checkNotEmpty(userAgent); - this.contentTypePredicate = contentTypePredicate; - this.requestProperties = new RequestProperties(); + this.userAgent = userAgent; this.connectTimeoutMillis = connectTimeoutMillis; this.readTimeoutMillis = readTimeoutMillis; this.allowCrossProtocolRedirects = allowCrossProtocolRedirects; this.defaultRequestProperties = defaultRequestProperties; + this.contentTypePredicate = contentTypePredicate; + this.requestProperties = new RequestProperties(); } /** - * Sets a content type {@link Predicate}. If a content type is rejected by the predicate then a - * {@link HttpDataSource.InvalidContentTypeException} is thrown from {@link #open(DataSpec)}. - * - * @param contentTypePredicate The content type {@link Predicate}, or {@code null} to clear a - * predicate that was previously set. + * @deprecated Use {@link DefaultHttpDataSource.Factory#setContentTypePredicate(Predicate)} + * instead. */ + @Deprecated public void setContentTypePredicate(@Nullable Predicate contentTypePredicate) { this.contentTypePredicate = contentTypePredicate; } @@ -278,14 +319,14 @@ public Map> getResponseHeaders() { @Override public void setRequestProperty(String name, String value) { - Assertions.checkNotNull(name); - Assertions.checkNotNull(value); + checkNotNull(name); + checkNotNull(value); requestProperties.set(name, value); } @Override public void clearRequestProperty(String name) { - Assertions.checkNotNull(name); + checkNotNull(name); requestProperties.remove(name); } @@ -303,6 +344,7 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { this.bytesRead = 0; this.bytesSkipped = 0; transferInitializing(dataSpec); + try { connection = makeConnection(dataSpec); } catch (IOException e) { @@ -315,6 +357,7 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { "Unable to connect", e, dataSpec, HttpDataSourceException.TYPE_OPEN); } + HttpURLConnection connection = this.connection; String responseMessage; try { responseCode = connection.getResponseCode(); @@ -334,8 +377,7 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { errorResponseBody = errorStream != null ? Util.toByteArray(errorStream) : Util.EMPTY_BYTE_ARRAY; } catch (IOException e) { - throw new HttpDataSourceException( - "Error reading non-2xx response body", e, dataSpec, HttpDataSourceException.TYPE_OPEN); + errorResponseBody = Util.EMPTY_BYTE_ARRAY; } closeConnectionQuietly(); InvalidResponseCodeException exception = @@ -399,19 +441,22 @@ public int read(byte[] buffer, int offset, int readLength) throws HttpDataSource skipInternal(); return readInternal(buffer, offset, readLength); } catch (IOException e) { - throw new HttpDataSourceException(e, dataSpec, HttpDataSourceException.TYPE_READ); + throw new HttpDataSourceException( + e, castNonNull(dataSpec), HttpDataSourceException.TYPE_READ); } } @Override public void close() throws HttpDataSourceException { try { + @Nullable InputStream inputStream = this.inputStream; if (inputStream != null) { maybeTerminateInputStream(connection, bytesRemaining()); try { inputStream.close(); } catch (IOException e) { - throw new HttpDataSourceException(e, dataSpec, HttpDataSourceException.TYPE_CLOSE); + throw new HttpDataSourceException( + e, castNonNull(dataSpec), HttpDataSourceException.TYPE_CLOSE); } } } finally { @@ -578,7 +623,9 @@ private HttpURLConnection makeConnection( } connection.setRequestProperty("Range", rangeRequest); } - connection.setRequestProperty("User-Agent", userAgent); + if (userAgent != null) { + connection.setRequestProperty("User-Agent", userAgent); + } connection.setRequestProperty("Accept-Encoding", allowGzip ? "gzip" : "identity"); connection.setInstanceFollowRedirects(followRedirects); connection.setDoOutput(httpBody != null); @@ -653,7 +700,9 @@ private static long getContentLength(HttpURLConnection connection) { if (matcher.find()) { try { long contentLengthFromRange = - Long.parseLong(matcher.group(2)) - Long.parseLong(matcher.group(1)) + 1; + Long.parseLong(checkNotNull(matcher.group(2))) + - Long.parseLong(checkNotNull(matcher.group(1))) + + 1; if (contentLength < 0) { // Some proxy servers strip the Content-Length header. Fall back to the length // calculated here in this case. @@ -688,15 +737,13 @@ private void skipInternal() throws IOException { return; } - // Acquire the shared skip buffer. - byte[] skipBuffer = skipBufferReference.getAndSet(null); if (skipBuffer == null) { skipBuffer = new byte[4096]; } while (bytesSkipped != bytesToSkip) { int readLength = (int) min(bytesToSkip - bytesSkipped, skipBuffer.length); - int read = inputStream.read(skipBuffer, 0, readLength); + int read = castNonNull(inputStream).read(skipBuffer, 0, readLength); if (Thread.currentThread().isInterrupted()) { throw new InterruptedIOException(); } @@ -706,9 +753,6 @@ private void skipInternal() throws IOException { bytesSkipped += read; bytesTransferred(read); } - - // Release the shared skip buffer. - skipBufferReference.set(skipBuffer); } /** @@ -737,7 +781,7 @@ private int readInternal(byte[] buffer, int offset, int readLength) throws IOExc readLength = (int) min(readLength, bytesRemaining); } - int read = inputStream.read(buffer, offset, readLength); + int read = castNonNull(inputStream).read(buffer, offset, readLength); if (read == -1) { if (bytesToRead != C.LENGTH_UNSET) { // End of stream reached having not read sufficient data. @@ -762,8 +806,9 @@ private int readInternal(byte[] buffer, int offset, int readLength) throws IOExc * @param bytesRemaining The number of bytes remaining to be read from the input stream if its * length is known. {@link C#LENGTH_UNSET} otherwise. */ - private static void maybeTerminateInputStream(HttpURLConnection connection, long bytesRemaining) { - if (Util.SDK_INT != 19 && Util.SDK_INT != 20) { + private static void maybeTerminateInputStream( + @Nullable HttpURLConnection connection, long bytesRemaining) { + if (connection == null || Util.SDK_INT < 19 || Util.SDK_INT > 20) { return; } @@ -784,7 +829,8 @@ private static void maybeTerminateInputStream(HttpURLConnection connection, long || "com.android.okhttp.internal.http.HttpTransport$FixedLengthInputStream" .equals(className)) { Class superclass = inputStream.getClass().getSuperclass(); - Method unexpectedEndOfInput = superclass.getDeclaredMethod("unexpectedEndOfInput"); + Method unexpectedEndOfInput = + checkNotNull(superclass).getDeclaredMethod("unexpectedEndOfInput"); unexpectedEndOfInput.setAccessible(true); unexpectedEndOfInput.invoke(inputStream); } @@ -795,7 +841,6 @@ private static void maybeTerminateInputStream(HttpURLConnection connection, long } } - /** * Closes the current connection quietly, if there is one. */ diff --git a/library/common/src/main/java/com/google/android/exoplayer2/upstream/HttpDataSource.java b/library/common/src/main/java/com/google/android/exoplayer2/upstream/HttpDataSource.java index 1c450ca02d9..e2ae38b013e 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/upstream/HttpDataSource.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/upstream/HttpDataSource.java @@ -42,43 +42,23 @@ interface Factory extends DataSource.Factory { @Override HttpDataSource createDataSource(); - /** - * Gets the default request properties used by all {@link HttpDataSource}s created by the - * factory. Changes to the properties will be reflected in any future requests made by - * {@link HttpDataSource}s created by the factory. - * - * @return The default request properties of the factory. - */ - RequestProperties getDefaultRequestProperties(); - - /** - * Sets a default request header for {@link HttpDataSource} instances created by the factory. - * - * @deprecated Use {@link #getDefaultRequestProperties} instead. - * @param name The name of the header field. - * @param value The value of the field. - */ + /** @deprecated Use {@link #setDefaultRequestProperties(Map)} instead. */ @Deprecated - void setDefaultRequestProperty(String name, String value); + RequestProperties getDefaultRequestProperties(); /** - * Clears a default request header for {@link HttpDataSource} instances created by the factory. + * Sets the default request headers for {@link HttpDataSource} instances created by the factory. * - * @deprecated Use {@link #getDefaultRequestProperties} instead. - * @param name The name of the header field. - */ - @Deprecated - void clearDefaultRequestProperty(String name); - - /** - * Clears all default request headers for all {@link HttpDataSource} instances created by the - * factory. + *

    The new request properties will be used for future requests made by {@link HttpDataSource + * HttpDataSources} created by the factory, including instances that have already been created. + * Modifying the {@code defaultRequestProperties} map after a call to this method will have no + * effect, and so it's necessary to call this method again each time the request properties need + * to be updated. * - * @deprecated Use {@link #getDefaultRequestProperties} instead. + * @param defaultRequestProperties The default request properties. + * @return This factory. */ - @Deprecated - void clearAllDefaultRequestProperties(); - + Factory setDefaultRequestProperties(Map defaultRequestProperties); } /** @@ -159,12 +139,9 @@ public synchronized Map getSnapshot() { } return requestPropertiesSnapshot; } - } - /** - * Base implementation of {@link Factory} that sets default request properties. - */ + /** Base implementation of {@link Factory} that sets default request properties. */ abstract class BaseFactory implements Factory { private final RequestProperties defaultRequestProperties; @@ -178,30 +155,17 @@ public final HttpDataSource createDataSource() { return createDataSourceInternal(defaultRequestProperties); } + /** @deprecated Use {@link #setDefaultRequestProperties(Map)} instead. */ + @Deprecated @Override public final RequestProperties getDefaultRequestProperties() { return defaultRequestProperties; } - /** @deprecated Use {@link #getDefaultRequestProperties} instead. */ - @Deprecated - @Override - public final void setDefaultRequestProperty(String name, String value) { - defaultRequestProperties.set(name, value); - } - - /** @deprecated Use {@link #getDefaultRequestProperties} instead. */ - @Deprecated @Override - public final void clearDefaultRequestProperty(String name) { - defaultRequestProperties.remove(name); - } - - /** @deprecated Use {@link #getDefaultRequestProperties} instead. */ - @Deprecated - @Override - public final void clearAllDefaultRequestProperties() { - defaultRequestProperties.clear(); + public final Factory setDefaultRequestProperties(Map defaultRequestProperties) { + this.defaultRequestProperties.clearAndSet(defaultRequestProperties); + return this; } /** @@ -211,9 +175,8 @@ public final void clearAllDefaultRequestProperties() { * {@link HttpDataSource} instance. * @return A {@link HttpDataSource} instance. */ - protected abstract HttpDataSource createDataSourceInternal(RequestProperties - defaultRequestProperties); - + protected abstract HttpDataSource createDataSourceInternal( + RequestProperties defaultRequestProperties); } /** A {@link Predicate} that rejects content types often used for pay-walls. */ diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/Assertions.java b/library/common/src/main/java/com/google/android/exoplayer2/util/Assertions.java index 0f3bbfa14d2..c6173730ffa 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/util/Assertions.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/Assertions.java @@ -20,6 +20,7 @@ import androidx.annotation.Nullable; import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import org.checkerframework.checker.nullness.qual.EnsuresNonNull; +import org.checkerframework.dataflow.qual.Pure; /** * Provides methods for asserting the truth of expressions and properties. @@ -34,6 +35,7 @@ private Assertions() {} * @param expression The expression to evaluate. * @throws IllegalArgumentException If {@code expression} is false. */ + @Pure public static void checkArgument(boolean expression) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && !expression) { throw new IllegalArgumentException(); @@ -48,6 +50,7 @@ public static void checkArgument(boolean expression) { * to a {@link String} using {@link String#valueOf(Object)}. * @throws IllegalArgumentException If {@code expression} is false. */ + @Pure public static void checkArgument(boolean expression, Object errorMessage) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && !expression) { throw new IllegalArgumentException(String.valueOf(errorMessage)); @@ -63,6 +66,7 @@ public static void checkArgument(boolean expression, Object errorMessage) { * @return The {@code index} that was validated. * @throws IndexOutOfBoundsException If {@code index} falls outside the specified bounds. */ + @Pure public static int checkIndex(int index, int start, int limit) { if (index < start || index >= limit) { throw new IndexOutOfBoundsException(); @@ -76,6 +80,7 @@ public static int checkIndex(int index, int start, int limit) { * @param expression The expression to evaluate. * @throws IllegalStateException If {@code expression} is false. */ + @Pure public static void checkState(boolean expression) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && !expression) { throw new IllegalStateException(); @@ -90,6 +95,7 @@ public static void checkState(boolean expression) { * to a {@link String} using {@link String#valueOf(Object)}. * @throws IllegalStateException If {@code expression} is false. */ + @Pure public static void checkState(boolean expression, Object errorMessage) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && !expression) { throw new IllegalStateException(String.valueOf(errorMessage)); @@ -106,6 +112,7 @@ public static void checkState(boolean expression, Object errorMessage) { */ @SuppressWarnings({"contracts.postcondition.not.satisfied", "return.type.incompatible"}) @EnsuresNonNull({"#1"}) + @Pure public static T checkStateNotNull(@Nullable T reference) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && reference == null) { throw new IllegalStateException(); @@ -125,6 +132,7 @@ public static T checkStateNotNull(@Nullable T reference) { */ @SuppressWarnings({"contracts.postcondition.not.satisfied", "return.type.incompatible"}) @EnsuresNonNull({"#1"}) + @Pure public static T checkStateNotNull(@Nullable T reference, Object errorMessage) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && reference == null) { throw new IllegalStateException(String.valueOf(errorMessage)); @@ -142,6 +150,7 @@ public static T checkStateNotNull(@Nullable T reference, Object errorMessage */ @SuppressWarnings({"contracts.postcondition.not.satisfied", "return.type.incompatible"}) @EnsuresNonNull({"#1"}) + @Pure public static T checkNotNull(@Nullable T reference) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && reference == null) { throw new NullPointerException(); @@ -161,6 +170,7 @@ public static T checkNotNull(@Nullable T reference) { */ @SuppressWarnings({"contracts.postcondition.not.satisfied", "return.type.incompatible"}) @EnsuresNonNull({"#1"}) + @Pure public static T checkNotNull(@Nullable T reference, Object errorMessage) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && reference == null) { throw new NullPointerException(String.valueOf(errorMessage)); @@ -177,6 +187,7 @@ public static T checkNotNull(@Nullable T reference, Object errorMessage) { */ @SuppressWarnings({"contracts.postcondition.not.satisfied", "return.type.incompatible"}) @EnsuresNonNull({"#1"}) + @Pure public static String checkNotEmpty(@Nullable String string) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && TextUtils.isEmpty(string)) { throw new IllegalArgumentException(); @@ -195,6 +206,7 @@ public static String checkNotEmpty(@Nullable String string) { */ @SuppressWarnings({"contracts.postcondition.not.satisfied", "return.type.incompatible"}) @EnsuresNonNull({"#1"}) + @Pure public static String checkNotEmpty(@Nullable String string, Object errorMessage) { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && TextUtils.isEmpty(string)) { throw new IllegalArgumentException(String.valueOf(errorMessage)); @@ -208,6 +220,7 @@ public static String checkNotEmpty(@Nullable String string, Object errorMessage) * * @throws IllegalStateException If the calling thread is not the application's main thread. */ + @Pure public static void checkMainThread() { if (ExoPlayerLibraryInfo.ASSERTIONS_ENABLED && Looper.myLooper() != Looper.getMainLooper()) { throw new IllegalStateException("Not in applications main thread"); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/util/Clock.java b/library/common/src/main/java/com/google/android/exoplayer2/util/Clock.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/util/Clock.java rename to library/common/src/main/java/com/google/android/exoplayer2/util/Clock.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/CodecSpecificDataUtil.java b/library/common/src/main/java/com/google/android/exoplayer2/util/CodecSpecificDataUtil.java index 3360e88d4fa..0f8edb4acd1 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/util/CodecSpecificDataUtil.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/CodecSpecificDataUtil.java @@ -26,6 +26,8 @@ public final class CodecSpecificDataUtil { private static final byte[] NAL_START_CODE = new byte[] {0, 0, 0, 1}; + private static final String[] HEVC_GENERAL_PROFILE_SPACE_STRINGS = + new String[] {"", "A", "B", "C"}; /** * Parses an ALAC AudioSpecificConfig (i.e. an 0 && constraintBytes[trailingZeroIndex - 1] == 0) { + trailingZeroIndex--; + } + for (int i = 0; i < trailingZeroIndex; i++) { + builder.append(String.format(".%02X", constraintBytes[i])); + } + return builder.toString(); + } + /** * Constructs a NAL unit consisting of the NAL start code followed by the specified data. * diff --git a/library/core/src/main/java/com/google/android/exoplayer2/util/ConditionVariable.java b/library/common/src/main/java/com/google/android/exoplayer2/util/ConditionVariable.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/util/ConditionVariable.java rename to library/common/src/main/java/com/google/android/exoplayer2/util/ConditionVariable.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/FileTypes.java b/library/common/src/main/java/com/google/android/exoplayer2/util/FileTypes.java index d4b87abfddf..53396e135bc 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/util/FileTypes.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/FileTypes.java @@ -33,11 +33,13 @@ public final class FileTypes { /** * File types. One of {@link #UNKNOWN}, {@link #AC3}, {@link #AC4}, {@link #ADTS}, {@link #AMR}, * {@link #FLAC}, {@link #FLV}, {@link #MATROSKA}, {@link #MP3}, {@link #MP4}, {@link #OGG}, - * {@link #PS}, {@link #TS}, {@link #WAV} and {@link #WEBVTT}. + * {@link #PS}, {@link #TS}, {@link #WAV}, {@link #WEBVTT} and {@link #JPEG}. */ @Documented @Retention(RetentionPolicy.SOURCE) - @IntDef({UNKNOWN, AC3, AC4, ADTS, AMR, FLAC, FLV, MATROSKA, MP3, MP4, OGG, PS, TS, WAV, WEBVTT}) + @IntDef({ + UNKNOWN, AC3, AC4, ADTS, AMR, FLAC, FLV, MATROSKA, MP3, MP4, OGG, PS, TS, WAV, WEBVTT, JPEG + }) public @interface Type {} /** Unknown file type. */ public static final int UNKNOWN = -1; @@ -69,6 +71,8 @@ public final class FileTypes { public static final int WAV = 12; /** File type for the WebVTT format. */ public static final int WEBVTT = 13; + /** File type for the JPEG format. */ + public static final int JPEG = 14; @VisibleForTesting /* package */ static final String HEADER_CONTENT_TYPE = "Content-Type"; @@ -99,6 +103,8 @@ public final class FileTypes { private static final String EXTENSION_WAVE = ".wave"; private static final String EXTENSION_VTT = ".vtt"; private static final String EXTENSION_WEBVTT = ".webvtt"; + private static final String EXTENSION_JPG = ".jpg"; + private static final String EXTENSION_JPEG = ".jpeg"; private FileTypes() {} @@ -159,6 +165,8 @@ public static int inferFileTypeFromMimeType(@Nullable String mimeType) { return FileTypes.WAV; case MimeTypes.TEXT_VTT: return FileTypes.WEBVTT; + case MimeTypes.IMAGE_JPEG: + return FileTypes.JPEG; default: return FileTypes.UNKNOWN; } @@ -219,6 +227,8 @@ public static int inferFileTypeFromUri(Uri uri) { return FileTypes.WAV; } else if (filename.endsWith(EXTENSION_VTT) || filename.endsWith(EXTENSION_WEBVTT)) { return FileTypes.WEBVTT; + } else if (filename.endsWith(EXTENSION_JPG) || filename.endsWith(EXTENSION_JPEG)) { + return FileTypes.JPEG; } else { return FileTypes.UNKNOWN; } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/util/HandlerWrapper.java b/library/common/src/main/java/com/google/android/exoplayer2/util/HandlerWrapper.java similarity index 65% rename from library/core/src/main/java/com/google/android/exoplayer2/util/HandlerWrapper.java rename to library/common/src/main/java/com/google/android/exoplayer2/util/HandlerWrapper.java index 5b85b26c3f5..edf775bd5b9 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/util/HandlerWrapper.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/HandlerWrapper.java @@ -26,36 +26,42 @@ */ public interface HandlerWrapper { - /** @see Handler#getLooper() */ + /** See {@link Handler#getLooper()}. */ Looper getLooper(); - /** @see Handler#obtainMessage(int) */ + /** See {@link Handler#hasMessages(int)}. */ + boolean hasMessages(int what); + + /** See {@link Handler#obtainMessage(int)}. */ Message obtainMessage(int what); - /** @see Handler#obtainMessage(int, Object) */ + /** See {@link Handler#obtainMessage(int, Object)}. */ Message obtainMessage(int what, @Nullable Object obj); - /** @see Handler#obtainMessage(int, int, int) */ + /** See {@link Handler#obtainMessage(int, int, int)}. */ Message obtainMessage(int what, int arg1, int arg2); - /** @see Handler#obtainMessage(int, int, int, Object) */ + /** See {@link Handler#obtainMessage(int, int, int, Object)}. */ Message obtainMessage(int what, int arg1, int arg2, @Nullable Object obj); - /** @see Handler#sendEmptyMessage(int) */ + /** See {@link Handler#sendEmptyMessage(int)}. */ boolean sendEmptyMessage(int what); - /** @see Handler#sendEmptyMessageAtTime(int, long) */ + /** See {@link Handler#sendEmptyMessageDelayed(int, long)}. */ + boolean sendEmptyMessageDelayed(int what, int delayMs); + + /** See {@link Handler#sendEmptyMessageAtTime(int, long)}. */ boolean sendEmptyMessageAtTime(int what, long uptimeMs); - /** @see Handler#removeMessages(int) */ + /** See {@link Handler#removeMessages(int)}. */ void removeMessages(int what); - /** @see Handler#removeCallbacksAndMessages(Object) */ + /** See {@link Handler#removeCallbacksAndMessages(Object)}. */ void removeCallbacksAndMessages(@Nullable Object token); - /** @see Handler#post(Runnable) */ + /** See {@link Handler#post(Runnable)}. */ boolean post(Runnable runnable); - /** @see Handler#postDelayed(Runnable, long) */ + /** See {@link Handler#postDelayed(Runnable, long)}. */ boolean postDelayed(Runnable runnable, long delayMs); } diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/ListenerSet.java b/library/common/src/main/java/com/google/android/exoplayer2/util/ListenerSet.java new file mode 100644 index 00000000000..a9a749e47f3 --- /dev/null +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/ListenerSet.java @@ -0,0 +1,329 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.util; + +import android.os.Looper; +import android.os.Message; +import androidx.annotation.CheckResult; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.common.base.Supplier; +import java.util.ArrayDeque; +import java.util.concurrent.CopyOnWriteArraySet; +import javax.annotation.Nonnull; + +/** + * A set of listeners. + * + *

    Events are guaranteed to arrive in the order in which they happened even if a new event is + * triggered recursively from another listener. + * + *

    Events are also guaranteed to be only sent to the listeners registered at the time the event + * was enqueued and haven't been removed since. + * + * @param The listener type. + * @param The {@link MutableFlags} type used to indicate which events occurred. + */ +public final class ListenerSet { + + /** + * An event sent to a listener. + * + * @param The listener type. + */ + public interface Event { + + /** Invokes the event notification on the given listener. */ + void invoke(T listener); + } + + /** + * An event sent to a listener when all other events sent during one {@link Looper} message queue + * iteration were handled by the listener. + * + * @param The listener type. + * @param The {@link MutableFlags} type used to indicate which events occurred. + */ + public interface IterationFinishedEvent { + + /** + * Invokes the iteration finished event. + * + * @param listener The listener to invoke the event on. + * @param eventFlags The combined event flags of all events sent in this iteration. + */ + void invoke(T listener, E eventFlags); + } + + private static final int MSG_ITERATION_FINISHED = 0; + private static final int MSG_LAZY_RELEASE = 1; + + private final Clock clock; + private final HandlerWrapper handler; + private final Supplier eventFlagsSupplier; + private final IterationFinishedEvent iterationFinishedEvent; + private final CopyOnWriteArraySet> listeners; + private final ArrayDeque flushingEvents; + private final ArrayDeque queuedEvents; + + private boolean released; + + /** + * Creates a new listener set. + * + * @param looper A {@link Looper} used to call listeners on. The same {@link Looper} must be used + * to call all other methods of this class. + * @param clock A {@link Clock}. + * @param eventFlagsSupplier A {@link Supplier} for new instances of {@link E the event flags + * type}. + * @param iterationFinishedEvent An {@link IterationFinishedEvent} sent when all other events sent + * during one {@link Looper} message queue iteration were handled by the listeners. + */ + public ListenerSet( + Looper looper, + Clock clock, + Supplier eventFlagsSupplier, + IterationFinishedEvent iterationFinishedEvent) { + this( + /* listeners= */ new CopyOnWriteArraySet<>(), + looper, + clock, + eventFlagsSupplier, + iterationFinishedEvent); + } + + private ListenerSet( + CopyOnWriteArraySet> listeners, + Looper looper, + Clock clock, + Supplier eventFlagsSupplier, + IterationFinishedEvent iterationFinishedEvent) { + this.clock = clock; + this.listeners = listeners; + this.eventFlagsSupplier = eventFlagsSupplier; + this.iterationFinishedEvent = iterationFinishedEvent; + flushingEvents = new ArrayDeque<>(); + queuedEvents = new ArrayDeque<>(); + // It's safe to use "this" because we don't send a message before exiting the constructor. + @SuppressWarnings("methodref.receiver.bound.invalid") + HandlerWrapper handler = clock.createHandler(looper, this::handleMessage); + this.handler = handler; + } + + /** + * Copies the listener set. + * + * @param looper The new {@link Looper} for the copied listener set. + * @param iterationFinishedEvent The new {@link IterationFinishedEvent} sent when all other events + * sent during one {@link Looper} message queue iteration were handled by the listeners. + * @return The copied listener set. + */ + @CheckResult + public ListenerSet copy( + Looper looper, IterationFinishedEvent iterationFinishedEvent) { + return new ListenerSet<>(listeners, looper, clock, eventFlagsSupplier, iterationFinishedEvent); + } + + /** + * Adds a listener to the set. + * + *

    If a listener is already present, it will not be added again. + * + * @param listener The listener to be added. + */ + public void add(T listener) { + if (released) { + return; + } + Assertions.checkNotNull(listener); + listeners.add(new ListenerHolder<>(listener, eventFlagsSupplier)); + } + + /** + * Removes a listener from the set. + * + *

    If the listener is not present, nothing happens. + * + * @param listener The listener to be removed. + */ + public void remove(T listener) { + for (ListenerHolder listenerHolder : listeners) { + if (listenerHolder.listener.equals(listener)) { + listenerHolder.release(iterationFinishedEvent); + listeners.remove(listenerHolder); + } + } + } + + /** + * Adds an event that is sent to the listeners when {@link #flushEvents} is called. + * + * @param eventFlag An integer indicating the type of the event, or {@link C#INDEX_UNSET} to + * report this event without flag. + * @param event The event. + */ + public void queueEvent(int eventFlag, Event event) { + CopyOnWriteArraySet> listenerSnapshot = + new CopyOnWriteArraySet<>(listeners); + queuedEvents.add( + () -> { + for (ListenerHolder holder : listenerSnapshot) { + holder.invoke(eventFlag, event); + } + }); + } + + /** Notifies listeners of events previously enqueued with {@link #queueEvent(int, Event)}. */ + public void flushEvents() { + if (queuedEvents.isEmpty()) { + return; + } + if (!handler.hasMessages(MSG_ITERATION_FINISHED)) { + handler.obtainMessage(MSG_ITERATION_FINISHED).sendToTarget(); + } + boolean recursiveFlushInProgress = !flushingEvents.isEmpty(); + flushingEvents.addAll(queuedEvents); + queuedEvents.clear(); + if (recursiveFlushInProgress) { + // Recursive call to flush. Let the outer call handle the flush queue. + return; + } + while (!flushingEvents.isEmpty()) { + flushingEvents.peekFirst().run(); + flushingEvents.removeFirst(); + } + } + + /** + * {@link #queueEvent(int, Event) Queues} a single event and immediately {@link #flushEvents() + * flushes} the event queue to notify all listeners. + * + * @param eventFlag An integer flag indicating the type of the event, or {@link C#INDEX_UNSET} to + * report this event without flag. + * @param event The event. + */ + public void sendEvent(int eventFlag, Event event) { + queueEvent(eventFlag, event); + flushEvents(); + } + + /** + * Releases the set of listeners immediately. + * + *

    This will ensure no events are sent to any listener after this method has been called. + */ + public void release() { + for (ListenerHolder listenerHolder : listeners) { + listenerHolder.release(iterationFinishedEvent); + } + listeners.clear(); + released = true; + } + + /** + * Releases the set of listeners after all already scheduled {@link Looper} messages were able to + * trigger final events. + * + *

    After the specified released callback event, no other events are sent to a listener. + * + * @param releaseEventFlag An integer flag indicating the type of the release event, or {@link + * C#INDEX_UNSET} to report this event without a flag. + * @param releaseEvent The release event. + */ + public void lazyRelease(int releaseEventFlag, Event releaseEvent) { + handler.obtainMessage(MSG_LAZY_RELEASE, releaseEventFlag, 0, releaseEvent).sendToTarget(); + } + + private boolean handleMessage(Message message) { + if (message.what == MSG_ITERATION_FINISHED) { + for (ListenerHolder holder : listeners) { + holder.iterationFinished(eventFlagsSupplier, iterationFinishedEvent); + if (handler.hasMessages(MSG_ITERATION_FINISHED)) { + // The invocation above triggered new events (and thus scheduled a new message). We need + // to stop here because this new message will take care of informing every listener about + // the new update (including the ones already called here). + break; + } + } + } else if (message.what == MSG_LAZY_RELEASE) { + int releaseEventFlag = message.arg1; + @SuppressWarnings("unchecked") + Event releaseEvent = (Event) message.obj; + sendEvent(releaseEventFlag, releaseEvent); + release(); + } + return true; + } + + private static final class ListenerHolder { + + @Nonnull public final T listener; + + private E eventsFlags; + private boolean needsIterationFinishedEvent; + private boolean released; + + public ListenerHolder(@Nonnull T listener, Supplier eventFlagSupplier) { + this.listener = listener; + this.eventsFlags = eventFlagSupplier.get(); + } + + public void release(IterationFinishedEvent event) { + released = true; + if (needsIterationFinishedEvent) { + event.invoke(listener, eventsFlags); + } + } + + public void invoke(int eventFlag, Event event) { + if (!released) { + if (eventFlag != C.INDEX_UNSET) { + eventsFlags.add(eventFlag); + } + needsIterationFinishedEvent = true; + event.invoke(listener); + } + } + + public void iterationFinished( + Supplier eventFlagSupplier, IterationFinishedEvent event) { + if (!released && needsIterationFinishedEvent) { + // Reset flags before invoking the listener to ensure we keep all new flags that are set by + // recursive events triggered from this callback. + E flagToNotify = eventsFlags; + eventsFlags = eventFlagSupplier.get(); + needsIterationFinishedEvent = false; + event.invoke(listener, flagToNotify); + } + } + + @Override + public boolean equals(@Nullable Object other) { + if (this == other) { + return true; + } + if (other == null || getClass() != other.getClass()) { + return false; + } + return listener.equals(((ListenerHolder) other).listener); + } + + @Override + public int hashCode() { + return listener.hashCode(); + } + } +} diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/Log.java b/library/common/src/main/java/com/google/android/exoplayer2/util/Log.java index e5e6f88d4d9..fd1b74ca6e8 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/util/Log.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/Log.java @@ -22,6 +22,7 @@ import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.net.UnknownHostException; +import org.checkerframework.dataflow.qual.Pure; /** Wrapper around {@link android.util.Log} which allows to set the log level. */ public final class Log { @@ -51,11 +52,13 @@ public final class Log { private Log() {} /** Returns current {@link LogLevel} for ExoPlayer logcat logging. */ + @Pure public static @LogLevel int getLogLevel() { return logLevel; } /** Returns whether stack traces of {@link Throwable}s will be logged to logcat. */ + @Pure public boolean getLogStackTraces() { return logStackTraces; } @@ -80,6 +83,7 @@ public static void setLogStackTraces(boolean logStackTraces) { } /** @see android.util.Log#d(String, String) */ + @Pure public static void d(String tag, String message) { if (logLevel == LOG_LEVEL_ALL) { android.util.Log.d(tag, message); @@ -87,11 +91,13 @@ public static void d(String tag, String message) { } /** @see android.util.Log#d(String, String, Throwable) */ + @Pure public static void d(String tag, String message, @Nullable Throwable throwable) { d(tag, appendThrowableString(message, throwable)); } /** @see android.util.Log#i(String, String) */ + @Pure public static void i(String tag, String message) { if (logLevel <= LOG_LEVEL_INFO) { android.util.Log.i(tag, message); @@ -99,11 +105,13 @@ public static void i(String tag, String message) { } /** @see android.util.Log#i(String, String, Throwable) */ + @Pure public static void i(String tag, String message, @Nullable Throwable throwable) { i(tag, appendThrowableString(message, throwable)); } /** @see android.util.Log#w(String, String) */ + @Pure public static void w(String tag, String message) { if (logLevel <= LOG_LEVEL_WARNING) { android.util.Log.w(tag, message); @@ -111,11 +119,13 @@ public static void w(String tag, String message) { } /** @see android.util.Log#w(String, String, Throwable) */ + @Pure public static void w(String tag, String message, @Nullable Throwable throwable) { w(tag, appendThrowableString(message, throwable)); } /** @see android.util.Log#e(String, String) */ + @Pure public static void e(String tag, String message) { if (logLevel <= LOG_LEVEL_ERROR) { android.util.Log.e(tag, message); @@ -123,6 +133,7 @@ public static void e(String tag, String message) { } /** @see android.util.Log#e(String, String, Throwable) */ + @Pure public static void e(String tag, String message, @Nullable Throwable throwable) { e(tag, appendThrowableString(message, throwable)); } @@ -139,6 +150,7 @@ public static void e(String tag, String message, @Nullable Throwable throwable) * @return The string representation of the {@link Throwable}. */ @Nullable + @Pure public static String getThrowableString(@Nullable Throwable throwable) { if (throwable == null) { return null; @@ -157,6 +169,7 @@ public static String getThrowableString(@Nullable Throwable throwable) { } } + @Pure private static String appendThrowableString(String message, @Nullable Throwable throwable) { @Nullable String throwableString = getThrowableString(throwable); if (!TextUtils.isEmpty(throwableString)) { @@ -165,6 +178,7 @@ private static String appendThrowableString(String message, @Nullable Throwable return message; } + @Pure private static boolean isCausedByUnknownHostException(@Nullable Throwable throwable) { while (throwable != null) { if (throwable instanceof UnknownHostException) { diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/MimeTypes.java b/library/common/src/main/java/com/google/android/exoplayer2/util/MimeTypes.java index d6dd67ee7d8..13cf6b18c3a 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/util/MimeTypes.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/MimeTypes.java @@ -32,6 +32,7 @@ public final class MimeTypes { public static final String BASE_TYPE_VIDEO = "video"; public static final String BASE_TYPE_AUDIO = "audio"; public static final String BASE_TYPE_TEXT = "text"; + public static final String BASE_TYPE_IMAGE = "image"; public static final String BASE_TYPE_APPLICATION = "application"; public static final String VIDEO_MP4 = BASE_TYPE_VIDEO + "/mp4"; @@ -113,6 +114,8 @@ public final class MimeTypes { public static final String APPLICATION_ICY = BASE_TYPE_APPLICATION + "/x-icy"; public static final String APPLICATION_AIT = BASE_TYPE_APPLICATION + "/vnd.dvb.ait"; + public static final String IMAGE_JPEG = BASE_TYPE_IMAGE + "/jpeg"; + private static final ArrayList customMimeTypes = new ArrayList<>(); private static final Pattern MP4A_RFC_6381_CODEC_PATTERN = diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/MutableFlags.java b/library/common/src/main/java/com/google/android/exoplayer2/util/MutableFlags.java new file mode 100644 index 00000000000..6ed425de18d --- /dev/null +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/MutableFlags.java @@ -0,0 +1,108 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.util; + +import android.util.SparseBooleanArray; +import androidx.annotation.Nullable; + +/** + * A set of integer flags. + * + *

    Intended for usages where the number of flags may exceed 32 and can no longer be represented + * by an IntDef. + */ +public class MutableFlags { + + private final SparseBooleanArray flags; + + /** Creates the set of flags. */ + public MutableFlags() { + flags = new SparseBooleanArray(); + } + + /** Clears all previously set flags. */ + public void clear() { + flags.clear(); + } + + /** + * Adds a flag to the set. + * + * @param flag The flag to add. + */ + public void add(int flag) { + flags.append(flag, /* value= */ true); + } + + /** + * Returns whether the set contains the given flag. + * + * @param flag The flag. + * @return Whether the set contains the flag. + */ + public boolean contains(int flag) { + return flags.get(flag); + } + + /** + * Returns whether the set contains at least one of the given flags. + * + * @param flags The flags. + * @return Whether the set contains at least one of the flags. + */ + public boolean containsAny(int... flags) { + for (int flag : flags) { + if (contains(flag)) { + return true; + } + } + return false; + } + + /** Returns the number of flags in this set. */ + public int size() { + return flags.size(); + } + + /** + * Returns the flag at the given index. + * + * @param index The index. Must be between 0 (inclusive) and {@link #size()} (exclusive). + * @return The flag at the given index. + * @throws IllegalArgumentException If index is outside the allowed range. + */ + public int get(int index) { + Assertions.checkArgument(index >= 0 && index < size()); + return flags.keyAt(index); + } + + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (!(o instanceof MutableFlags)) { + return false; + } + MutableFlags that = (MutableFlags) o; + return flags.equals(that.flags); + } + + @Override + public int hashCode() { + return flags.hashCode(); + } +} diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/ParsableByteArray.java b/library/common/src/main/java/com/google/android/exoplayer2/util/ParsableByteArray.java index 413b8c540b9..dff16d39f18 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/util/ParsableByteArray.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/ParsableByteArray.java @@ -19,6 +19,7 @@ import com.google.common.base.Charsets; import java.nio.ByteBuffer; import java.nio.charset.Charset; +import java.util.Arrays; /** * Wraps a byte array, providing a set of methods for parsing data from it. Numerical values are @@ -68,8 +69,8 @@ public ParsableByteArray(byte[] data, int limit) { } /** - * Resets the position to zero and the limit to the specified value. If the limit exceeds the - * capacity, {@code data} is replaced with a new array of sufficient size. + * Resets the position to zero and the limit to the specified value. This might replace or wipe + * the {@link #getData() underlying array}, potentially invalidating any local references. * * @param limit The limit to set. */ @@ -100,8 +101,21 @@ public void reset(byte[] data, int limit) { } /** - * Returns the number of bytes yet to be read. + * Ensures the backing array is at least {@code requiredCapacity} long. + * + *

    {@link #getPosition() position}, {@link #limit() limit}, and all data in the underlying + * array (including that beyond {@link #limit()}) are preserved. + * + *

    This might replace or wipe the {@link #getData() underlying array}, potentially invalidating + * any local references. */ + public void ensureCapacity(int requiredCapacity) { + if (requiredCapacity > capacity()) { + data = Arrays.copyOf(data, requiredCapacity); + } + } + + /** Returns the number of bytes yet to be read. */ public int bytesLeft() { return limit - position; } @@ -148,8 +162,8 @@ public void setPosition(int position) { * *

    Changes to this array are reflected in the results of the {@code read...()} methods. * - *

    This reference must be assumed to become invalid when {@link #reset} is called (because the - * array might get reallocated). + *

    This reference must be assumed to become invalid when {@link #reset} or {@link + * #ensureCapacity} are called (because the array might get reallocated). */ public byte[] getData() { return data; @@ -496,11 +510,22 @@ public String readNullTerminatedString(int length) { */ @Nullable public String readNullTerminatedString() { + return readDelimiterTerminatedString('\0'); + } + + /** + * Reads up to the next delimiter byte (or the limit) as UTF-8 characters. + * + * @return The string not including any terminating delimiter byte, or null if the end of the data + * has already been reached. + */ + @Nullable + public String readDelimiterTerminatedString(char delimiter) { if (bytesLeft() == 0) { return null; } int stringLimit = position; - while (stringLimit < limit && data[stringLimit] != 0) { + while (stringLimit < limit && data[stringLimit] != delimiter) { stringLimit++; } String string = Util.fromUtf8Bytes(data, position, stringLimit - position); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/util/SystemClock.java b/library/common/src/main/java/com/google/android/exoplayer2/util/SystemClock.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/util/SystemClock.java rename to library/common/src/main/java/com/google/android/exoplayer2/util/SystemClock.java diff --git a/library/core/src/main/java/com/google/android/exoplayer2/util/SystemHandlerWrapper.java b/library/common/src/main/java/com/google/android/exoplayer2/util/SystemHandlerWrapper.java similarity index 90% rename from library/core/src/main/java/com/google/android/exoplayer2/util/SystemHandlerWrapper.java rename to library/common/src/main/java/com/google/android/exoplayer2/util/SystemHandlerWrapper.java index 1fbea2ed7eb..7b504f07798 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/util/SystemHandlerWrapper.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/SystemHandlerWrapper.java @@ -33,6 +33,11 @@ public Looper getLooper() { return handler.getLooper(); } + @Override + public boolean hasMessages(int what) { + return handler.hasMessages(what); + } + @Override public Message obtainMessage(int what) { return handler.obtainMessage(what); @@ -58,6 +63,11 @@ public boolean sendEmptyMessage(int what) { return handler.sendEmptyMessage(what); } + @Override + public boolean sendEmptyMessageDelayed(int what, int delayMs) { + return handler.sendEmptyMessageDelayed(what, delayMs); + } + @Override public boolean sendEmptyMessageAtTime(int what, long uptimeMs) { return handler.sendEmptyMessageAtTime(what, uptimeMs); diff --git a/library/common/src/main/java/com/google/android/exoplayer2/util/Util.java b/library/common/src/main/java/com/google/android/exoplayer2/util/Util.java index 61c762615cd..61907c51753 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/util/Util.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/util/Util.java @@ -49,6 +49,7 @@ import android.telephony.TelephonyManager; import android.text.TextUtils; import android.util.Base64; +import android.util.SparseLongArray; import android.view.Display; import android.view.SurfaceView; import android.view.WindowManager; @@ -82,6 +83,7 @@ import java.util.List; import java.util.Locale; import java.util.MissingResourceException; +import java.util.NoSuchElementException; import java.util.TimeZone; import java.util.UUID; import java.util.concurrent.ExecutorService; @@ -89,6 +91,7 @@ import java.util.regex.Matcher; import java.util.regex.Pattern; import java.util.zip.DataFormatException; +import java.util.zip.GZIPOutputStream; import java.util.zip.Inflater; import org.checkerframework.checker.initialization.qual.UnknownInitialization; import org.checkerframework.checker.nullness.compatqual.NullableType; @@ -104,7 +107,10 @@ public final class Util { * Like {@link android.os.Build.VERSION#SDK_INT}, but in a place where it can be conveniently * overridden for local testing. */ - public static final int SDK_INT = "R".equals(Build.VERSION.CODENAME) ? 30 : Build.VERSION.SDK_INT; + public static final int SDK_INT = + "S".equals(Build.VERSION.CODENAME) + ? 31 + : "R".equals(Build.VERSION.CODENAME) ? 30 : Build.VERSION.SDK_INT; /** * Like {@link Build#DEVICE}, but in a place where it can be conveniently overridden for local @@ -505,6 +511,10 @@ public static Handler createHandler( * run. {@code false} otherwise. */ public static boolean postOrRun(Handler handler, Runnable runnable) { + Looper looper = handler.getLooper(); + if (!looper.getThread().isAlive()) { + return false; + } if (handler.getLooper() == Looper.myLooper()) { runnable.run(); return true; @@ -1166,6 +1176,25 @@ public static int compareLong(long left, long right) { return left < right ? -1 : left == right ? 0 : 1; } + /** + * Returns the minimum value in the given {@link SparseLongArray}. + * + * @param sparseLongArray The {@link SparseLongArray}. + * @return The minimum value. + * @throws NoSuchElementException If the array is empty. + */ + @RequiresApi(18) + public static long minValue(SparseLongArray sparseLongArray) { + if (sparseLongArray.size() == 0) { + throw new NoSuchElementException(); + } + long min = Long.MAX_VALUE; + for (int i = 0; i < sparseLongArray.size(); i++) { + min = min(min, sparseLongArray.valueAt(i)); + } + return min; + } + /** * Parses an xs:duration attribute value, returning the parsed duration in milliseconds. * @@ -1335,7 +1364,7 @@ public static void scaleLargeTimestampsInPlace(long[] timestamps, long multiplie * Returns the duration of media that will elapse in {@code playoutDuration}. * * @param playoutDuration The duration to scale. - * @param speed The playback speed. + * @param speed The factor by which playback is sped up. * @return The scaled duration, in the same units as {@code playoutDuration}. */ public static long getMediaDurationForPlayoutDuration(long playoutDuration, float speed) { @@ -2017,8 +2046,6 @@ private static boolean shouldEscapeCharacter(char c) { /** Returns a data URI with the specified MIME type and data. */ public static Uri getDataUriForString(String mimeType, String data) { - // TODO(internal: b/169937045): For now we don't pass the URL_SAFE flag as DataSchemeDataSource - // doesn't decode using it. return Uri.parse( "data:" + mimeType + ";base64," + Base64.encodeToString(data.getBytes(), Base64.NO_WRAP)); } @@ -2057,7 +2084,7 @@ public static File createTempDirectory(Context context, String prefix) throws IO /** Creates a new empty file in the directory returned by {@link Context#getCacheDir()}. */ public static File createTempFile(Context context, String prefix) throws IOException { - return File.createTempFile(prefix, null, context.getCacheDir()); + return File.createTempFile(prefix, null, checkNotNull(context.getCacheDir())); } /** @@ -2095,6 +2122,17 @@ public static int crc8(byte[] bytes, int start, int end, int initialValue) { return initialValue; } + /** Compresses {@code input} using gzip and returns the result in a newly allocated byte array. */ + public static byte[] gzip(byte[] input) { + ByteArrayOutputStream output = new ByteArrayOutputStream(); + try (GZIPOutputStream os = new GZIPOutputStream(output)) { + os.write(input); + } catch (IOException e) { + throw new AssertionError(e); + } + return output.toByteArray(); + } + /** * Absolute get method for reading an int value in {@link ByteOrder#BIG_ENDIAN} in a {@link * ByteBuffer}. Same as {@link ByteBuffer#getInt(int)} except the buffer's order as returned by @@ -2151,7 +2189,7 @@ public static int getNetworkType(Context context) { return getMobileNetworkType(networkInfo); case ConnectivityManager.TYPE_ETHERNET: return C.NETWORK_TYPE_ETHERNET; - default: // VPN, Bluetooth, Dummy. + default: return C.NETWORK_TYPE_OTHER; } } @@ -2207,9 +2245,8 @@ public static boolean inflate( if (input.bytesLeft() <= 0) { return false; } - byte[] outputData = output.getData(); - if (outputData.length < input.bytesLeft()) { - outputData = new byte[2 * input.bytesLeft()]; + if (output.capacity() < input.bytesLeft()) { + output.ensureCapacity(2 * input.bytesLeft()); } if (inflater == null) { inflater = new Inflater(); @@ -2218,16 +2255,17 @@ public static boolean inflate( try { int outputSize = 0; while (true) { - outputSize += inflater.inflate(outputData, outputSize, outputData.length - outputSize); + outputSize += + inflater.inflate(output.getData(), outputSize, output.capacity() - outputSize); if (inflater.finished()) { - output.reset(outputData, outputSize); + output.setLimit(outputSize); return true; } if (inflater.needsDictionary() || inflater.needsInput()) { return false; } - if (outputSize == outputData.length) { - outputData = Arrays.copyOf(outputData, outputData.length * 2); + if (outputSize == output.capacity()) { + output.ensureCapacity(output.capacity() * 2); } } } catch (DataFormatException e) { @@ -2582,7 +2620,7 @@ private static String maybeReplaceLegacyLanguageTags(String languageTag) { "hsn", "zh-hsn" }; - // Legacy ("grandfathered") tags, replaced by modern equivalents (including macrolanguage) + // Legacy tags that have been replaced by modern equivalents (including macrolanguage) // See https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry. private static final String[] isoLegacyTagReplacements = new String[] { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/util/XmlPullParserUtil.java b/library/common/src/main/java/com/google/android/exoplayer2/util/XmlPullParserUtil.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/util/XmlPullParserUtil.java rename to library/common/src/main/java/com/google/android/exoplayer2/util/XmlPullParserUtil.java diff --git a/library/common/src/main/java/com/google/android/exoplayer2/video/AvcConfig.java b/library/common/src/main/java/com/google/android/exoplayer2/video/AvcConfig.java index b794d2db905..183cfe09dfa 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/video/AvcConfig.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/video/AvcConfig.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.video; +import androidx.annotation.Nullable; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.ParserException; import com.google.android.exoplayer2.util.CodecSpecificDataUtil; @@ -34,13 +35,14 @@ public final class AvcConfig { public final int width; public final int height; public final float pixelWidthAspectRatio; + @Nullable public final String codecs; /** * Parses AVC configuration data. * * @param data A {@link ParsableByteArray}, whose position is set to the start of the AVC * configuration data to parse. - * @return A parsed representation of the HEVC configuration data. + * @return A parsed representation of the AVC configuration data. * @throws ParserException If an error occurred parsing the data. */ public static AvcConfig parse(ParsableByteArray data) throws ParserException { @@ -63,6 +65,7 @@ public static AvcConfig parse(ParsableByteArray data) throws ParserException { int width = Format.NO_VALUE; int height = Format.NO_VALUE; float pixelWidthAspectRatio = 1; + @Nullable String codecs = null; if (numSequenceParameterSets > 0) { byte[] sps = initializationData.get(0); SpsData spsData = NalUnitUtil.parseSpsNalUnit(initializationData.get(0), @@ -70,21 +73,36 @@ public static AvcConfig parse(ParsableByteArray data) throws ParserException { width = spsData.width; height = spsData.height; pixelWidthAspectRatio = spsData.pixelWidthAspectRatio; + codecs = + CodecSpecificDataUtil.buildAvcCodecString( + spsData.profileIdc, spsData.constraintsFlagsAndReservedZero2Bits, spsData.levelIdc); } - return new AvcConfig(initializationData, nalUnitLengthFieldLength, width, height, - pixelWidthAspectRatio); + + return new AvcConfig( + initializationData, + nalUnitLengthFieldLength, + width, + height, + pixelWidthAspectRatio, + codecs); } catch (ArrayIndexOutOfBoundsException e) { throw new ParserException("Error parsing AVC config", e); } } - private AvcConfig(List initializationData, int nalUnitLengthFieldLength, - int width, int height, float pixelWidthAspectRatio) { + private AvcConfig( + List initializationData, + int nalUnitLengthFieldLength, + int width, + int height, + float pixelWidthAspectRatio, + @Nullable String codecs) { this.initializationData = initializationData; this.nalUnitLengthFieldLength = nalUnitLengthFieldLength; this.width = width; this.height = height; this.pixelWidthAspectRatio = pixelWidthAspectRatio; + this.codecs = codecs; } private static byte[] buildNalUnitForChild(ParsableByteArray data) { diff --git a/library/common/src/main/java/com/google/android/exoplayer2/video/DolbyVisionConfig.java b/library/common/src/main/java/com/google/android/exoplayer2/video/DolbyVisionConfig.java index 3a13540e12f..9321330061e 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/video/DolbyVisionConfig.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/video/DolbyVisionConfig.java @@ -45,7 +45,7 @@ public static DolbyVisionConfig parse(ParsableByteArray data) { } else { return null; } - String codecs = codecsPrefix + ".0" + dvProfile + ".0" + dvLevel; + String codecs = codecsPrefix + ".0" + dvProfile + (dvLevel < 10 ? ".0" : ".") + dvLevel; return new DolbyVisionConfig(dvProfile, dvLevel, codecs); } diff --git a/library/common/src/main/java/com/google/android/exoplayer2/video/HevcConfig.java b/library/common/src/main/java/com/google/android/exoplayer2/video/HevcConfig.java index 100a824a970..9ef12a5c336 100644 --- a/library/common/src/main/java/com/google/android/exoplayer2/video/HevcConfig.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/video/HevcConfig.java @@ -17,8 +17,10 @@ import androidx.annotation.Nullable; import com.google.android.exoplayer2.ParserException; +import com.google.android.exoplayer2.util.CodecSpecificDataUtil; import com.google.android.exoplayer2.util.NalUnitUtil; import com.google.android.exoplayer2.util.ParsableByteArray; +import com.google.android.exoplayer2.util.ParsableNalUnitBitArray; import java.util.Collections; import java.util.List; @@ -27,9 +29,6 @@ */ public final class HevcConfig { - @Nullable public final List initializationData; - public final int nalUnitLengthFieldLength; - /** * Parses HEVC configuration data. * @@ -61,8 +60,9 @@ public static HevcConfig parse(ParsableByteArray data) throws ParserException { data.setPosition(csdStartPosition); byte[] buffer = new byte[csdLength]; int bufferPosition = 0; + @Nullable String codecs = null; for (int i = 0; i < numberOfArrays; i++) { - data.skipBytes(1); // completeness (1), nal_unit_type (7) + int nalUnitType = data.readUnsignedByte() & 0x7F; // completeness (1), nal_unit_type (7) int numberOfNalUnits = data.readUnsignedShort(); for (int j = 0; j < numberOfNalUnits; j++) { int nalUnitLength = data.readUnsignedShort(); @@ -71,21 +71,52 @@ public static HevcConfig parse(ParsableByteArray data) throws ParserException { bufferPosition += NalUnitUtil.NAL_START_CODE.length; System.arraycopy( data.getData(), data.getPosition(), buffer, bufferPosition, nalUnitLength); + if (nalUnitType == SPS_NAL_UNIT_TYPE && j == 0) { + ParsableNalUnitBitArray bitArray = + new ParsableNalUnitBitArray( + buffer, + /* offset= */ bufferPosition, + /* limit= */ bufferPosition + nalUnitLength); + codecs = CodecSpecificDataUtil.buildHevcCodecStringFromSps(bitArray); + } bufferPosition += nalUnitLength; data.skipBytes(nalUnitLength); } } + @Nullable List initializationData = csdLength == 0 ? null : Collections.singletonList(buffer); - return new HevcConfig(initializationData, lengthSizeMinusOne + 1); + return new HevcConfig(initializationData, lengthSizeMinusOne + 1, codecs); } catch (ArrayIndexOutOfBoundsException e) { throw new ParserException("Error parsing HEVC config", e); } } - private HevcConfig(@Nullable List initializationData, int nalUnitLengthFieldLength) { + private static final int SPS_NAL_UNIT_TYPE = 33; + + /** + * List of buffers containing the codec-specific data to be provided to the decoder, or {@code + * null} if not known. + * + * @see com.google.android.exoplayer2.Format#initializationData + */ + @Nullable public final List initializationData; + /** The length of the NAL unit length field in the bitstream's container, in bytes. */ + public final int nalUnitLengthFieldLength; + /** + * An RFC 6381 codecs string representing the video format, or {@code null} if not known. + * + * @see com.google.android.exoplayer2.Format#codecs + */ + @Nullable public final String codecs; + + private HevcConfig( + @Nullable List initializationData, + int nalUnitLengthFieldLength, + @Nullable String codecs) { this.initializationData = initializationData; this.nalUnitLengthFieldLength = nalUnitLengthFieldLength; + this.codecs = codecs; } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/VideoFrameMetadataListener.java b/library/common/src/main/java/com/google/android/exoplayer2/video/VideoFrameMetadataListener.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/video/VideoFrameMetadataListener.java rename to library/common/src/main/java/com/google/android/exoplayer2/video/VideoFrameMetadataListener.java diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/VideoListener.java b/library/common/src/main/java/com/google/android/exoplayer2/video/VideoListener.java similarity index 86% rename from library/core/src/main/java/com/google/android/exoplayer2/video/VideoListener.java rename to library/common/src/main/java/com/google/android/exoplayer2/video/VideoListener.java index 589371cde5d..eb013ed4253 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/video/VideoListener.java +++ b/library/common/src/main/java/com/google/android/exoplayer2/video/VideoListener.java @@ -15,6 +15,8 @@ */ package com.google.android.exoplayer2.video; +import com.google.android.exoplayer2.C; + /** A listener for metadata corresponding to video being rendered. */ public interface VideoListener { @@ -41,12 +43,10 @@ default void onVideoSizeChanged( * Called each time there's a change in the size of the surface onto which the video is being * rendered. * - * @param width The surface width in pixels. May be {@link - * com.google.android.exoplayer2.C#LENGTH_UNSET} if unknown, or 0 if the video is not rendered - * onto a surface. - * @param height The surface height in pixels. May be {@link - * com.google.android.exoplayer2.C#LENGTH_UNSET} if unknown, or 0 if the video is not rendered - * onto a surface. + * @param width The surface width in pixels. May be {@link C#LENGTH_UNSET} if unknown, or 0 if the + * video is not rendered onto a surface. + * @param height The surface height in pixels. May be {@link C#LENGTH_UNSET} if unknown, or 0 if + * the video is not rendered onto a surface. */ default void onSurfaceSizeChanged(int width, int height) {} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/spherical/CameraMotionListener.java b/library/common/src/main/java/com/google/android/exoplayer2/video/spherical/CameraMotionListener.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/video/spherical/CameraMotionListener.java rename to library/common/src/main/java/com/google/android/exoplayer2/video/spherical/CameraMotionListener.java diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/spherical/package-info.java b/library/common/src/main/java/com/google/android/exoplayer2/video/spherical/package-info.java similarity index 100% rename from library/core/src/main/java/com/google/android/exoplayer2/video/spherical/package-info.java rename to library/common/src/main/java/com/google/android/exoplayer2/video/spherical/package-info.java diff --git a/library/common/src/test/java/com/google/android/exoplayer2/MediaItemTest.java b/library/common/src/test/java/com/google/android/exoplayer2/MediaItemTest.java index 5d00b1e3dd9..0243fe50f47 100644 --- a/library/common/src/test/java/com/google/android/exoplayer2/MediaItemTest.java +++ b/library/common/src/test/java/com/google/android/exoplayer2/MediaItemTest.java @@ -281,7 +281,20 @@ public void builderSetAdTagUri_setsAdTagUri() { MediaItem mediaItem = new MediaItem.Builder().setUri(URI_STRING).setAdTagUri(adTagUri).build(); - assertThat(mediaItem.playbackProperties.adTagUri).isEqualTo(adTagUri); + assertThat(mediaItem.playbackProperties.adsConfiguration.adTagUri).isEqualTo(adTagUri); + assertThat(mediaItem.playbackProperties.adsConfiguration.adsId).isNull(); + } + + @Test + public void builderSetAdTagUriAndAdsId_setsAdsConfiguration() { + Uri adTagUri = Uri.parse(URI_STRING + "/ad"); + Object adsId = new Object(); + + MediaItem mediaItem = + new MediaItem.Builder().setUri(URI_STRING).setAdTagUri(adTagUri, adsId).build(); + + assertThat(mediaItem.playbackProperties.adsConfiguration.adTagUri).isEqualTo(adTagUri); + assertThat(mediaItem.playbackProperties.adsConfiguration.adsId).isEqualTo(adsId); } @Test @@ -294,6 +307,46 @@ public void builderSetMediaMetadata_setsMetadata() { assertThat(mediaItem.mediaMetadata).isEqualTo(mediaMetadata); } + @Test + public void builderSetLiveTargetOffsetMs_setsLiveTargetOffsetMs() { + MediaItem mediaItem = + new MediaItem.Builder().setUri(URI_STRING).setLiveTargetOffsetMs(10_000).build(); + + assertThat(mediaItem.liveConfiguration.targetOffsetMs).isEqualTo(10_000); + } + + @Test + public void builderSetMinLivePlaybackSpeed_setsMinLivePlaybackSpeed() { + MediaItem mediaItem = + new MediaItem.Builder().setUri(URI_STRING).setLiveMinPlaybackSpeed(.9f).build(); + + assertThat(mediaItem.liveConfiguration.minPlaybackSpeed).isEqualTo(.9f); + } + + @Test + public void builderSetMaxLivePlaybackSpeed_setsMaxLivePlaybackSpeed() { + MediaItem mediaItem = + new MediaItem.Builder().setUri(URI_STRING).setLiveMaxPlaybackSpeed(1.1f).build(); + + assertThat(mediaItem.liveConfiguration.maxPlaybackSpeed).isEqualTo(1.1f); + } + + @Test + public void builderSetMinLiveOffset_setsMinLiveOffset() { + MediaItem mediaItem = + new MediaItem.Builder().setUri(URI_STRING).setLiveMinOffsetMs(1234).build(); + + assertThat(mediaItem.liveConfiguration.minOffsetMs).isEqualTo(1234); + } + + @Test + public void builderSetMaxLiveOffset_setsMaxLiveOffset() { + MediaItem mediaItem = + new MediaItem.Builder().setUri(URI_STRING).setLiveMaxOffsetMs(1234).build(); + + assertThat(mediaItem.liveConfiguration.maxOffsetMs).isEqualTo(1234); + } + @Test public void buildUpon_equalsToOriginal() { MediaItem mediaItem = @@ -319,6 +372,11 @@ public void buildUpon_equalsToOriginal() { .setMimeType(MimeTypes.APPLICATION_MP4) .setUri(URI_STRING) .setStreamKeys(Collections.singletonList(new StreamKey(1, 0, 0))) + .setLiveTargetOffsetMs(20_000) + .setLiveMinPlaybackSpeed(.9f) + .setLiveMaxPlaybackSpeed(1.1f) + .setLiveMinOffsetMs(2222) + .setLiveMaxOffsetMs(4444) .setSubtitles( Collections.singletonList( new MediaItem.Subtitle( diff --git a/library/common/src/test/java/com/google/android/exoplayer2/drm/DrmInitDataTest.java b/library/common/src/test/java/com/google/android/exoplayer2/drm/DrmInitDataTest.java index f1966413322..e292bfba782 100644 --- a/library/common/src/test/java/com/google/android/exoplayer2/drm/DrmInitDataTest.java +++ b/library/common/src/test/java/com/google/android/exoplayer2/drm/DrmInitDataTest.java @@ -93,34 +93,6 @@ public void equals() { assertThat(testInitData).isNotEqualTo(drmInitData); } - @Test - @SuppressWarnings("deprecation") - public void getByUuid() { - // Basic matching. - DrmInitData testInitData = new DrmInitData(DATA_1, DATA_2); - assertThat(testInitData.get(WIDEVINE_UUID)).isEqualTo(DATA_1); - assertThat(testInitData.get(PLAYREADY_UUID)).isEqualTo(DATA_2); - assertThat(testInitData.get(UUID_NIL)).isNull(); - - // Basic matching including universal data. - testInitData = new DrmInitData(DATA_1, DATA_2, DATA_UNIVERSAL); - assertThat(testInitData.get(WIDEVINE_UUID)).isEqualTo(DATA_1); - assertThat(testInitData.get(PLAYREADY_UUID)).isEqualTo(DATA_2); - assertThat(testInitData.get(UUID_NIL)).isEqualTo(DATA_UNIVERSAL); - - // Passing the scheme data in reverse order shouldn't affect equality. - testInitData = new DrmInitData(DATA_UNIVERSAL, DATA_2, DATA_1); - assertThat(testInitData.get(WIDEVINE_UUID)).isEqualTo(DATA_1); - assertThat(testInitData.get(PLAYREADY_UUID)).isEqualTo(DATA_2); - assertThat(testInitData.get(UUID_NIL)).isEqualTo(DATA_UNIVERSAL); - - // Universal data should be returned in the absence of a specific match. - testInitData = new DrmInitData(DATA_1, DATA_UNIVERSAL); - assertThat(testInitData.get(WIDEVINE_UUID)).isEqualTo(DATA_1); - assertThat(testInitData.get(PLAYREADY_UUID)).isEqualTo(DATA_UNIVERSAL); - assertThat(testInitData.get(UUID_NIL)).isEqualTo(DATA_UNIVERSAL); - } - @Test public void getByIndex() { DrmInitData testInitData = new DrmInitData(DATA_1, DATA_2); @@ -128,12 +100,10 @@ public void getByIndex() { } @Test - @SuppressWarnings("deprecation") public void schemeDatasWithSameUuid() { DrmInitData testInitData = new DrmInitData(DATA_1, DATA_1B); + assertThat(testInitData.schemeDataCount).isEqualTo(2); - // Deprecated get method should return first entry. - assertThat(testInitData.get(WIDEVINE_UUID)).isEqualTo(DATA_1); // Test retrieval of first and second entry. assertThat(testInitData.get(0)).isEqualTo(DATA_1); assertThat(testInitData.get(1)).isEqualTo(DATA_1B); diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/MdtaMetadataEntryTest.java b/library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/MdtaMetadataEntryTest.java similarity index 96% rename from library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/MdtaMetadataEntryTest.java rename to library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/MdtaMetadataEntryTest.java index 6fba801355d..15b2c112e60 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/MdtaMetadataEntryTest.java +++ b/library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/MdtaMetadataEntryTest.java @@ -13,7 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.google.android.exoplayer2.extractor.mp4; +package com.google.android.exoplayer2.metadata.mp4; import static com.google.common.truth.Truth.assertThat; diff --git a/library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/MotionPhotoMetadataTest.java b/library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/MotionPhotoMetadataTest.java new file mode 100644 index 00000000000..81c36696769 --- /dev/null +++ b/library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/MotionPhotoMetadataTest.java @@ -0,0 +1,51 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.metadata.mp4; + +import static com.google.common.truth.Truth.assertThat; + +import android.os.Parcel; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Test for {@link MotionPhotoMetadata}. */ +@RunWith(AndroidJUnit4.class) +public class MotionPhotoMetadataTest { + + @Test + public void parcelable() { + MotionPhotoMetadata motionPhotoMetadataToParcel = + new MotionPhotoMetadata( + /* photoStartPosition= */ 0, + /* photoSize= */ 10, + /* photoPresentationTimestampUs= */ C.TIME_UNSET, + /* videoStartPosition= */ 15, + /* videoSize= */ 35); + + Parcel parcel = Parcel.obtain(); + motionPhotoMetadataToParcel.writeToParcel(parcel, /* flags= */ 0); + parcel.setDataPosition(0); + + MotionPhotoMetadata motionPhotoMetadataFromParcel = + MotionPhotoMetadata.CREATOR.createFromParcel(parcel); + assertThat(motionPhotoMetadataFromParcel).isEqualTo(motionPhotoMetadataToParcel); + + parcel.recycle(); + } +} diff --git a/library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/SmtaMetadataEntryTest.java b/library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/SmtaMetadataEntryTest.java new file mode 100644 index 00000000000..7cc48a80211 --- /dev/null +++ b/library/common/src/test/java/com/google/android/exoplayer2/metadata/mp4/SmtaMetadataEntryTest.java @@ -0,0 +1,44 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.metadata.mp4; + +import static com.google.common.truth.Truth.assertThat; + +import android.os.Parcel; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Test for {@link SmtaMetadataEntry}. */ +@RunWith(AndroidJUnit4.class) +public class SmtaMetadataEntryTest { + + @Test + public void parcelable() { + SmtaMetadataEntry smtaMetadataEntryToParcel = + new SmtaMetadataEntry(/* captureFrameRate= */ 120, /* svcTemporalLayerCount= */ 4); + + Parcel parcel = Parcel.obtain(); + smtaMetadataEntryToParcel.writeToParcel(parcel, /* flags= */ 0); + parcel.setDataPosition(0); + + SmtaMetadataEntry smtaMetadataEntryFromParcel = + SmtaMetadataEntry.CREATOR.createFromParcel(parcel); + assertThat(smtaMetadataEntryFromParcel).isEqualTo(smtaMetadataEntryToParcel); + + parcel.recycle(); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/BaseDataSourceTest.java b/library/common/src/test/java/com/google/android/exoplayer2/upstream/BaseDataSourceTest.java similarity index 100% rename from library/core/src/test/java/com/google/android/exoplayer2/upstream/BaseDataSourceTest.java rename to library/common/src/test/java/com/google/android/exoplayer2/upstream/BaseDataSourceTest.java diff --git a/library/common/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceContractTest.java b/library/common/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceContractTest.java new file mode 100644 index 00000000000..0fb570f8b94 --- /dev/null +++ b/library/common/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceContractTest.java @@ -0,0 +1,46 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.upstream; + +import android.net.Uri; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.HttpDataSourceTestEnv; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link DefaultHttpDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class DefaultHttpDataSourceContractTest extends DataSourceContractTest { + + @Rule public HttpDataSourceTestEnv httpDataSourceTestEnv = new HttpDataSourceTestEnv(); + + @Override + protected DataSource createDataSource() { + return new DefaultHttpDataSource.Factory().createDataSource(); + } + + @Override + protected ImmutableList getTestResources() { + return httpDataSourceTestEnv.getServedResources(); + } + + @Override + protected Uri getNotFoundUri() { + return Uri.parse(httpDataSourceTestEnv.getNonexistentUrl()); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceTest.java b/library/common/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceTest.java similarity index 73% rename from library/core/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceTest.java rename to library/common/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceTest.java index 8d5a7479e58..f84b4489774 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceTest.java +++ b/library/common/src/test/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceTest.java @@ -57,19 +57,18 @@ public void open_withSpecifiedRequestParameters_usesCorrectParameters() throws E MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); - String propertyFromConstructor = "fromConstructor"; - HttpDataSource.RequestProperties constructorProperties = new HttpDataSource.RequestProperties(); - constructorProperties.set("0", propertyFromConstructor); - constructorProperties.set("1", propertyFromConstructor); - constructorProperties.set("2", propertyFromConstructor); - constructorProperties.set("4", propertyFromConstructor); + String propertyFromFactory = "fromFactory"; + Map defaultRequestProperties = new HashMap<>(); + defaultRequestProperties.put("0", propertyFromFactory); + defaultRequestProperties.put("1", propertyFromFactory); + defaultRequestProperties.put("2", propertyFromFactory); + defaultRequestProperties.put("4", propertyFromFactory); DefaultHttpDataSource dataSource = - new DefaultHttpDataSource( - /* userAgent= */ "testAgent", - /* connectTimeoutMillis= */ 1000, - /* readTimeoutMillis= */ 1000, - /* allowCrossProtocolRedirects= */ false, - constructorProperties); + new DefaultHttpDataSource.Factory() + .setConnectTimeoutMs(1000) + .setReadTimeoutMs(1000) + .setDefaultRequestProperties(defaultRequestProperties) + .createDataSource(); String propertyFromSetter = "fromSetter"; dataSource.setRequestProperty("1", propertyFromSetter); @@ -92,7 +91,7 @@ public void open_withSpecifiedRequestParameters_usesCorrectParameters() throws E dataSource.open(dataSpec); Headers headers = mockWebServer.takeRequest(10, SECONDS).getHeaders(); - assertThat(headers.get("0")).isEqualTo(propertyFromConstructor); + assertThat(headers.get("0")).isEqualTo(propertyFromFactory); assertThat(headers.get("1")).isEqualTo(propertyFromSetter); assertThat(headers.get("2")).isEqualTo(propertyFromDataSpec); assertThat(headers.get("3")).isEqualTo(propertyFromDataSpec); @@ -102,14 +101,12 @@ public void open_withSpecifiedRequestParameters_usesCorrectParameters() throws E } @Test - public void open_invalidResponseCode() throws Exception { + public void open_invalidResponseCode() { DefaultHttpDataSource defaultHttpDataSource = - new DefaultHttpDataSource( - /* userAgent= */ "testAgent", - /* connectTimeoutMillis= */ 1000, - /* readTimeoutMillis= */ 1000, - /* allowCrossProtocolRedirects= */ false, - /* defaultRequestProperties= */ null); + new DefaultHttpDataSource.Factory() + .setConnectTimeoutMs(1000) + .setReadTimeoutMs(1000) + .createDataSource(); MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue( @@ -128,4 +125,22 @@ public void open_invalidResponseCode() throws Exception { assertThat(exception.responseCode).isEqualTo(404); assertThat(exception.responseBody).isEqualTo(TestUtil.createByteArray(1, 2, 3)); } + + @Test + public void factory_setRequestPropertyAfterCreation_setsCorrectHeaders() throws Exception { + MockWebServer mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse()); + DataSpec dataSpec = + new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build(); + DefaultHttpDataSource.Factory factory = new DefaultHttpDataSource.Factory(); + HttpDataSource dataSource = factory.createDataSource(); + + Map defaultRequestProperties = new HashMap<>(); + defaultRequestProperties.put("0", "afterCreation"); + factory.setDefaultRequestProperties(defaultRequestProperties); + dataSource.open(dataSpec); + + Headers headers = mockWebServer.takeRequest(10, SECONDS).getHeaders(); + assertThat(headers.get("0")).isEqualTo("afterCreation"); + } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/util/ConditionVariableTest.java b/library/common/src/test/java/com/google/android/exoplayer2/util/ConditionVariableTest.java similarity index 100% rename from library/core/src/test/java/com/google/android/exoplayer2/util/ConditionVariableTest.java rename to library/common/src/test/java/com/google/android/exoplayer2/util/ConditionVariableTest.java diff --git a/library/common/src/test/java/com/google/android/exoplayer2/util/ListenerSetTest.java b/library/common/src/test/java/com/google/android/exoplayer2/util/ListenerSetTest.java new file mode 100644 index 00000000000..7d0b77664d2 --- /dev/null +++ b/library/common/src/test/java/com/google/android/exoplayer2/util/ListenerSetTest.java @@ -0,0 +1,431 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.util; + +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.spy; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoMoreInteractions; + +import android.os.Handler; +import android.os.Looper; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.InOrder; +import org.mockito.Mockito; +import org.robolectric.shadows.ShadowLooper; + +/** Unit test for {@link ListenerSet}. */ +@RunWith(AndroidJUnit4.class) +public class ListenerSetTest { + + private static final int EVENT_ID_1 = 0; + private static final int EVENT_ID_2 = 1; + private static final int EVENT_ID_3 = 2; + + @Test + public void queueEvent_withoutFlush_sendsNoEvents() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener = mock(TestListener.class); + listenerSet.add(listener); + + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.queueEvent(EVENT_ID_2, TestListener::callback2); + ShadowLooper.runMainLooperToNextTask(); + + verifyNoMoreInteractions(listener); + } + + @Test + public void flushEvents_sendsPreviouslyQueuedEventsToAllListeners() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener1 = mock(TestListener.class); + TestListener listener2 = mock(TestListener.class); + listenerSet.add(listener1); + listenerSet.add(listener2); + + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.queueEvent(EVENT_ID_2, TestListener::callback2); + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.flushEvents(); + + InOrder inOrder = Mockito.inOrder(listener1, listener2); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener2).callback1(); + inOrder.verify(listener1).callback2(); + inOrder.verify(listener2).callback2(); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener2).callback1(); + inOrder.verifyNoMoreInteractions(); + } + + @Test + public void flushEvents_recursive_sendsEventsInCorrectOrder() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + // Listener1 sends callback3 recursively when receiving callback1. + TestListener listener1 = + spy( + new TestListener() { + @Override + public void callback1() { + listenerSet.queueEvent(EVENT_ID_3, TestListener::callback3); + listenerSet.flushEvents(); + } + }); + TestListener listener2 = mock(TestListener.class); + listenerSet.add(listener1); + listenerSet.add(listener2); + + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.queueEvent(EVENT_ID_2, TestListener::callback2); + listenerSet.flushEvents(); + + InOrder inOrder = Mockito.inOrder(listener1, listener2); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener2).callback1(); + inOrder.verify(listener1).callback2(); + inOrder.verify(listener2).callback2(); + inOrder.verify(listener1).callback3(); + inOrder.verify(listener2).callback3(); + inOrder.verifyNoMoreInteractions(); + } + + @Test + public void + flushEvents_withMultipleMessageQueueIterations_sendsIterationFinishedEventPerIteration() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + // Listener1 sends callback1 recursively when receiving callback3. + TestListener listener1 = + spy( + new TestListener() { + @Override + public void callback3() { + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + } + }); + TestListener listener2 = mock(TestListener.class); + listenerSet.add(listener1); + listenerSet.add(listener2); + + // Iteration with single flush. + listenerSet.queueEvent(EVENT_ID_2, TestListener::callback2); + listenerSet.flushEvents(); + ShadowLooper.runMainLooperToNextTask(); + + // Iteration with multiple flushes. + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.queueEvent(EVENT_ID_2, TestListener::callback2); + listenerSet.flushEvents(); + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.flushEvents(); + ShadowLooper.runMainLooperToNextTask(); + + // Iteration with recursive call. + listenerSet.sendEvent(EVENT_ID_3, TestListener::callback3); + ShadowLooper.runMainLooperToNextTask(); + + InOrder inOrder = Mockito.inOrder(listener1, listener2); + inOrder.verify(listener1).callback2(); + inOrder.verify(listener2).callback2(); + inOrder.verify(listener1).iterationFinished(Flags.create(EVENT_ID_2)); + inOrder.verify(listener2).iterationFinished(Flags.create(EVENT_ID_2)); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener2).callback1(); + inOrder.verify(listener1).callback2(); + inOrder.verify(listener2).callback2(); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener2).callback1(); + inOrder.verify(listener1).iterationFinished(Flags.create(EVENT_ID_1, EVENT_ID_2)); + inOrder.verify(listener2).iterationFinished(Flags.create(EVENT_ID_1, EVENT_ID_2)); + inOrder.verify(listener1).callback3(); + inOrder.verify(listener2).callback3(); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener2).callback1(); + inOrder.verify(listener1).iterationFinished(Flags.create(EVENT_ID_1, EVENT_ID_3)); + inOrder.verify(listener2).iterationFinished(Flags.create(EVENT_ID_1, EVENT_ID_3)); + inOrder.verifyNoMoreInteractions(); + } + + @Test + public void flushEvents_calledFromIterationFinishedCallback_restartsIterationFinishedEvents() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + // Listener2 sends callback1 recursively when receiving the iteration finished event. + TestListener listener2 = + spy( + new TestListener() { + boolean eventSent; + + @Override + public void iterationFinished(Flags flags) { + if (!eventSent) { + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + eventSent = true; + } + } + }); + TestListener listener1 = mock(TestListener.class); + TestListener listener3 = mock(TestListener.class); + listenerSet.add(listener1); + listenerSet.add(listener2); + listenerSet.add(listener3); + + listenerSet.sendEvent(EVENT_ID_2, TestListener::callback2); + ShadowLooper.runMainLooperToNextTask(); + + InOrder inOrder = Mockito.inOrder(listener1, listener2, listener3); + inOrder.verify(listener1).callback2(); + inOrder.verify(listener2).callback2(); + inOrder.verify(listener3).callback2(); + inOrder.verify(listener1).iterationFinished(Flags.create(EVENT_ID_2)); + inOrder.verify(listener2).iterationFinished(Flags.create(EVENT_ID_2)); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener2).callback1(); + inOrder.verify(listener3).callback1(); + inOrder.verify(listener1).iterationFinished(Flags.create(EVENT_ID_1)); + inOrder.verify(listener2).iterationFinished(Flags.create(EVENT_ID_1)); + inOrder.verify(listener3).iterationFinished(Flags.create(EVENT_ID_1, EVENT_ID_2)); + inOrder.verifyNoMoreInteractions(); + } + + @Test + public void flushEvents_withUnsetEventFlag_doesNotThrow() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + + listenerSet.queueEvent(/* eventFlag= */ C.INDEX_UNSET, TestListener::callback1); + listenerSet.flushEvents(); + ShadowLooper.runMainLooperToNextTask(); + + // Asserts that negative event flag (INDEX_UNSET) can be used without throwing. + } + + @Test + public void add_withRecursion_onlyReceivesUpdatesForFutureEvents() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener2 = mock(TestListener.class); + // Listener1 adds listener2 recursively. + TestListener listener1 = + spy( + new TestListener() { + @Override + public void callback1() { + listenerSet.add(listener2); + } + }); + + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.add(listener1); + // This should add listener2, but the event should not be received yet as it happened before + // listener2 was added. + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.sendEvent(EVENT_ID_2, TestListener::callback2); + ShadowLooper.runMainLooperToNextTask(); + + InOrder inOrder = Mockito.inOrder(listener1, listener2); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener1).callback2(); + inOrder.verify(listener2).callback2(); + inOrder.verify(listener1).iterationFinished(Flags.create(EVENT_ID_1, EVENT_ID_2)); + inOrder.verify(listener2).iterationFinished(Flags.create(EVENT_ID_2)); + inOrder.verifyNoMoreInteractions(); + } + + @Test + public void add_withQueueing_onlyReceivesUpdatesForFutureEvents() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener1 = mock(TestListener.class); + TestListener listener2 = mock(TestListener.class); + + // This event is flushed after listener2 was added, but shouldn't be sent to listener2 because + // the event itself occurred before the listener was added. + listenerSet.add(listener1); + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.add(listener2); + listenerSet.queueEvent(EVENT_ID_2, TestListener::callback2); + listenerSet.flushEvents(); + ShadowLooper.runMainLooperToNextTask(); + + InOrder inOrder = Mockito.inOrder(listener1, listener2); + inOrder.verify(listener1).callback1(); + inOrder.verify(listener1).callback2(); + inOrder.verify(listener2).callback2(); + inOrder.verify(listener1).iterationFinished(Flags.create(EVENT_ID_1, EVENT_ID_2)); + inOrder.verify(listener2).iterationFinished(Flags.create(EVENT_ID_2)); + inOrder.verifyNoMoreInteractions(); + } + + @Test + public void remove_withRecursion_stopsReceivingEventsImmediately() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener2 = mock(TestListener.class); + // Listener1 removes listener2 recursively. + TestListener listener1 = + spy( + new TestListener() { + @Override + public void callback1() { + listenerSet.remove(listener2); + } + }); + listenerSet.add(listener1); + listenerSet.add(listener2); + + // Listener2 shouldn't even get this event as it's removed before the event can be invoked. + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.remove(listener1); + listenerSet.sendEvent(EVENT_ID_2, TestListener::callback2); + ShadowLooper.runMainLooperToNextTask(); + + verify(listener1).callback1(); + verify(listener1).iterationFinished(Flags.create(EVENT_ID_1)); + verifyNoMoreInteractions(listener1, listener2); + } + + @Test + public void remove_withQueueing_stopsReceivingEventsImmediately() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener1 = mock(TestListener.class); + TestListener listener2 = mock(TestListener.class); + listenerSet.add(listener1); + listenerSet.add(listener2); + + // Listener1 shouldn't even get this event as it's removed before the event can be invoked. + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.remove(listener1); + listenerSet.queueEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.flushEvents(); + ShadowLooper.runMainLooperToNextTask(); + + verify(listener2, times(2)).callback1(); + verify(listener2).iterationFinished(Flags.create(EVENT_ID_1)); + verifyNoMoreInteractions(listener1, listener2); + } + + @Test + public void release_stopsForwardingEventsImmediately() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener2 = mock(TestListener.class); + // Listener1 releases the set from within the callback. + TestListener listener1 = + spy( + new TestListener() { + @Override + public void callback1() { + listenerSet.release(); + } + }); + listenerSet.add(listener1); + listenerSet.add(listener2); + + // Listener2 shouldn't even get this event as it's released before the event can be invoked. + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + listenerSet.sendEvent(EVENT_ID_2, TestListener::callback2); + ShadowLooper.runMainLooperToNextTask(); + + verify(listener1).callback1(); + verify(listener1).iterationFinished(Flags.create(EVENT_ID_1)); + verifyNoMoreInteractions(listener1, listener2); + } + + @Test + public void release_preventsRegisteringNewListeners() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener = mock(TestListener.class); + + listenerSet.release(); + listenerSet.add(listener); + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + + verify(listener, never()).callback1(); + } + + @Test + public void lazyRelease_stopsForwardingEventsFromNewHandlerMessagesAndCallsReleaseCallback() { + ListenerSet listenerSet = + new ListenerSet<>( + Looper.myLooper(), Clock.DEFAULT, Flags::new, TestListener::iterationFinished); + TestListener listener = mock(TestListener.class); + listenerSet.add(listener); + + // In-line event before release. + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + // Message triggering event sent before release. + new Handler().post(() -> listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1)); + // Lazy release with release callback. + listenerSet.lazyRelease(EVENT_ID_3, TestListener::callback3); + // In-line event after release. + listenerSet.sendEvent(EVENT_ID_1, TestListener::callback1); + // Message triggering event sent after release. + new Handler().post(() -> listenerSet.sendEvent(EVENT_ID_2, TestListener::callback2)); + ShadowLooper.runMainLooperToNextTask(); + + // Verify all events are delivered except for the one triggered by the message sent after the + // lazy release. + verify(listener, times(3)).callback1(); + verify(listener).callback3(); + verify(listener).iterationFinished(Flags.create(EVENT_ID_1)); + verify(listener).iterationFinished(Flags.create(EVENT_ID_1, EVENT_ID_3)); + verifyNoMoreInteractions(listener); + } + + private interface TestListener { + default void callback1() {} + + default void callback2() {} + + default void callback3() {} + + default void iterationFinished(Flags flags) {} + } + + private static final class Flags extends MutableFlags { + + public static Flags create(int... flagValues) { + Flags flags = new Flags(); + for (int value : flagValues) { + flags.add(value); + } + return flags; + } + } +} diff --git a/library/common/src/test/java/com/google/android/exoplayer2/util/ParsableByteArrayTest.java b/library/common/src/test/java/com/google/android/exoplayer2/util/ParsableByteArrayTest.java index 919f50fdc55..444e5f7b469 100644 --- a/library/common/src/test/java/com/google/android/exoplayer2/util/ParsableByteArrayTest.java +++ b/library/common/src/test/java/com/google/android/exoplayer2/util/ParsableByteArrayTest.java @@ -20,6 +20,7 @@ import static org.junit.Assert.fail; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.primitives.Bytes; import java.nio.ByteBuffer; import java.util.Arrays; import org.junit.Test; @@ -38,6 +39,36 @@ private static ParsableByteArray getTestDataArray() { return testArray; } + @Test + public void ensureCapacity_doesntReallocateNeedlesslyAndPreservesPositionAndLimit() { + ParsableByteArray array = getTestDataArray(); + byte[] dataBefore = array.getData(); + byte[] copyOfDataBefore = dataBefore.clone(); + + array.setPosition(3); + array.setLimit(4); + array.ensureCapacity(array.capacity() - 1); + + assertThat(array.getData()).isSameInstanceAs(dataBefore); + assertThat(array.getData()).isEqualTo(copyOfDataBefore); + assertThat(array.getPosition()).isEqualTo(3); + assertThat(array.limit()).isEqualTo(4); + } + + @Test + public void ensureCapacity_preservesDataPositionAndLimitWhenReallocating() { + ParsableByteArray array = getTestDataArray(); + byte[] copyOfDataBefore = array.getData().clone(); + + array.setPosition(3); + array.setLimit(4); + array.ensureCapacity(array.capacity() + 1); + + assertThat(array.getData()).isEqualTo(Bytes.concat(copyOfDataBefore, new byte[] {0})); + assertThat(array.getPosition()).isEqualTo(3); + assertThat(array.limit()).isEqualTo(4); + } + @Test public void readShort() { testReadShort((short) -1); @@ -483,6 +514,38 @@ public void readNullTerminatedStringWithoutEndingNull() { assertThat(parser.readNullTerminatedString()).isNull(); } + @Test + public void readDelimiterTerminatedString() { + byte[] bytes = new byte[] {'f', 'o', 'o', '*', 'b', 'a', 'r', '*'}; + // Test normal case. + ParsableByteArray parser = new ParsableByteArray(bytes); + assertThat(parser.readDelimiterTerminatedString('*')).isEqualTo("foo"); + assertThat(parser.getPosition()).isEqualTo(4); + assertThat(parser.readDelimiterTerminatedString('*')).isEqualTo("bar"); + assertThat(parser.getPosition()).isEqualTo(8); + assertThat(parser.readDelimiterTerminatedString('*')).isNull(); + + // Test with limit at delimiter. + parser = new ParsableByteArray(bytes, 4); + assertThat(parser.readDelimiterTerminatedString('*')).isEqualTo("foo"); + assertThat(parser.getPosition()).isEqualTo(4); + assertThat(parser.readDelimiterTerminatedString('*')).isNull(); + // Test with limit before delimiter. + parser = new ParsableByteArray(bytes, 3); + assertThat(parser.readDelimiterTerminatedString('*')).isEqualTo("foo"); + assertThat(parser.getPosition()).isEqualTo(3); + assertThat(parser.readDelimiterTerminatedString('*')).isNull(); + } + + @Test + public void readDelimiterTerminatedStringWithoutEndingDelimiter() { + byte[] bytes = new byte[] {'f', 'o', 'o', '*', 'b', 'a', 'r'}; + ParsableByteArray parser = new ParsableByteArray(bytes); + assertThat(parser.readDelimiterTerminatedString('*')).isEqualTo("foo"); + assertThat(parser.readDelimiterTerminatedString('*')).isEqualTo("bar"); + assertThat(parser.readDelimiterTerminatedString('*')).isNull(); + } + @Test public void readSingleLineWithoutEndingTrail() { byte[] bytes = new byte[] { diff --git a/library/common/src/test/java/com/google/android/exoplayer2/util/UtilTest.java b/library/common/src/test/java/com/google/android/exoplayer2/util/UtilTest.java index dd2ee7af890..d4aaa869f3d 100644 --- a/library/common/src/test/java/com/google/android/exoplayer2/util/UtilTest.java +++ b/library/common/src/test/java/com/google/android/exoplayer2/util/UtilTest.java @@ -20,10 +20,13 @@ import static com.google.android.exoplayer2.util.Util.escapeFileName; import static com.google.android.exoplayer2.util.Util.getCodecsOfType; import static com.google.android.exoplayer2.util.Util.getStringForTime; +import static com.google.android.exoplayer2.util.Util.gzip; +import static com.google.android.exoplayer2.util.Util.minValue; import static com.google.android.exoplayer2.util.Util.parseXsDateTime; import static com.google.android.exoplayer2.util.Util.parseXsDuration; import static com.google.android.exoplayer2.util.Util.unescapeFileName; import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; import android.database.sqlite.SQLiteDatabase; import android.database.sqlite.SQLiteOpenHelper; @@ -32,15 +35,21 @@ import android.text.Spanned; import android.text.style.StrikethroughSpan; import android.text.style.UnderlineSpan; +import android.util.SparseLongArray; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.common.io.ByteStreams; +import java.io.ByteArrayInputStream; import java.nio.ByteBuffer; import java.nio.ByteOrder; import java.util.ArrayList; import java.util.Arrays; import java.util.Formatter; +import java.util.NoSuchElementException; import java.util.Random; import java.util.zip.Deflater; +import java.util.zip.GZIPInputStream; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.annotation.Config; @@ -723,6 +732,21 @@ public void listBinarySearchCeil_targetBiggerThanValuesAndStayInBoundsTrue_retur .isEqualTo(2); } + @Test + public void sparseLongArrayMinValue_returnsMinValue() { + SparseLongArray sparseLongArray = new SparseLongArray(); + sparseLongArray.put(0, 12); + sparseLongArray.put(25, 10); + sparseLongArray.put(42, 11); + + assertThat(minValue(sparseLongArray)).isEqualTo(10); + } + + @Test + public void sparseLongArrayMinValue_emptyArray_throws() { + assertThrows(NoSuchElementException.class, () -> minValue(new SparseLongArray())); + } + @Test public void parseXsDuration_returnsParsedDurationInMillis() { assertThat(parseXsDuration("PT150.279S")).isEqualTo(150279L); @@ -908,6 +932,17 @@ public void crc8_returnsUpdatedCrc8() { assertThat(result).isEqualTo(0x4); } + @Test + public void gzip_resultInflatesBackToOriginalValue() throws Exception { + byte[] input = TestUtil.buildTestData(20); + + byte[] deflated = gzip(input); + + byte[] inflated = + ByteStreams.toByteArray(new GZIPInputStream(new ByteArrayInputStream(deflated))); + assertThat(inflated).isEqualTo(input); + } + @Test public void getBigEndianInt_fromBigEndian() { byte[] bytes = {0x1F, 0x2E, 0x3D, 0x4C}; diff --git a/library/common/src/test/java/com/google/android/exoplayer2/video/HevcConfigTest.java b/library/common/src/test/java/com/google/android/exoplayer2/video/HevcConfigTest.java new file mode 100644 index 00000000000..a4c312e8f74 --- /dev/null +++ b/library/common/src/test/java/com/google/android/exoplayer2/video/HevcConfigTest.java @@ -0,0 +1,173 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.video; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.util.ParsableByteArray; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link HevcConfig}. */ +@RunWith(AndroidJUnit4.class) +public final class HevcConfigTest { + + private static final byte[] HVCC_BOX_PAYLOAD = + new byte[] { + // Header + 1, + 1, + 96, + 0, + 0, + 0, + -80, + 0, + 0, + 0, + 0, + 0, + -103, + -16, + 0, + -4, + -4, + -8, + -8, + 0, + 0, + 15, + + // Number of arrays + 3, + + // NAL unit type = VPS + 32, + // Number of NAL units + 0, + 1, + // NAL unit length + 0, + 23, + // NAL unit + 64, + 1, + 12, + 1, + -1, + -1, + 1, + 96, + 0, + 0, + 3, + 0, + -80, + 0, + 0, + 3, + 0, + 0, + 3, + 0, + -103, + -84, + 9, + + // NAL unit type = SPS + 33, + // Number of NAL units + 0, + 1, + // NAL unit length + 0, + 39, + // NAL unit + 66, + 1, + 1, + 1, + 96, + 0, + 0, + 3, + 0, + -80, + 0, + 0, + 3, + 0, + 0, + 3, + 0, + -103, + -96, + 1, + -32, + 32, + 2, + 32, + 124, + 78, + 90, + -18, + 76, + -110, + -22, + 86, + 10, + 12, + 12, + 5, + -38, + 20, + 37, + + // NAL unit type = PPS + 34, + // Number of NAL units + 0, + 1, + // NAL unit length + 0, + 14, + // NAL unit + 68, + 1, + -64, + -29, + 15, + 8, + -80, + 96, + 48, + 24, + 12, + 115, + 8, + 64 + }; + + @Test + public void parseHevcDecoderConfigurationRecord() throws Exception { + ParsableByteArray data = new ParsableByteArray(HVCC_BOX_PAYLOAD); + HevcConfig hevcConfig = HevcConfig.parse(data); + + assertThat(hevcConfig.codecs).isEqualTo("hvc1.1.6.L153.B0"); + assertThat(hevcConfig.nalUnitLengthFieldLength).isEqualTo(4); + } +} diff --git a/library/core/proguard-rules.txt b/library/core/proguard-rules.txt index 35a7fdfeaaa..53adccd090f 100644 --- a/library/core/proguard-rules.txt +++ b/library/core/proguard-rules.txt @@ -5,17 +5,6 @@ public static android.net.Uri buildRawResourceUri(int); } -# Methods accessed via reflection in DefaultExtractorsFactory --dontnote com.google.android.exoplayer2.ext.flac.FlacLibrary --keepclassmembers class com.google.android.exoplayer2.ext.flac.FlacLibrary { - public static boolean isAvailable(); -} - -# Some members of this class are being accessed from native methods. Keep them unobfuscated. --keep class com.google.android.exoplayer2.video.VideoDecoderOutputBuffer { - *; -} - # Constructors accessed via reflection in DefaultRenderersFactory -dontnote com.google.android.exoplayer2.ext.vp9.LibvpxVideoRenderer -keepclassmembers class com.google.android.exoplayer2.ext.vp9.LibvpxVideoRenderer { diff --git a/library/core/src/androidTest/AndroidManifest.xml b/library/core/src/androidTest/AndroidManifest.xml index 04a07c4d501..4ffc92ac244 100644 --- a/library/core/src/androidTest/AndroidManifest.xml +++ b/library/core/src/androidTest/AndroidManifest.xml @@ -26,7 +26,7 @@ tools:ignore="MissingApplicationIcon,HardcodedDebugMode"> + android:name="com.google.android.exoplayer2.upstream.TestContentProvider"/> getTestResources() throws Exception { + byte[] completeData = + TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), DATA_PATH); + return ImmutableList.of( + new TestResource.Builder() + .setName("simple (pipe=false)") + .setUri(TestContentProvider.buildUri(DATA_PATH, /* pipeMode= */ false)) + .setExpectedBytes(completeData) + .build(), + new TestResource.Builder() + .setName("simple (pipe=true)") + .setUri(TestContentProvider.buildUri(DATA_PATH, /* pipeMode= */ true)) + .setExpectedBytes(completeData) + .build()); + } + + @Override + protected Uri getNotFoundUri() { + return TestContentProvider.buildUri("not/a/real/path", /* pipeMode= */ false); + } +} diff --git a/library/core/src/androidTest/java/com/google/android/exoplayer2/upstream/ContentDataSourceTest.java b/library/core/src/androidTest/java/com/google/android/exoplayer2/upstream/ContentDataSourceTest.java index 22442ca85f8..79b661e7178 100644 --- a/library/core/src/androidTest/java/com/google/android/exoplayer2/upstream/ContentDataSourceTest.java +++ b/library/core/src/androidTest/java/com/google/android/exoplayer2/upstream/ContentDataSourceTest.java @@ -19,15 +19,7 @@ import static junit.framework.Assert.fail; import static org.junit.Assert.assertThrows; -import android.content.ContentProvider; -import android.content.ContentResolver; -import android.content.ContentValues; -import android.content.res.AssetFileDescriptor; -import android.database.Cursor; import android.net.Uri; -import android.os.Bundle; -import android.os.ParcelFileDescriptor; -import androidx.annotation.Nullable; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; @@ -35,7 +27,6 @@ import com.google.android.exoplayer2.upstream.ContentDataSource.ContentDataSourceException; import java.io.EOFException; import java.io.FileNotFoundException; -import java.io.FileOutputStream; import java.io.IOException; import java.util.Arrays; import org.junit.Test; @@ -45,7 +36,6 @@ @RunWith(AndroidJUnit4.class) public final class ContentDataSourceTest { - private static final String AUTHORITY = "com.google.android.exoplayer2.core.test"; private static final String DATA_PATH = "media/mp3/1024_incrementing_bytes.mp3"; @Test @@ -141,98 +131,4 @@ private static void assertData(int offset, int length, boolean pipeMode) throws } } - /** - * A {@link ContentProvider} for the test. - */ - public static final class TestContentProvider extends ContentProvider - implements ContentProvider.PipeDataWriter { - - private static final String PARAM_PIPE_MODE = "pipe-mode"; - - public static Uri buildUri(String filePath, boolean pipeMode) { - Uri.Builder builder = new Uri.Builder() - .scheme(ContentResolver.SCHEME_CONTENT) - .authority(AUTHORITY) - .path(filePath); - if (pipeMode) { - builder.appendQueryParameter(TestContentProvider.PARAM_PIPE_MODE, "1"); - } - return builder.build(); - } - - @Override - public boolean onCreate() { - return true; - } - - @Override - public Cursor query( - Uri uri, String[] projection, String selection, String[] selectionArgs, String sortOrder) { - throw new UnsupportedOperationException(); - } - - @Override - public AssetFileDescriptor openAssetFile(Uri uri, String mode) throws FileNotFoundException { - if (uri.getPath() == null) { - return null; - } - try { - String fileName = getFileName(uri); - boolean pipeMode = uri.getQueryParameter(PARAM_PIPE_MODE) != null; - if (pipeMode) { - ParcelFileDescriptor fileDescriptor = openPipeHelper(uri, null, null, null, this); - return new AssetFileDescriptor(fileDescriptor, 0, C.LENGTH_UNSET); - } else { - return getContext().getAssets().openFd(fileName); - } - } catch (IOException e) { - FileNotFoundException exception = new FileNotFoundException(e.getMessage()); - exception.initCause(e); - throw exception; - } - } - - @Override - public String getType(Uri uri) { - throw new UnsupportedOperationException(); - } - - @Override - public Uri insert(Uri uri, ContentValues values) { - throw new UnsupportedOperationException(); - } - - @Override - public int delete(Uri uri, String selection, String[] selectionArgs) { - throw new UnsupportedOperationException(); - } - - @Override - public int update(Uri uri, ContentValues values, String selection, String[] selectionArgs) { - throw new UnsupportedOperationException(); - } - - @Override - public void writeDataToPipe( - ParcelFileDescriptor output, - Uri uri, - String mimeType, - @Nullable Bundle opts, - @Nullable Object args) { - try { - byte[] data = TestUtil.getByteArray(getContext(), getFileName(uri)); - FileOutputStream outputStream = new FileOutputStream(output.getFileDescriptor()); - outputStream.write(data); - outputStream.close(); - } catch (IOException e) { - throw new RuntimeException("Error writing to pipe", e); - } - } - - private static String getFileName(Uri uri) { - return uri.getPath().replaceFirst("/", ""); - } - - } - } diff --git a/library/core/src/androidTest/java/com/google/android/exoplayer2/upstream/TestContentProvider.java b/library/core/src/androidTest/java/com/google/android/exoplayer2/upstream/TestContentProvider.java new file mode 100644 index 00000000000..42bfd178e2d --- /dev/null +++ b/library/core/src/androidTest/java/com/google/android/exoplayer2/upstream/TestContentProvider.java @@ -0,0 +1,128 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.upstream; + +import android.content.ContentProvider; +import android.content.ContentResolver; +import android.content.ContentValues; +import android.content.res.AssetFileDescriptor; +import android.database.Cursor; +import android.net.Uri; +import android.os.Bundle; +import android.os.ParcelFileDescriptor; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.testutil.TestUtil; +import java.io.FileNotFoundException; +import java.io.FileOutputStream; +import java.io.IOException; + +/** A {@link ContentProvider} for tests of {@link ContentDataSource}. */ +public final class TestContentProvider extends ContentProvider + implements ContentProvider.PipeDataWriter { + + private static final String AUTHORITY = "com.google.android.exoplayer2.core.test"; + private static final String PARAM_PIPE_MODE = "pipe-mode"; + + public static Uri buildUri(String filePath, boolean pipeMode) { + Uri.Builder builder = + new Uri.Builder() + .scheme(ContentResolver.SCHEME_CONTENT) + .authority(AUTHORITY) + .path(filePath); + if (pipeMode) { + builder.appendQueryParameter(TestContentProvider.PARAM_PIPE_MODE, "1"); + } + return builder.build(); + } + + @Override + public boolean onCreate() { + return true; + } + + @Override + public Cursor query( + Uri uri, String[] projection, String selection, String[] selectionArgs, String sortOrder) { + throw new UnsupportedOperationException(); + } + + @Override + public AssetFileDescriptor openAssetFile(Uri uri, String mode) throws FileNotFoundException { + if (uri.getPath() == null) { + return null; + } + try { + String fileName = getFileName(uri); + boolean pipeMode = uri.getQueryParameter(PARAM_PIPE_MODE) != null; + if (pipeMode) { + ParcelFileDescriptor fileDescriptor = + openPipeHelper( + uri, /* mimeType= */ null, /* opts= */ null, /* args= */ null, /* func= */ this); + return new AssetFileDescriptor( + fileDescriptor, /* startOffset= */ 0, /* length= */ C.LENGTH_UNSET); + } else { + return getContext().getAssets().openFd(fileName); + } + } catch (IOException e) { + FileNotFoundException exception = new FileNotFoundException(e.getMessage()); + exception.initCause(e); + throw exception; + } + } + + @Override + public String getType(Uri uri) { + throw new UnsupportedOperationException(); + } + + @Override + public Uri insert(Uri uri, ContentValues values) { + throw new UnsupportedOperationException(); + } + + @Override + public int delete(Uri uri, String selection, String[] selectionArgs) { + throw new UnsupportedOperationException(); + } + + @Override + public int update(Uri uri, ContentValues values, String selection, String[] selectionArgs) { + throw new UnsupportedOperationException(); + } + + @Override + public void writeDataToPipe( + ParcelFileDescriptor output, + Uri uri, + String mimeType, + @Nullable Bundle opts, + @Nullable Object args) { + try { + byte[] data = TestUtil.getByteArray(getContext(), getFileName(uri)); + FileOutputStream outputStream = new FileOutputStream(output.getFileDescriptor()); + outputStream.write(data); + outputStream.close(); + } catch (IOException e) { + throw new RuntimeException("Error writing to pipe", e); + } + } + + private static String getFileName(Uri uri) { + return uri.getPath().replaceFirst("/", ""); + } + +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/BaseRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/BaseRenderer.java index 351f6c50f21..36dc8027f64 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/BaseRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/BaseRenderer.java @@ -340,8 +340,21 @@ protected final int getIndex() { * @param format The current format used by the renderer. May be null. */ protected final ExoPlaybackException createRendererException( - Exception cause, @Nullable Format format) { - @FormatSupport int formatSupport = RendererCapabilities.FORMAT_HANDLED; + Throwable cause, @Nullable Format format) { + return createRendererException(cause, format, /* isRecoverable= */ false); + } + + /** + * Creates an {@link ExoPlaybackException} of type {@link ExoPlaybackException#TYPE_RENDERER} for + * this renderer. + * + * @param cause The cause of the exception. + * @param format The current format used by the renderer. May be null. + * @param isRecoverable If the error is recoverable by disabling and re-enabling the renderer. + */ + protected final ExoPlaybackException createRendererException( + Throwable cause, @Nullable Format format, boolean isRecoverable) { + @C.FormatSupport int formatSupport = C.FORMAT_HANDLED; if (format != null && !throwRendererExceptionIsExecuting) { // Prevent recursive re-entry from subclass supportsFormat implementations. throwRendererExceptionIsExecuting = true; @@ -354,7 +367,7 @@ protected final ExoPlaybackException createRendererException( } } return ExoPlaybackException.createForRenderer( - cause, getName(), getIndex(), format, formatSupport); + cause, getName(), getIndex(), format, formatSupport, isRecoverable); } /** diff --git a/library/core/src/main/java/com/google/android/exoplayer2/ControlDispatcher.java b/library/core/src/main/java/com/google/android/exoplayer2/ControlDispatcher.java index 0d5e55fc833..d3ec2cb9db8 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/ControlDispatcher.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/ControlDispatcher.java @@ -113,6 +113,15 @@ public interface ControlDispatcher { */ boolean dispatchStop(Player player, boolean reset); + /** + * Dispatches a {@link Player#setPlaybackParameters(PlaybackParameters)} operation. + * + * @param player The {@link Player} to which the operation should be dispatched. + * @param playbackParameters The playback parameters. + * @return True if the operation was dispatched. False if suppressed. + */ + boolean dispatchSetPlaybackParameters(Player player, PlaybackParameters playbackParameters); + /** Returns {@code true} if rewind is enabled, {@code false} otherwise. */ boolean isRewindEnabled(); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/DefaultControlDispatcher.java b/library/core/src/main/java/com/google/android/exoplayer2/DefaultControlDispatcher.java index 25c468330c9..fe23f28db71 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/DefaultControlDispatcher.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/DefaultControlDispatcher.java @@ -99,7 +99,7 @@ public boolean dispatchNext(Player player) { int nextWindowIndex = player.getNextWindowIndex(); if (nextWindowIndex != C.INDEX_UNSET) { player.seekTo(nextWindowIndex, C.TIME_UNSET); - } else if (timeline.getWindow(windowIndex, window).isLive) { + } else if (timeline.getWindow(windowIndex, window).isLive()) { player.seekTo(windowIndex, C.TIME_UNSET); } return true; @@ -139,6 +139,13 @@ public boolean dispatchStop(Player player, boolean reset) { return true; } + @Override + public boolean dispatchSetPlaybackParameters( + Player player, PlaybackParameters playbackParameters) { + player.setPlaybackParameters(playbackParameters); + return true; + } + @Override public boolean isRewindEnabled() { return rewindIncrementMs > 0; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/DefaultLivePlaybackSpeedControl.java b/library/core/src/main/java/com/google/android/exoplayer2/DefaultLivePlaybackSpeedControl.java new file mode 100644 index 00000000000..e4fa65c4362 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/DefaultLivePlaybackSpeedControl.java @@ -0,0 +1,444 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2; + +import static com.google.common.primitives.Longs.max; +import static java.lang.Math.abs; +import static java.lang.Math.max; + +import android.os.SystemClock; +import com.google.android.exoplayer2.MediaItem.LiveConfiguration; +import com.google.android.exoplayer2.util.Assertions; +import com.google.android.exoplayer2.util.Util; + +/** + * A {@link LivePlaybackSpeedControl} that adjusts the playback speed using a proportional + * controller. + * + *

    The control mechanism calculates the adjusted speed as {@code 1.0 + proportionalControlFactor + * x (currentLiveOffsetSec - targetLiveOffsetSec)}. Unit speed (1.0f) is used, if the {@code + * currentLiveOffsetSec} is closer to {@code targetLiveOffsetSec} than the value set with {@link + * Builder#setMaxLiveOffsetErrorMsForUnitSpeed(long)}. + * + *

    The resulting speed is clamped to a minimum and maximum speed defined by the media, the + * fallback values set with {@link Builder#setFallbackMinPlaybackSpeed(float)} and {@link + * Builder#setFallbackMaxPlaybackSpeed(float)} or the {@link #DEFAULT_FALLBACK_MIN_PLAYBACK_SPEED + * minimum} and {@link #DEFAULT_FALLBACK_MAX_PLAYBACK_SPEED maximum} fallback default values. + * + *

    When the player rebuffers, the target live offset {@link + * Builder#setTargetLiveOffsetIncrementOnRebufferMs(long) is increased} to adjust to the reduced + * network capabilities. The live playback speed control also {@link + * Builder#setMinPossibleLiveOffsetSmoothingFactor(float) keeps track} of the minimum possible live + * offset to decrease the target live offset again if conditions improve. The minimum possible live + * offset is derived from the current offset and the duration of buffered media. + */ +public final class DefaultLivePlaybackSpeedControl implements LivePlaybackSpeedControl { + + /** + * The default minimum factor by which playback can be sped up that should be used if no minimum + * playback speed is defined by the media. + */ + public static final float DEFAULT_FALLBACK_MIN_PLAYBACK_SPEED = 0.97f; + + /** + * The default maximum factor by which playback can be sped up that should be used if no maximum + * playback speed is defined by the media. + */ + public static final float DEFAULT_FALLBACK_MAX_PLAYBACK_SPEED = 1.03f; + + /** + * The default {@link Builder#setMinUpdateIntervalMs(long) minimum interval} between playback + * speed changes, in milliseconds. + */ + public static final long DEFAULT_MIN_UPDATE_INTERVAL_MS = 1_000; + + /** + * The default {@link Builder#setProportionalControlFactor(float) proportional control factor} + * used to adjust the playback speed. + */ + public static final float DEFAULT_PROPORTIONAL_CONTROL_FACTOR = 0.1f; + + /** + * The default increment applied to the target live offset each time the player is rebuffering, in + * milliseconds + */ + public static final long DEFAULT_TARGET_LIVE_OFFSET_INCREMENT_ON_REBUFFER_MS = 500; + + /** + * The default smoothing factor when smoothing the minimum possible live offset that can be + * achieved during playback. + */ + public static final float DEFAULT_MIN_POSSIBLE_LIVE_OFFSET_SMOOTHING_FACTOR = 0.999f; + + /** + * The default maximum difference between the current live offset and the target live offset, in + * milliseconds, for which unit speed (1.0f) is used. + */ + public static final long DEFAULT_MAX_LIVE_OFFSET_ERROR_MS_FOR_UNIT_SPEED = 20; + + /** Builder for a {@link DefaultLivePlaybackSpeedControl}. */ + public static final class Builder { + + private float fallbackMinPlaybackSpeed; + private float fallbackMaxPlaybackSpeed; + private long minUpdateIntervalMs; + private float proportionalControlFactorUs; + private long maxLiveOffsetErrorUsForUnitSpeed; + private long targetLiveOffsetIncrementOnRebufferUs; + private float minPossibleLiveOffsetSmoothingFactor; + + /** Creates a builder. */ + public Builder() { + fallbackMinPlaybackSpeed = DEFAULT_FALLBACK_MIN_PLAYBACK_SPEED; + fallbackMaxPlaybackSpeed = DEFAULT_FALLBACK_MAX_PLAYBACK_SPEED; + minUpdateIntervalMs = DEFAULT_MIN_UPDATE_INTERVAL_MS; + proportionalControlFactorUs = DEFAULT_PROPORTIONAL_CONTROL_FACTOR / C.MICROS_PER_SECOND; + maxLiveOffsetErrorUsForUnitSpeed = C.msToUs(DEFAULT_MAX_LIVE_OFFSET_ERROR_MS_FOR_UNIT_SPEED); + targetLiveOffsetIncrementOnRebufferUs = + C.msToUs(DEFAULT_TARGET_LIVE_OFFSET_INCREMENT_ON_REBUFFER_MS); + minPossibleLiveOffsetSmoothingFactor = DEFAULT_MIN_POSSIBLE_LIVE_OFFSET_SMOOTHING_FACTOR; + } + + /** + * Sets the minimum playback speed that should be used if no minimum playback speed is defined + * by the media. + * + *

    The default is {@link #DEFAULT_FALLBACK_MIN_PLAYBACK_SPEED}. + * + * @param fallbackMinPlaybackSpeed The fallback minimum factor by which playback can be sped up. + * @return This builder, for convenience. + */ + public Builder setFallbackMinPlaybackSpeed(float fallbackMinPlaybackSpeed) { + Assertions.checkArgument(0 < fallbackMinPlaybackSpeed && fallbackMinPlaybackSpeed <= 1f); + this.fallbackMinPlaybackSpeed = fallbackMinPlaybackSpeed; + return this; + } + + /** + * Sets the maximum playback speed that should be used if no maximum playback speed is defined + * by the media. + * + *

    The default is {@link #DEFAULT_FALLBACK_MAX_PLAYBACK_SPEED}. + * + * @param fallbackMaxPlaybackSpeed The fallback maximum factor by which playback can be sped up. + * @return This builder, for convenience. + */ + public Builder setFallbackMaxPlaybackSpeed(float fallbackMaxPlaybackSpeed) { + Assertions.checkArgument(fallbackMaxPlaybackSpeed >= 1f); + this.fallbackMaxPlaybackSpeed = fallbackMaxPlaybackSpeed; + return this; + } + + /** + * Sets the minimum interval between playback speed changes, in milliseconds. + * + *

    The default is {@link #DEFAULT_MIN_UPDATE_INTERVAL_MS}. + * + * @param minUpdateIntervalMs The minimum interval between playback speed changes, in + * milliseconds. + * @return This builder, for convenience. + */ + public Builder setMinUpdateIntervalMs(long minUpdateIntervalMs) { + Assertions.checkArgument(minUpdateIntervalMs > 0); + this.minUpdateIntervalMs = minUpdateIntervalMs; + return this; + } + + /** + * Sets the proportional control factor used to adjust the playback speed. + * + *

    The factor by which playback will be sped up is calculated as {@code 1.0 + + * proportionalControlFactor x (currentLiveOffsetSec - targetLiveOffsetSec)}. + * + *

    The default is {@link #DEFAULT_PROPORTIONAL_CONTROL_FACTOR}. + * + * @param proportionalControlFactor The proportional control factor used to adjust the playback + * speed. + * @return This builder, for convenience. + */ + public Builder setProportionalControlFactor(float proportionalControlFactor) { + Assertions.checkArgument(proportionalControlFactor > 0); + this.proportionalControlFactorUs = proportionalControlFactor / C.MICROS_PER_SECOND; + return this; + } + + /** + * Sets the maximum difference between the current live offset and the target live offset, in + * milliseconds, for which unit speed (1.0f) is used. + * + *

    The default is {@link #DEFAULT_MAX_LIVE_OFFSET_ERROR_MS_FOR_UNIT_SPEED}. + * + * @param maxLiveOffsetErrorMsForUnitSpeed The maximum live offset error for which unit speed is + * used, in milliseconds. + * @return This builder, for convenience. + */ + public Builder setMaxLiveOffsetErrorMsForUnitSpeed(long maxLiveOffsetErrorMsForUnitSpeed) { + Assertions.checkArgument(maxLiveOffsetErrorMsForUnitSpeed > 0); + this.maxLiveOffsetErrorUsForUnitSpeed = C.msToUs(maxLiveOffsetErrorMsForUnitSpeed); + return this; + } + + /** + * Sets the increment applied to the target live offset each time the player is rebuffering, in + * milliseconds. + * + * @param targetLiveOffsetIncrementOnRebufferMs The increment applied to the target live offset + * when the player is rebuffering, in milliseconds + * @return This builder, for convenience. + */ + public Builder setTargetLiveOffsetIncrementOnRebufferMs( + long targetLiveOffsetIncrementOnRebufferMs) { + Assertions.checkArgument(targetLiveOffsetIncrementOnRebufferMs >= 0); + this.targetLiveOffsetIncrementOnRebufferUs = C.msToUs(targetLiveOffsetIncrementOnRebufferMs); + return this; + } + + /** + * Sets the smoothing factor when smoothing the minimum possible live offset that can be + * achieved during playback. + * + *

    The live playback speed control keeps track of the minimum possible live offset achievable + * during playback to know whether it can reduce the current target live offset. The minimum + * possible live offset is defined as {@code currentLiveOffset - bufferedDuration}. As the + * minimum possible live offset is constantly changing, it is smoothed over recent samples by + * applying exponential smoothing: {@code smoothedMinPossibleOffset = smoothingFactor x + * smoothedMinPossibleOffset + (1-smoothingFactor) x currentMinPossibleOffset}. + * + * @param minPossibleLiveOffsetSmoothingFactor The smoothing factor. Must be ≥ 0 and < 1. + * @return This builder, for convenience. + */ + public Builder setMinPossibleLiveOffsetSmoothingFactor( + float minPossibleLiveOffsetSmoothingFactor) { + Assertions.checkArgument( + minPossibleLiveOffsetSmoothingFactor >= 0 && minPossibleLiveOffsetSmoothingFactor < 1f); + this.minPossibleLiveOffsetSmoothingFactor = minPossibleLiveOffsetSmoothingFactor; + return this; + } + + /** Builds an instance. */ + public DefaultLivePlaybackSpeedControl build() { + return new DefaultLivePlaybackSpeedControl( + fallbackMinPlaybackSpeed, + fallbackMaxPlaybackSpeed, + minUpdateIntervalMs, + proportionalControlFactorUs, + maxLiveOffsetErrorUsForUnitSpeed, + targetLiveOffsetIncrementOnRebufferUs, + minPossibleLiveOffsetSmoothingFactor); + } + } + + private final float fallbackMinPlaybackSpeed; + private final float fallbackMaxPlaybackSpeed; + private final long minUpdateIntervalMs; + private final float proportionalControlFactor; + private final long maxLiveOffsetErrorUsForUnitSpeed; + private final long targetLiveOffsetRebufferDeltaUs; + private final float minPossibleLiveOffsetSmoothingFactor; + + private long mediaConfigurationTargetLiveOffsetUs; + private long targetLiveOffsetOverrideUs; + private long idealTargetLiveOffsetUs; + private long minTargetLiveOffsetUs; + private long maxTargetLiveOffsetUs; + private long currentTargetLiveOffsetUs; + + private float maxPlaybackSpeed; + private float minPlaybackSpeed; + private float adjustedPlaybackSpeed; + private long lastPlaybackSpeedUpdateMs; + + private long smoothedMinPossibleLiveOffsetUs; + private long smoothedMinPossibleLiveOffsetDeviationUs; + + private DefaultLivePlaybackSpeedControl( + float fallbackMinPlaybackSpeed, + float fallbackMaxPlaybackSpeed, + long minUpdateIntervalMs, + float proportionalControlFactor, + long maxLiveOffsetErrorUsForUnitSpeed, + long targetLiveOffsetRebufferDeltaUs, + float minPossibleLiveOffsetSmoothingFactor) { + this.fallbackMinPlaybackSpeed = fallbackMinPlaybackSpeed; + this.fallbackMaxPlaybackSpeed = fallbackMaxPlaybackSpeed; + this.minUpdateIntervalMs = minUpdateIntervalMs; + this.proportionalControlFactor = proportionalControlFactor; + this.maxLiveOffsetErrorUsForUnitSpeed = maxLiveOffsetErrorUsForUnitSpeed; + this.targetLiveOffsetRebufferDeltaUs = targetLiveOffsetRebufferDeltaUs; + this.minPossibleLiveOffsetSmoothingFactor = minPossibleLiveOffsetSmoothingFactor; + mediaConfigurationTargetLiveOffsetUs = C.TIME_UNSET; + targetLiveOffsetOverrideUs = C.TIME_UNSET; + minTargetLiveOffsetUs = C.TIME_UNSET; + maxTargetLiveOffsetUs = C.TIME_UNSET; + minPlaybackSpeed = fallbackMinPlaybackSpeed; + maxPlaybackSpeed = fallbackMaxPlaybackSpeed; + adjustedPlaybackSpeed = 1.0f; + lastPlaybackSpeedUpdateMs = C.TIME_UNSET; + idealTargetLiveOffsetUs = C.TIME_UNSET; + currentTargetLiveOffsetUs = C.TIME_UNSET; + smoothedMinPossibleLiveOffsetUs = C.TIME_UNSET; + smoothedMinPossibleLiveOffsetDeviationUs = C.TIME_UNSET; + } + + @Override + public void setLiveConfiguration(LiveConfiguration liveConfiguration) { + mediaConfigurationTargetLiveOffsetUs = C.msToUs(liveConfiguration.targetOffsetMs); + minTargetLiveOffsetUs = C.msToUs(liveConfiguration.minOffsetMs); + maxTargetLiveOffsetUs = C.msToUs(liveConfiguration.maxOffsetMs); + minPlaybackSpeed = + liveConfiguration.minPlaybackSpeed != C.RATE_UNSET + ? liveConfiguration.minPlaybackSpeed + : fallbackMinPlaybackSpeed; + maxPlaybackSpeed = + liveConfiguration.maxPlaybackSpeed != C.RATE_UNSET + ? liveConfiguration.maxPlaybackSpeed + : fallbackMaxPlaybackSpeed; + maybeResetTargetLiveOffsetUs(); + } + + @Override + public void setTargetLiveOffsetOverrideUs(long liveOffsetUs) { + targetLiveOffsetOverrideUs = liveOffsetUs; + maybeResetTargetLiveOffsetUs(); + } + + @Override + public void notifyRebuffer() { + if (currentTargetLiveOffsetUs == C.TIME_UNSET) { + return; + } + currentTargetLiveOffsetUs += targetLiveOffsetRebufferDeltaUs; + if (maxTargetLiveOffsetUs != C.TIME_UNSET + && currentTargetLiveOffsetUs > maxTargetLiveOffsetUs) { + currentTargetLiveOffsetUs = maxTargetLiveOffsetUs; + } + lastPlaybackSpeedUpdateMs = C.TIME_UNSET; + } + + @Override + public float getAdjustedPlaybackSpeed(long liveOffsetUs, long bufferedDurationUs) { + if (mediaConfigurationTargetLiveOffsetUs == C.TIME_UNSET) { + return 1f; + } + + updateSmoothedMinPossibleLiveOffsetUs(liveOffsetUs, bufferedDurationUs); + + if (lastPlaybackSpeedUpdateMs != C.TIME_UNSET + && SystemClock.elapsedRealtime() - lastPlaybackSpeedUpdateMs < minUpdateIntervalMs) { + return adjustedPlaybackSpeed; + } + lastPlaybackSpeedUpdateMs = SystemClock.elapsedRealtime(); + + adjustTargetLiveOffsetUs(liveOffsetUs); + long liveOffsetErrorUs = liveOffsetUs - currentTargetLiveOffsetUs; + if (Math.abs(liveOffsetErrorUs) < maxLiveOffsetErrorUsForUnitSpeed) { + adjustedPlaybackSpeed = 1f; + } else { + float calculatedSpeed = 1f + proportionalControlFactor * liveOffsetErrorUs; + adjustedPlaybackSpeed = + Util.constrainValue(calculatedSpeed, minPlaybackSpeed, maxPlaybackSpeed); + } + return adjustedPlaybackSpeed; + } + + @Override + public long getTargetLiveOffsetUs() { + return currentTargetLiveOffsetUs; + } + + private void maybeResetTargetLiveOffsetUs() { + long idealOffsetUs = C.TIME_UNSET; + if (mediaConfigurationTargetLiveOffsetUs != C.TIME_UNSET) { + idealOffsetUs = + targetLiveOffsetOverrideUs != C.TIME_UNSET + ? targetLiveOffsetOverrideUs + : mediaConfigurationTargetLiveOffsetUs; + if (minTargetLiveOffsetUs != C.TIME_UNSET && idealOffsetUs < minTargetLiveOffsetUs) { + idealOffsetUs = minTargetLiveOffsetUs; + } + if (maxTargetLiveOffsetUs != C.TIME_UNSET && idealOffsetUs > maxTargetLiveOffsetUs) { + idealOffsetUs = maxTargetLiveOffsetUs; + } + } + if (idealTargetLiveOffsetUs == idealOffsetUs) { + return; + } + idealTargetLiveOffsetUs = idealOffsetUs; + currentTargetLiveOffsetUs = idealOffsetUs; + smoothedMinPossibleLiveOffsetUs = C.TIME_UNSET; + smoothedMinPossibleLiveOffsetDeviationUs = C.TIME_UNSET; + lastPlaybackSpeedUpdateMs = C.TIME_UNSET; + } + + private void updateSmoothedMinPossibleLiveOffsetUs(long liveOffsetUs, long bufferedDurationUs) { + long minPossibleLiveOffsetUs = liveOffsetUs - bufferedDurationUs; + if (smoothedMinPossibleLiveOffsetUs == C.TIME_UNSET) { + smoothedMinPossibleLiveOffsetUs = minPossibleLiveOffsetUs; + smoothedMinPossibleLiveOffsetDeviationUs = 0; + } else { + // Use the maximum here to ensure we keep track of the upper bound of what is safely possible, + // not the average. + smoothedMinPossibleLiveOffsetUs = + max( + minPossibleLiveOffsetUs, + smooth( + smoothedMinPossibleLiveOffsetUs, + minPossibleLiveOffsetUs, + minPossibleLiveOffsetSmoothingFactor)); + long minPossibleLiveOffsetDeviationUs = + abs(minPossibleLiveOffsetUs - smoothedMinPossibleLiveOffsetUs); + smoothedMinPossibleLiveOffsetDeviationUs = + smooth( + smoothedMinPossibleLiveOffsetDeviationUs, + minPossibleLiveOffsetDeviationUs, + minPossibleLiveOffsetSmoothingFactor); + } + } + + private void adjustTargetLiveOffsetUs(long liveOffsetUs) { + // Stay in a safe distance (3 standard deviations = >99%) to the minimum possible live offset. + long safeOffsetUs = + smoothedMinPossibleLiveOffsetUs + 3 * smoothedMinPossibleLiveOffsetDeviationUs; + if (currentTargetLiveOffsetUs > safeOffsetUs) { + // There is room for decreasing the target offset towards the ideal or safe offset (whichever + // is larger). We want to limit the decrease so that the playback speed delta we achieve is + // the same as the maximum delta when slowing down towards the target. + long minUpdateIntervalUs = C.msToUs(minUpdateIntervalMs); + long decrementToOffsetCurrentSpeedUs = + (long) ((adjustedPlaybackSpeed - 1f) * minUpdateIntervalUs); + long decrementToIncreaseSpeedUs = (long) ((maxPlaybackSpeed - 1f) * minUpdateIntervalUs); + long maxDecrementUs = decrementToOffsetCurrentSpeedUs + decrementToIncreaseSpeedUs; + currentTargetLiveOffsetUs = + max(safeOffsetUs, idealTargetLiveOffsetUs, currentTargetLiveOffsetUs - maxDecrementUs); + } else { + // We'd like to reach a stable condition where the current live offset stays just below the + // safe offset. But don't increase the target offset to more than what would allow us to slow + // down gradually from the current offset. + long offsetWhenSlowingDownNowUs = + liveOffsetUs - (long) (max(0f, adjustedPlaybackSpeed - 1f) / proportionalControlFactor); + currentTargetLiveOffsetUs = + Util.constrainValue(offsetWhenSlowingDownNowUs, currentTargetLiveOffsetUs, safeOffsetUs); + if (maxTargetLiveOffsetUs != C.TIME_UNSET + && currentTargetLiveOffsetUs > maxTargetLiveOffsetUs) { + currentTargetLiveOffsetUs = maxTargetLiveOffsetUs; + } + } + } + + private static long smooth(long smoothedValue, long newValue, float smoothingFactor) { + return (long) (smoothingFactor * smoothedValue + (1f - smoothingFactor) * newValue); + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/DefaultLoadControl.java b/library/core/src/main/java/com/google/android/exoplayer2/DefaultLoadControl.java index 2b72fc6c095..2692925333d 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/DefaultLoadControl.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/DefaultLoadControl.java @@ -15,21 +15,20 @@ */ package com.google.android.exoplayer2; +import static com.google.android.exoplayer2.util.Assertions.checkState; import static java.lang.Math.max; import static java.lang.Math.min; import androidx.annotation.Nullable; import com.google.android.exoplayer2.source.TrackGroupArray; -import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.DefaultAllocator; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.Util; -/** - * The default {@link LoadControl} implementation. - */ +/** The default {@link LoadControl} implementation. */ public class DefaultLoadControl implements LoadControl { /** @@ -129,7 +128,7 @@ public Builder() { * @throws IllegalStateException If {@link #build()} has already been called. */ public Builder setAllocator(DefaultAllocator allocator) { - Assertions.checkState(!buildCalled); + checkState(!buildCalled); this.allocator = allocator; return this; } @@ -154,7 +153,7 @@ public Builder setBufferDurationsMs( int maxBufferMs, int bufferForPlaybackMs, int bufferForPlaybackAfterRebufferMs) { - Assertions.checkState(!buildCalled); + checkState(!buildCalled); assertGreaterOrEqual(bufferForPlaybackMs, 0, "bufferForPlaybackMs", "0"); assertGreaterOrEqual( bufferForPlaybackAfterRebufferMs, 0, "bufferForPlaybackAfterRebufferMs", "0"); @@ -181,7 +180,7 @@ public Builder setBufferDurationsMs( * @throws IllegalStateException If {@link #build()} has already been called. */ public Builder setTargetBufferBytes(int targetBufferBytes) { - Assertions.checkState(!buildCalled); + checkState(!buildCalled); this.targetBufferBytes = targetBufferBytes; return this; } @@ -196,7 +195,7 @@ public Builder setTargetBufferBytes(int targetBufferBytes) { * @throws IllegalStateException If {@link #build()} has already been called. */ public Builder setPrioritizeTimeOverSizeThresholds(boolean prioritizeTimeOverSizeThresholds) { - Assertions.checkState(!buildCalled); + checkState(!buildCalled); this.prioritizeTimeOverSizeThresholds = prioritizeTimeOverSizeThresholds; return this; } @@ -212,7 +211,7 @@ public Builder setPrioritizeTimeOverSizeThresholds(boolean prioritizeTimeOverSiz * @throws IllegalStateException If {@link #build()} has already been called. */ public Builder setBackBuffer(int backBufferDurationMs, boolean retainBackBufferFromKeyframe) { - Assertions.checkState(!buildCalled); + checkState(!buildCalled); assertGreaterOrEqual(backBufferDurationMs, 0, "backBufferDurationMs", "0"); this.backBufferDurationMs = backBufferDurationMs; this.retainBackBufferFromKeyframe = retainBackBufferFromKeyframe; @@ -227,7 +226,7 @@ public DefaultLoadControl createDefaultLoadControl() { /** Creates a {@link DefaultLoadControl}. */ public DefaultLoadControl build() { - Assertions.checkState(!buildCalled); + checkState(!buildCalled); buildCalled = true; if (allocator == null) { allocator = new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE); @@ -257,19 +256,13 @@ public DefaultLoadControl build() { private final boolean retainBackBufferFromKeyframe; private int targetBufferBytes; - private boolean isBuffering; + private boolean isLoading; /** Constructs a new instance, using the {@code DEFAULT_*} constants defined in this class. */ @SuppressWarnings("deprecation") public DefaultLoadControl() { - this(new DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE)); - } - - /** @deprecated Use {@link Builder} instead. */ - @Deprecated - public DefaultLoadControl(DefaultAllocator allocator) { this( - allocator, + new DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE), DEFAULT_MIN_BUFFER_MS, DEFAULT_MAX_BUFFER_MS, DEFAULT_BUFFER_FOR_PLAYBACK_MS, @@ -280,28 +273,6 @@ public DefaultLoadControl(DefaultAllocator allocator) { DEFAULT_RETAIN_BACK_BUFFER_FROM_KEYFRAME); } - /** @deprecated Use {@link Builder} instead. */ - @Deprecated - public DefaultLoadControl( - DefaultAllocator allocator, - int minBufferMs, - int maxBufferMs, - int bufferForPlaybackMs, - int bufferForPlaybackAfterRebufferMs, - int targetBufferBytes, - boolean prioritizeTimeOverSizeThresholds) { - this( - allocator, - minBufferMs, - maxBufferMs, - bufferForPlaybackMs, - bufferForPlaybackAfterRebufferMs, - targetBufferBytes, - prioritizeTimeOverSizeThresholds, - DEFAULT_BACK_BUFFER_DURATION_MS, - DEFAULT_RETAIN_BACK_BUFFER_FROM_KEYFRAME); - } - protected DefaultLoadControl( DefaultAllocator allocator, int minBufferMs, @@ -345,8 +316,8 @@ public void onPrepared() { } @Override - public void onTracksSelected(Renderer[] renderers, TrackGroupArray trackGroups, - TrackSelectionArray trackSelections) { + public void onTracksSelected( + Renderer[] renderers, TrackGroupArray trackGroups, ExoTrackSelection[] trackSelections) { targetBufferBytes = targetBufferBytesOverwrite == C.LENGTH_UNSET ? calculateTargetBufferBytes(renderers, trackSelections) @@ -394,23 +365,26 @@ public boolean shouldContinueLoading( // Prevent playback from getting stuck if minBufferUs is too small. minBufferUs = max(minBufferUs, 500_000); if (bufferedDurationUs < minBufferUs) { - isBuffering = prioritizeTimeOverSizeThresholds || !targetBufferSizeReached; - if (!isBuffering && bufferedDurationUs < 500_000) { + isLoading = prioritizeTimeOverSizeThresholds || !targetBufferSizeReached; + if (!isLoading && bufferedDurationUs < 500_000) { Log.w( "DefaultLoadControl", "Target buffer size reached with less than 500ms of buffered media data."); } } else if (bufferedDurationUs >= maxBufferUs || targetBufferSizeReached) { - isBuffering = false; - } // Else don't change the buffering state - return isBuffering; + isLoading = false; + } // Else don't change the loading state. + return isLoading; } @Override public boolean shouldStartPlayback( - long bufferedDurationUs, float playbackSpeed, boolean rebuffering) { + long bufferedDurationUs, float playbackSpeed, boolean rebuffering, long targetLiveOffsetUs) { bufferedDurationUs = Util.getPlayoutDurationForMediaDuration(bufferedDurationUs, playbackSpeed); long minBufferDurationUs = rebuffering ? bufferForPlaybackAfterRebufferUs : bufferForPlaybackUs; + if (targetLiveOffsetUs != C.TIME_UNSET) { + minBufferDurationUs = min(targetLiveOffsetUs / 2, minBufferDurationUs); + } return minBufferDurationUs <= 0 || bufferedDurationUs >= minBufferDurationUs || (!prioritizeTimeOverSizeThresholds @@ -426,10 +400,10 @@ public boolean shouldStartPlayback( * @return The target buffer size in bytes. */ protected int calculateTargetBufferBytes( - Renderer[] renderers, TrackSelectionArray trackSelectionArray) { + Renderer[] renderers, ExoTrackSelection[] trackSelectionArray) { int targetBufferSize = 0; for (int i = 0; i < renderers.length; i++) { - if (trackSelectionArray.get(i) != null) { + if (trackSelectionArray[i] != null) { targetBufferSize += getDefaultBufferSize(renderers[i].getTrackType()); } } @@ -441,7 +415,7 @@ private void reset(boolean resetAllocator) { targetBufferBytesOverwrite == C.LENGTH_UNSET ? DEFAULT_MIN_BUFFER_SIZE : targetBufferBytesOverwrite; - isBuffering = false; + isLoading = false; if (resetAllocator) { allocator.reset(); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/DefaultRenderersFactory.java b/library/core/src/main/java/com/google/android/exoplayer2/DefaultRenderersFactory.java index b00cafece25..5176e49df07 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/DefaultRenderersFactory.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/DefaultRenderersFactory.java @@ -28,7 +28,6 @@ import com.google.android.exoplayer2.audio.DefaultAudioSink; import com.google.android.exoplayer2.audio.DefaultAudioSink.DefaultAudioProcessorChain; import com.google.android.exoplayer2.audio.MediaCodecAudioRenderer; -import com.google.android.exoplayer2.mediacodec.MediaCodecRenderer; import com.google.android.exoplayer2.mediacodec.MediaCodecSelector; import com.google.android.exoplayer2.metadata.MetadataOutput; import com.google.android.exoplayer2.metadata.MetadataRenderer; @@ -78,22 +77,27 @@ public class DefaultRenderersFactory implements RenderersFactory { /** * Allow use of extension renderers. Extension renderers are indexed before core renderers of the * same type. A {@link TrackSelector} that prefers the first suitable renderer will therefore - * prefer to use an extension renderer to a core renderer in the case that both are able to play - * a given track. + * prefer to use an extension renderer to a core renderer in the case that both are able to play a + * given track. */ public static final int EXTENSION_RENDERER_MODE_PREFER = 2; - private static final String TAG = "DefaultRenderersFactory"; + /** + * The maximum number of frames that can be dropped between invocations of {@link + * VideoRendererEventListener#onDroppedFrames(int, long)}. + */ + public static final int MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY = 50; - protected static final int MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY = 50; + private static final String TAG = "DefaultRenderersFactory"; private final Context context; @ExtensionRendererMode private int extensionRendererMode; private long allowedVideoJoiningTimeMs; private boolean enableDecoderFallback; private MediaCodecSelector mediaCodecSelector; - private @MediaCodecRenderer.MediaCodecOperationMode int audioMediaCodecOperationMode; - private @MediaCodecRenderer.MediaCodecOperationMode int videoMediaCodecOperationMode; + private boolean enableAsyncQueueing; + private boolean forceAsyncQueueingSynchronizationWorkaround; + private boolean enableSynchronizeCodecInteractionsWithQueueing; private boolean enableFloatOutput; private boolean enableAudioTrackPlaybackParams; private boolean enableOffload; @@ -104,8 +108,6 @@ public DefaultRenderersFactory(Context context) { extensionRendererMode = EXTENSION_RENDERER_MODE_OFF; allowedVideoJoiningTimeMs = DEFAULT_ALLOWED_VIDEO_JOINING_TIME_MS; mediaCodecSelector = MediaCodecSelector.DEFAULT; - audioMediaCodecOperationMode = MediaCodecRenderer.OPERATION_MODE_SYNCHRONOUS; - videoMediaCodecOperationMode = MediaCodecRenderer.OPERATION_MODE_SYNCHRONOUS; } /** @@ -151,48 +153,49 @@ public DefaultRenderersFactory setExtensionRendererMode( } /** - * Set the {@link MediaCodecRenderer.MediaCodecOperationMode} of {@link MediaCodecAudioRenderer} - * instances. + * Enable asynchronous buffer queueing for both {@link MediaCodecAudioRenderer} and {@link + * MediaCodecVideoRenderer} instances. * *

    This method is experimental, and will be renamed or removed in a future release. * - * @param mode The {@link MediaCodecRenderer.MediaCodecOperationMode} to set. + * @param enabled Whether asynchronous queueing is enabled. * @return This factory, for convenience. */ - public DefaultRenderersFactory experimentalSetAudioMediaCodecOperationMode( - @MediaCodecRenderer.MediaCodecOperationMode int mode) { - audioMediaCodecOperationMode = mode; + public DefaultRenderersFactory experimentalSetAsynchronousBufferQueueingEnabled(boolean enabled) { + enableAsyncQueueing = enabled; return this; } /** - * Set the {@link MediaCodecRenderer.MediaCodecOperationMode} of {@link MediaCodecVideoRenderer} - * instances. + * Enable the asynchronous queueing synchronization workaround. + * + *

    When enabled, the queueing threads for {@link MediaCodec} instances will synchronize on a + * shared lock when submitting buffers to the respective {@link MediaCodec}. * *

    This method is experimental, and will be renamed or removed in a future release. * - * @param mode The {@link MediaCodecRenderer.MediaCodecOperationMode} to set. + * @param enabled Whether the asynchronous queueing synchronization workaround is enabled by + * default. * @return This factory, for convenience. */ - public DefaultRenderersFactory experimentalSetVideoMediaCodecOperationMode( - @MediaCodecRenderer.MediaCodecOperationMode int mode) { - videoMediaCodecOperationMode = mode; + public DefaultRenderersFactory experimentalSetForceAsyncQueueingSynchronizationWorkaround( + boolean enabled) { + this.forceAsyncQueueingSynchronizationWorkaround = enabled; return this; } /** - * Set the {@link MediaCodecRenderer.MediaCodecOperationMode} for both {@link - * MediaCodecAudioRenderer} {@link MediaCodecVideoRenderer} instances. + * Enable synchronizing codec interactions with asynchronous buffer queueing. * *

    This method is experimental, and will be renamed or removed in a future release. * - * @param mode The {@link MediaCodecRenderer.MediaCodecOperationMode} to set. + * @param enabled Whether codec interactions will be synchronized with asynchronous buffer + * queueing. * @return This factory, for convenience. */ - public DefaultRenderersFactory experimentalSetMediaCodecOperationMode( - @MediaCodecRenderer.MediaCodecOperationMode int mode) { - experimentalSetAudioMediaCodecOperationMode(mode); - experimentalSetVideoMediaCodecOperationMode(mode); + public DefaultRenderersFactory experimentalSetSynchronizeCodecInteractionsWithQueueingEnabled( + boolean enabled) { + enableSynchronizeCodecInteractionsWithQueueing = enabled; return this; } @@ -372,7 +375,11 @@ protected void buildVideoRenderers( eventHandler, eventListener, MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY); - videoRenderer.experimentalSetMediaCodecOperationMode(videoMediaCodecOperationMode); + videoRenderer.experimentalSetAsynchronousBufferQueueingEnabled(enableAsyncQueueing); + videoRenderer.experimentalSetForceAsyncQueueingSynchronizationWorkaround( + forceAsyncQueueingSynchronizationWorkaround); + videoRenderer.experimentalSetSynchronizeCodecInteractionsWithQueueingEnabled( + enableSynchronizeCodecInteractionsWithQueueing); out.add(videoRenderer); if (extensionRendererMode == EXTENSION_RENDERER_MODE_OFF) { @@ -469,7 +476,11 @@ protected void buildAudioRenderers( eventHandler, eventListener, audioSink); - audioRenderer.experimentalSetMediaCodecOperationMode(audioMediaCodecOperationMode); + audioRenderer.experimentalSetAsynchronousBufferQueueingEnabled(enableAsyncQueueing); + audioRenderer.experimentalSetForceAsyncQueueingSynchronizationWorkaround( + forceAsyncQueueingSynchronizationWorkaround); + audioRenderer.experimentalSetSynchronizeCodecInteractionsWithQueueingEnabled( + enableSynchronizeCodecInteractionsWithQueueing); out.add(audioRenderer); if (extensionRendererMode == EXTENSION_RENDERER_MODE_OFF) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayer.java b/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayer.java index ccb67866a41..9169271d126 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayer.java @@ -133,6 +133,12 @@ */ public interface ExoPlayer extends Player { + /** + * The default timeout for calls to {@link #release} and {@link #setForegroundMode}, in + * milliseconds. + */ + long DEFAULT_RELEASE_TIMEOUT_MS = 500; + /** * A builder for {@link ExoPlayer} instances. * @@ -152,10 +158,11 @@ final class Builder { private boolean useLazyPreparation; private SeekParameters seekParameters; private boolean pauseAtEndOfMediaItems; + private long releaseTimeoutMs; + private LivePlaybackSpeedControl livePlaybackSpeedControl; private boolean buildCalled; - private long releaseTimeoutMs; - private boolean throwWhenStuckBuffering; + private long setForegroundModeTimeoutMs; /** * Creates a builder with a list of {@link Renderer Renderers}. @@ -167,12 +174,14 @@ final class Builder { *

  • {@link MediaSourceFactory}: {@link DefaultMediaSourceFactory} *
  • {@link LoadControl}: {@link DefaultLoadControl} *
  • {@link BandwidthMeter}: {@link DefaultBandwidthMeter#getSingletonInstance(Context)} + *
  • {@link LivePlaybackSpeedControl}: {@link DefaultLivePlaybackSpeedControl} *
  • {@link Looper}: The {@link Looper} associated with the current thread, or the {@link * Looper} of the application's main thread if the current thread doesn't have a {@link * Looper} *
  • {@link AnalyticsCollector}: {@link AnalyticsCollector} with {@link Clock#DEFAULT} *
  • {@code useLazyPreparation}: {@code true} *
  • {@link SeekParameters}: {@link SeekParameters#DEFAULT} + *
  • {@code releaseTimeoutMs}: {@link ExoPlayer#DEFAULT_RELEASE_TIMEOUT_MS} *
  • {@code pauseAtEndOfMediaItems}: {@code false} *
  • {@link Clock}: {@link Clock#DEFAULT} * @@ -216,34 +225,22 @@ public Builder( looper = Util.getCurrentOrMainLooper(); useLazyPreparation = true; seekParameters = SeekParameters.DEFAULT; + livePlaybackSpeedControl = new DefaultLivePlaybackSpeedControl.Builder().build(); clock = Clock.DEFAULT; - throwWhenStuckBuffering = true; - } - - /** - * Set a limit on the time a call to {@link ExoPlayer#release()} can spend. If a call to {@link - * ExoPlayer#release()} takes more than {@code timeoutMs} milliseconds to complete, the player - * will raise an error via {@link Player.EventListener#onPlayerError}. - * - *

    This method is experimental, and will be renamed or removed in a future release. - * - * @param timeoutMs The time limit in milliseconds, or 0 for no limit. - */ - public Builder experimentalSetReleaseTimeoutMs(long timeoutMs) { - releaseTimeoutMs = timeoutMs; - return this; + releaseTimeoutMs = DEFAULT_RELEASE_TIMEOUT_MS; } /** - * Sets whether the player should throw when it detects it's stuck buffering. + * Set a limit on the time a call to {@link ExoPlayer#setForegroundMode} can spend. If a call to + * {@link ExoPlayer#setForegroundMode} takes more than {@code timeoutMs} milliseconds to + * complete, the player will raise an error via {@link Player.EventListener#onPlayerError}. * *

    This method is experimental, and will be renamed or removed in a future release. * - * @param throwWhenStuckBuffering Whether to throw when the player detects it's stuck buffering. - * @return This builder. + * @param timeoutMs The time limit in milliseconds. */ - public Builder experimentalSetThrowWhenStuckBuffering(boolean throwWhenStuckBuffering) { - this.throwWhenStuckBuffering = throwWhenStuckBuffering; + public Builder experimentalSetForegroundModeTimeoutMs(long timeoutMs) { + setForegroundModeTimeoutMs = timeoutMs; return this; } @@ -356,6 +353,23 @@ public Builder setSeekParameters(SeekParameters seekParameters) { return this; } + /** + * Sets a timeout for calls to {@link #release} and {@link #setForegroundMode}. + * + *

    If a call to {@link #release} or {@link #setForegroundMode} takes more than {@code + * timeoutMs} to complete, the player will report an error via {@link + * Player.EventListener#onPlayerError}. + * + * @param releaseTimeoutMs The release timeout, in milliseconds. + * @return This builder. + * @throws IllegalStateException If {@link #build()} has already been called. + */ + public Builder setReleaseTimeoutMs(long releaseTimeoutMs) { + Assertions.checkState(!buildCalled); + this.releaseTimeoutMs = releaseTimeoutMs; + return this; + } + /** * Sets whether to pause playback at the end of each media item. * @@ -374,6 +388,20 @@ public Builder setPauseAtEndOfMediaItems(boolean pauseAtEndOfMediaItems) { return this; } + /** + * Sets the {@link LivePlaybackSpeedControl} that will control the playback speed when playing + * live streams, in order to maintain a steady target offset from the live stream edge. + * + * @param livePlaybackSpeedControl The {@link LivePlaybackSpeedControl}. + * @return This builder. + * @throws IllegalStateException If {@link #build()} has already been called. + */ + public Builder setLivePlaybackSpeedControl(LivePlaybackSpeedControl livePlaybackSpeedControl) { + Assertions.checkState(!buildCalled); + this.livePlaybackSpeedControl = livePlaybackSpeedControl; + return this; + } + /** * Sets the {@link Clock} that will be used by the player. Should only be set for testing * purposes. @@ -407,24 +435,32 @@ public ExoPlayer build() { analyticsCollector, useLazyPreparation, seekParameters, + livePlaybackSpeedControl, + releaseTimeoutMs, pauseAtEndOfMediaItems, clock, - looper); + looper, + /* wrappingPlayer= */ null); - if (releaseTimeoutMs > 0) { - player.experimentalSetReleaseTimeoutMs(releaseTimeoutMs); + if (setForegroundModeTimeoutMs > 0) { + player.experimentalSetForegroundModeTimeoutMs(setForegroundModeTimeoutMs); } - if (!throwWhenStuckBuffering) { - player.experimentalDisableThrowWhenStuckBuffering(); - } - return player; } } + /** + * Returns the track selector that this player uses, or null if track selection is not supported. + */ + @Nullable + TrackSelector getTrackSelector(); + /** Returns the {@link Looper} associated with the playback thread. */ Looper getPlaybackLooper(); + /** Returns the {@link Clock} used for playback. */ + Clock getClock(); + /** @deprecated Use {@link #prepare()} instead. */ @Deprecated void retry(); @@ -604,7 +640,7 @@ public ExoPlayer build() { boolean getPauseAtEndOfMediaItems(); /** - * Sets whether audio offload scheduling is enabled. If enabled, ExoPlayer's main loop will as + * Sets whether audio offload scheduling is enabled. If enabled, ExoPlayer's main loop will run as * rarely as possible when playing an audio stream using audio offload. * *

    Only use this scheduling mode if the player is not displaying anything to the user. For @@ -633,9 +669,19 @@ public ExoPlayer build() { *

  • The {@link AudioSink} is playing with an offload {@link AudioTrack}. * * + *

    The state where ExoPlayer main loop has been paused to save power during offload playback + * can be queried with {@link #experimentalIsSleepingForOffload()}. + * *

    This method is experimental, and will be renamed or removed in a future release. * * @param offloadSchedulingEnabled Whether to enable offload scheduling. */ void experimentalSetOffloadSchedulingEnabled(boolean offloadSchedulingEnabled); + + /** + * Returns whether the player has paused its main loop to save power in offload scheduling mode. + * + * @see #experimentalSetOffloadSchedulingEnabled(boolean) + */ + boolean experimentalIsSleepingForOffload(); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerFactory.java b/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerFactory.java index dfe96ffa322..40c3473abdc 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerFactory.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerFactory.java @@ -256,8 +256,11 @@ public static ExoPlayer newInstance( /* analyticsCollector= */ null, /* useLazyPreparation= */ true, SeekParameters.DEFAULT, + new DefaultLivePlaybackSpeedControl.Builder().build(), + ExoPlayer.DEFAULT_RELEASE_TIMEOUT_MS, /* pauseAtEndOfMediaItems= */ false, Clock.DEFAULT, - applicationLooper); + applicationLooper, + /* wrappingPlayer= */ null); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerImpl.java b/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerImpl.java index 17463b3cd18..de8aa48891c 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerImpl.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerImpl.java @@ -28,27 +28,27 @@ import androidx.annotation.Nullable; import com.google.android.exoplayer2.PlayerMessage.Target; import com.google.android.exoplayer2.analytics.AnalyticsCollector; +import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.source.MediaSource; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.MediaSourceFactory; import com.google.android.exoplayer2.source.ShuffleOrder; import com.google.android.exoplayer2.source.TrackGroupArray; -import com.google.android.exoplayer2.source.ads.AdsMediaSource; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; import com.google.android.exoplayer2.trackselection.TrackSelector; import com.google.android.exoplayer2.trackselection.TrackSelectorResult; import com.google.android.exoplayer2.upstream.BandwidthMeter; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Clock; +import com.google.android.exoplayer2.util.HandlerWrapper; +import com.google.android.exoplayer2.util.ListenerSet; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.Util; -import java.util.ArrayDeque; +import com.google.common.collect.ImmutableList; import java.util.ArrayList; import java.util.Collections; import java.util.List; -import java.util.concurrent.CopyOnWriteArrayList; -import java.util.concurrent.TimeoutException; /** * An {@link ExoPlayer} implementation. Instances can be obtained from {@link ExoPlayer.Builder}. @@ -68,19 +68,18 @@ private final Renderer[] renderers; private final TrackSelector trackSelector; - private final Handler playbackInfoUpdateHandler; + private final HandlerWrapper playbackInfoUpdateHandler; private final ExoPlayerImplInternal.PlaybackInfoUpdateListener playbackInfoUpdateListener; private final ExoPlayerImplInternal internalPlayer; - private final Handler internalPlayerHandler; - private final CopyOnWriteArrayList listeners; + private final ListenerSet listeners; private final Timeline.Period period; - private final ArrayDeque pendingListenerNotifications; private final List mediaSourceHolderSnapshots; private final boolean useLazyPreparation; private final MediaSourceFactory mediaSourceFactory; @Nullable private final AnalyticsCollector analyticsCollector; private final Looper applicationLooper; private final BandwidthMeter bandwidthMeter; + private final Clock clock; @RepeatMode private int repeatMode; private boolean shuffleModeEnabled; @@ -92,7 +91,6 @@ private SeekParameters seekParameters; private ShuffleOrder shuffleOrder; private boolean pauseAtEndOfMediaItems; - private boolean hasAdsMediaSource; // Playback information when there is no pending seek/set source operation. private PlaybackInfo playbackInfo; @@ -115,10 +113,14 @@ * loads and other initial preparation steps happen immediately. If true, these initial * preparations are triggered only when the player starts buffering the media. * @param seekParameters The {@link SeekParameters}. + * @param livePlaybackSpeedControl The {@link LivePlaybackSpeedControl}. + * @param releaseTimeoutMs The timeout for calls to {@link #release()} in milliseconds. * @param pauseAtEndOfMediaItems Whether to pause playback at the end of each media item. * @param clock The {@link Clock}. * @param applicationLooper The {@link Looper} that must be used for all calls to the player and * which is used to call listeners on. + * @param wrappingPlayer The {@link Player} wrapping this one if applicable. This player instance + * should be used for all externally visible callbacks. */ @SuppressLint("HandlerLeak") public ExoPlayerImpl( @@ -130,11 +132,21 @@ public ExoPlayerImpl( @Nullable AnalyticsCollector analyticsCollector, boolean useLazyPreparation, SeekParameters seekParameters, + LivePlaybackSpeedControl livePlaybackSpeedControl, + long releaseTimeoutMs, boolean pauseAtEndOfMediaItems, Clock clock, - Looper applicationLooper) { - Log.i(TAG, "Init " + Integer.toHexString(System.identityHashCode(this)) + " [" - + ExoPlayerLibraryInfo.VERSION_SLASHY + "] [" + Util.DEVICE_DEBUG_INFO + "]"); + Looper applicationLooper, + @Nullable Player wrappingPlayer) { + Log.i( + TAG, + "Init " + + Integer.toHexString(System.identityHashCode(this)) + + " [" + + ExoPlayerLibraryInfo.VERSION_SLASHY + + "] [" + + Util.DEVICE_DEBUG_INFO + + "]"); checkState(renderers.length > 0); this.renderers = checkNotNull(renderers); this.trackSelector = checkNotNull(trackSelector); @@ -145,25 +157,31 @@ public ExoPlayerImpl( this.seekParameters = seekParameters; this.pauseAtEndOfMediaItems = pauseAtEndOfMediaItems; this.applicationLooper = applicationLooper; + this.clock = clock; repeatMode = Player.REPEAT_MODE_OFF; - listeners = new CopyOnWriteArrayList<>(); + Player playerForListeners = wrappingPlayer != null ? wrappingPlayer : this; + listeners = + new ListenerSet<>( + applicationLooper, + clock, + Player.Events::new, + (listener, eventFlags) -> listener.onEvents(playerForListeners, eventFlags)); mediaSourceHolderSnapshots = new ArrayList<>(); shuffleOrder = new ShuffleOrder.DefaultShuffleOrder(/* length= */ 0); emptyTrackSelectorResult = new TrackSelectorResult( new RendererConfiguration[renderers.length], - new TrackSelection[renderers.length], + new ExoTrackSelection[renderers.length], /* info= */ null); period = new Timeline.Period(); maskingWindowIndex = C.INDEX_UNSET; - playbackInfoUpdateHandler = new Handler(applicationLooper); + playbackInfoUpdateHandler = clock.createHandler(applicationLooper, /* callback= */ null); playbackInfoUpdateListener = playbackInfoUpdate -> playbackInfoUpdateHandler.post(() -> handlePlaybackInfo(playbackInfoUpdate)); playbackInfo = PlaybackInfo.createDummy(emptyTrackSelectorResult); - pendingListenerNotifications = new ArrayDeque<>(); if (analyticsCollector != null) { - analyticsCollector.setPlayer(this); + analyticsCollector.setPlayer(playerForListeners, applicationLooper); addListener(analyticsCollector); bandwidthMeter.addEventListener(new Handler(applicationLooper), analyticsCollector); } @@ -178,35 +196,26 @@ public ExoPlayerImpl( shuffleModeEnabled, analyticsCollector, seekParameters, + livePlaybackSpeedControl, + releaseTimeoutMs, pauseAtEndOfMediaItems, applicationLooper, clock, playbackInfoUpdateListener); - internalPlayerHandler = new Handler(internalPlayer.getPlaybackLooper()); } /** - * Set a limit on the time a call to {@link #release()} can spend. If a call to {@link #release()} - * takes more than {@code timeoutMs} milliseconds to complete, the player will raise an error via - * {@link Player.EventListener#onPlayerError}. + * Set a limit on the time a call to {@link #setForegroundMode} can spend. If a call to {@link + * #setForegroundMode} takes more than {@code timeoutMs} milliseconds to complete, the player will + * raise an error via {@link Player.EventListener#onPlayerError}. * *

    This method is experimental, and will be renamed or removed in a future release. It should * only be called before the player is used. * - * @param timeoutMs The time limit in milliseconds, or 0 for no limit. - */ - public void experimentalSetReleaseTimeoutMs(long timeoutMs) { - internalPlayer.experimentalSetReleaseTimeoutMs(timeoutMs); - } - - /** - * Configures the player to not throw when it detects it's stuck buffering. - * - *

    This method is experimental, and will be renamed or removed in a future release. It should - * only be called before the player is used. + * @param timeoutMs The time limit in milliseconds. */ - public void experimentalDisableThrowWhenStuckBuffering() { - internalPlayer.experimentalDisableThrowWhenStuckBuffering(); + public void experimentalSetForegroundModeTimeoutMs(long timeoutMs) { + internalPlayer.experimentalSetForegroundModeTimeoutMs(timeoutMs); } @Override @@ -214,6 +223,11 @@ public void experimentalSetOffloadSchedulingEnabled(boolean offloadSchedulingEna internalPlayer.experimentalSetOffloadSchedulingEnabled(offloadSchedulingEnabled); } + @Override + public boolean experimentalIsSleepingForOffload() { + return playbackInfo.sleepingForOffload; + } + @Override @Nullable public AudioComponent getAudioComponent() { @@ -254,20 +268,19 @@ public Looper getApplicationLooper() { return applicationLooper; } + @Override + public Clock getClock() { + return clock; + } + @Override public void addListener(Player.EventListener listener) { - Assertions.checkNotNull(listener); - listeners.addIfAbsent(new ListenerHolder(listener)); + listeners.add(listener); } @Override public void removeListener(Player.EventListener listener) { - for (ListenerHolder listenerHolder : listeners) { - if (listenerHolder.listener.equals(listener)) { - listenerHolder.release(); - listeners.remove(listenerHolder); - } - } + listeners.remove(listener); } @Override @@ -423,7 +436,6 @@ public void addMediaSources(List mediaSources) { @Override public void addMediaSources(int index, List mediaSources) { Assertions.checkArgument(index >= 0); - validateMediaSources(mediaSources, /* mediaSourceReplacement= */ false); Timeline oldTimeline = getCurrentTimeline(); pendingOperationAcks++; List holders = addMediaSourceHolders(index, mediaSources); @@ -561,7 +573,8 @@ public void setRepeatMode(@RepeatMode int repeatMode) { if (this.repeatMode != repeatMode) { this.repeatMode = repeatMode; internalPlayer.setRepeatMode(repeatMode); - notifyListeners(listener -> listener.onRepeatModeChanged(repeatMode)); + listeners.sendEvent( + Player.EVENT_REPEAT_MODE_CHANGED, listener -> listener.onRepeatModeChanged(repeatMode)); } } @@ -575,7 +588,9 @@ public void setShuffleModeEnabled(boolean shuffleModeEnabled) { if (this.shuffleModeEnabled != shuffleModeEnabled) { this.shuffleModeEnabled = shuffleModeEnabled; internalPlayer.setShuffleModeEnabled(shuffleModeEnabled); - notifyListeners(listener -> listener.onShuffleModeEnabledChanged(shuffleModeEnabled)); + listeners.sendEvent( + Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED, + listener -> listener.onShuffleModeEnabledChanged(shuffleModeEnabled)); } } @@ -672,18 +687,29 @@ public void setForegroundMode(boolean foregroundMode) { if (this.foregroundMode != foregroundMode) { this.foregroundMode = foregroundMode; if (!internalPlayer.setForegroundMode(foregroundMode)) { - notifyListeners( - listener -> - listener.onPlayerError( - ExoPlaybackException.createForTimeout( - new TimeoutException("Setting foreground mode timed out."), - ExoPlaybackException.TIMEOUT_OPERATION_SET_FOREGROUND_MODE))); + // One of the renderers timed out releasing its resources. + stop( + /* reset= */ false, + ExoPlaybackException.createForRenderer( + new ExoTimeoutException( + ExoTimeoutException.TIMEOUT_OPERATION_SET_FOREGROUND_MODE))); } } } @Override public void stop(boolean reset) { + stop(reset, /* error= */ null); + } + + /** + * Stops the player. + * + * @param reset Whether the playlist should be cleared and whether the playback position and + * playback error should be reset. + * @param error An optional {@link ExoPlaybackException} to set. + */ + public void stop(boolean reset, @Nullable ExoPlaybackException error) { PlaybackInfo playbackInfo; if (reset) { playbackInfo = @@ -696,6 +722,9 @@ public void stop(boolean reset) { playbackInfo.totalBufferedDurationUs = 0; } playbackInfo = playbackInfo.copyWithPlaybackState(Player.STATE_IDLE); + if (error != null) { + playbackInfo = playbackInfo.copyWithPlaybackError(error); + } pendingOperationAcks++; internalPlayer.stop(); updatePlaybackInfo( @@ -709,17 +738,27 @@ public void stop(boolean reset) { @Override public void release() { - Log.i(TAG, "Release " + Integer.toHexString(System.identityHashCode(this)) + " [" - + ExoPlayerLibraryInfo.VERSION_SLASHY + "] [" + Util.DEVICE_DEBUG_INFO + "] [" - + ExoPlayerLibraryInfo.registeredModules() + "]"); + Log.i( + TAG, + "Release " + + Integer.toHexString(System.identityHashCode(this)) + + " [" + + ExoPlayerLibraryInfo.VERSION_SLASHY + + "] [" + + Util.DEVICE_DEBUG_INFO + + "] [" + + ExoPlayerLibraryInfo.registeredModules() + + "]"); if (!internalPlayer.release()) { - notifyListeners( + // One of the renderers timed out releasing its resources. + listeners.sendEvent( + Player.EVENT_PLAYER_ERROR, listener -> listener.onPlayerError( - ExoPlaybackException.createForTimeout( - new TimeoutException("Player release timed out."), - ExoPlaybackException.TIMEOUT_OPERATION_RELEASE))); + ExoPlaybackException.createForRenderer( + new ExoTimeoutException(ExoTimeoutException.TIMEOUT_OPERATION_RELEASE)))); } + listeners.release(); playbackInfoUpdateHandler.removeCallbacksAndMessages(null); if (analyticsCollector != null) { bandwidthMeter.removeEventListener(analyticsCollector); @@ -737,7 +776,8 @@ public PlayerMessage createMessage(Target target) { target, playbackInfo.timeline, getCurrentWindowIndex(), - internalPlayerHandler); + clock, + internalPlayer.getPlaybackLooper()); } @Override @@ -865,7 +905,12 @@ public TrackGroupArray getCurrentTrackGroups() { @Override public TrackSelectionArray getCurrentTrackSelections() { - return playbackInfo.trackSelectorResult.selections; + return new TrackSelectionArray(playbackInfo.trackSelectorResult.selections); + } + + @Override + public List getCurrentStaticMetadata() { + return playbackInfo.staticMetadata; } @Override @@ -927,6 +972,8 @@ private void handlePlaybackInfo(ExoPlayerImplInternal.PlaybackInfoUpdate playbac } } + // Calling deprecated listeners. + @SuppressWarnings("deprecation") private void updatePlaybackInfo( PlaybackInfo playbackInfo, boolean positionDiscontinuity, @@ -934,39 +981,124 @@ private void updatePlaybackInfo( @TimelineChangeReason int timelineChangeReason, @PlayWhenReadyChangeReason int playWhenReadyChangeReason, boolean seekProcessed) { - // Assign playback info immediately such that all getters return the right values. + // Assign playback info immediately such that all getters return the right values, but keep + // snapshot of previous and new state so that listener invocations are triggered correctly. PlaybackInfo previousPlaybackInfo = this.playbackInfo; + PlaybackInfo newPlaybackInfo = playbackInfo; this.playbackInfo = playbackInfo; Pair mediaItemTransitionInfo = evaluateMediaItemTransitionReason( - playbackInfo, + newPlaybackInfo, previousPlaybackInfo, positionDiscontinuity, positionDiscontinuityReason, - !previousPlaybackInfo.timeline.equals(playbackInfo.timeline)); + !previousPlaybackInfo.timeline.equals(newPlaybackInfo.timeline)); boolean mediaItemTransitioned = mediaItemTransitionInfo.first; int mediaItemTransitionReason = mediaItemTransitionInfo.second; - @Nullable MediaItem newMediaItem = null; - if (mediaItemTransitioned && !playbackInfo.timeline.isEmpty()) { - int windowIndex = - playbackInfo.timeline.getPeriodByUid(playbackInfo.periodId.periodUid, period).windowIndex; - newMediaItem = playbackInfo.timeline.getWindow(windowIndex, window).mediaItem; - } - notifyListeners( - new PlaybackInfoUpdate( - playbackInfo, - previousPlaybackInfo, - listeners, - trackSelector, - positionDiscontinuity, - positionDiscontinuityReason, - timelineChangeReason, - mediaItemTransitioned, - mediaItemTransitionReason, - newMediaItem, - playWhenReadyChangeReason, - seekProcessed)); + if (!previousPlaybackInfo.timeline.equals(newPlaybackInfo.timeline)) { + listeners.queueEvent( + Player.EVENT_TIMELINE_CHANGED, + listener -> listener.onTimelineChanged(newPlaybackInfo.timeline, timelineChangeReason)); + } + if (positionDiscontinuity) { + listeners.queueEvent( + Player.EVENT_POSITION_DISCONTINUITY, + listener -> listener.onPositionDiscontinuity(positionDiscontinuityReason)); + } + if (mediaItemTransitioned) { + @Nullable final MediaItem mediaItem; + if (!newPlaybackInfo.timeline.isEmpty()) { + int windowIndex = + newPlaybackInfo.timeline.getPeriodByUid(newPlaybackInfo.periodId.periodUid, period) + .windowIndex; + mediaItem = newPlaybackInfo.timeline.getWindow(windowIndex, window).mediaItem; + } else { + mediaItem = null; + } + listeners.queueEvent( + Player.EVENT_MEDIA_ITEM_TRANSITION, + listener -> listener.onMediaItemTransition(mediaItem, mediaItemTransitionReason)); + } + if (previousPlaybackInfo.playbackError != newPlaybackInfo.playbackError + && newPlaybackInfo.playbackError != null) { + listeners.queueEvent( + Player.EVENT_PLAYER_ERROR, + listener -> listener.onPlayerError(newPlaybackInfo.playbackError)); + } + if (previousPlaybackInfo.trackSelectorResult != newPlaybackInfo.trackSelectorResult) { + trackSelector.onSelectionActivated(newPlaybackInfo.trackSelectorResult.info); + TrackSelectionArray newSelection = + new TrackSelectionArray(newPlaybackInfo.trackSelectorResult.selections); + listeners.queueEvent( + Player.EVENT_TRACKS_CHANGED, + listener -> listener.onTracksChanged(newPlaybackInfo.trackGroups, newSelection)); + } + if (!previousPlaybackInfo.staticMetadata.equals(newPlaybackInfo.staticMetadata)) { + listeners.queueEvent( + Player.EVENT_STATIC_METADATA_CHANGED, + listener -> listener.onStaticMetadataChanged(newPlaybackInfo.staticMetadata)); + } + if (previousPlaybackInfo.isLoading != newPlaybackInfo.isLoading) { + listeners.queueEvent( + Player.EVENT_IS_LOADING_CHANGED, + listener -> listener.onIsLoadingChanged(newPlaybackInfo.isLoading)); + } + if (previousPlaybackInfo.playbackState != newPlaybackInfo.playbackState + || previousPlaybackInfo.playWhenReady != newPlaybackInfo.playWhenReady) { + listeners.queueEvent( + /* eventFlag= */ C.INDEX_UNSET, + listener -> + listener.onPlayerStateChanged( + newPlaybackInfo.playWhenReady, newPlaybackInfo.playbackState)); + } + if (previousPlaybackInfo.playbackState != newPlaybackInfo.playbackState) { + listeners.queueEvent( + Player.EVENT_PLAYBACK_STATE_CHANGED, + listener -> listener.onPlaybackStateChanged(newPlaybackInfo.playbackState)); + } + if (previousPlaybackInfo.playWhenReady != newPlaybackInfo.playWhenReady) { + listeners.queueEvent( + Player.EVENT_PLAY_WHEN_READY_CHANGED, + listener -> + listener.onPlayWhenReadyChanged( + newPlaybackInfo.playWhenReady, playWhenReadyChangeReason)); + } + if (previousPlaybackInfo.playbackSuppressionReason + != newPlaybackInfo.playbackSuppressionReason) { + listeners.queueEvent( + Player.EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED, + listener -> + listener.onPlaybackSuppressionReasonChanged( + newPlaybackInfo.playbackSuppressionReason)); + } + if (isPlaying(previousPlaybackInfo) != isPlaying(newPlaybackInfo)) { + listeners.queueEvent( + Player.EVENT_IS_PLAYING_CHANGED, + listener -> listener.onIsPlayingChanged(isPlaying(newPlaybackInfo))); + } + if (!previousPlaybackInfo.playbackParameters.equals(newPlaybackInfo.playbackParameters)) { + listeners.queueEvent( + Player.EVENT_PLAYBACK_PARAMETERS_CHANGED, + listener -> listener.onPlaybackParametersChanged(newPlaybackInfo.playbackParameters)); + } + if (seekProcessed) { + listeners.queueEvent(/* eventFlag= */ C.INDEX_UNSET, EventListener::onSeekProcessed); + } + if (previousPlaybackInfo.offloadSchedulingEnabled != newPlaybackInfo.offloadSchedulingEnabled) { + listeners.queueEvent( + /* eventFlag= */ C.INDEX_UNSET, + listener -> + listener.onExperimentalOffloadSchedulingEnabledChanged( + newPlaybackInfo.offloadSchedulingEnabled)); + } + if (previousPlaybackInfo.sleepingForOffload != newPlaybackInfo.sleepingForOffload) { + listeners.queueEvent( + /* eventFlag= */ C.INDEX_UNSET, + listener -> + listener.onExperimentalSleepingForOffloadChanged(newPlaybackInfo.sleepingForOffload)); + } + listeners.flushEvents(); } private Pair evaluateMediaItemTransitionReason( @@ -1020,7 +1152,6 @@ private void setMediaSourcesInternal( int startWindowIndex, long startPositionMs, boolean resetToDefaultPosition) { - validateMediaSources(mediaSources, /* mediaSourceReplacement= */ true); int currentWindowIndex = getCurrentWindowIndexInternal(); long currentPositionMs = getCurrentPosition(); pendingOperationAcks++; @@ -1119,39 +1250,6 @@ private void removeMediaSourceHolders(int fromIndex, int toIndexExclusive) { mediaSourceHolderSnapshots.remove(i); } shuffleOrder = shuffleOrder.cloneAndRemove(fromIndex, toIndexExclusive); - if (mediaSourceHolderSnapshots.isEmpty()) { - hasAdsMediaSource = false; - } - } - - /** - * Validates media sources before any modification of the existing list of media sources is made. - * This way we can throw an exception before changing the state of the player in case of a - * validation failure. - * - * @param mediaSources The media sources to set or add. - * @param mediaSourceReplacement Whether the given media sources will replace existing ones. - */ - private void validateMediaSources( - List mediaSources, boolean mediaSourceReplacement) { - if (hasAdsMediaSource && !mediaSourceReplacement && !mediaSources.isEmpty()) { - // Adding media sources to an ads media source is not allowed - // (see https://github.com/google/ExoPlayer/issues/3750). - throw new IllegalStateException(); - } - int sizeAfterModification = - mediaSources.size() + (mediaSourceReplacement ? 0 : mediaSourceHolderSnapshots.size()); - for (int i = 0; i < mediaSources.size(); i++) { - MediaSource mediaSource = checkNotNull(mediaSources.get(i)); - if (mediaSource instanceof AdsMediaSource) { - if (sizeAfterModification > 1) { - // Ads media sources only allowed with a single source - // (see https://github.com/google/ExoPlayer/issues/3750). - throw new IllegalArgumentException(); - } - hasAdsMediaSource = true; - } - } } private Timeline createMaskingTimeline() { @@ -1175,7 +1273,8 @@ private PlaybackInfo maskTimelineAndPosition( /* requestedContentPositionUs= */ C.msToUs(maskingWindowPositionMs), /* totalBufferedDurationUs= */ 0, TrackGroupArray.EMPTY, - emptyTrackSelectorResult); + emptyTrackSelectorResult, + /* staticMetadata= */ ImmutableList.of()); playbackInfo = playbackInfo.copyWithLoadingMediaPeriodId(dummyMediaPeriodId); playbackInfo.bufferedPositionUs = playbackInfo.positionUs; return playbackInfo; @@ -1202,7 +1301,8 @@ private PlaybackInfo maskTimelineAndPosition( /* requestedContentPositionUs= */ newContentPositionUs, /* totalBufferedDurationUs= */ 0, playingPeriodChanged ? TrackGroupArray.EMPTY : playbackInfo.trackGroups, - playingPeriodChanged ? emptyTrackSelectorResult : playbackInfo.trackSelectorResult); + playingPeriodChanged ? emptyTrackSelectorResult : playbackInfo.trackSelectorResult, + playingPeriodChanged ? ImmutableList.of() : playbackInfo.staticMetadata); playbackInfo = playbackInfo.copyWithLoadingMediaPeriodId(newPeriodId); playbackInfo.bufferedPositionUs = newContentPositionUs; } else if (newContentPositionUs == oldContentPositionUs) { @@ -1226,7 +1326,8 @@ private PlaybackInfo maskTimelineAndPosition( /* requestedContentPositionUs= */ playbackInfo.positionUs, /* totalBufferedDurationUs= */ maskedBufferedPositionUs - playbackInfo.positionUs, playbackInfo.trackGroups, - playbackInfo.trackSelectorResult); + playbackInfo.trackSelectorResult, + playbackInfo.staticMetadata); playbackInfo = playbackInfo.copyWithLoadingMediaPeriodId(newPeriodId); playbackInfo.bufferedPositionUs = maskedBufferedPositionUs; } @@ -1248,7 +1349,8 @@ private PlaybackInfo maskTimelineAndPosition( /* requestedContentPositionUs= */ newContentPositionUs, maskedTotalBufferedDurationUs, playbackInfo.trackGroups, - playbackInfo.trackSelectorResult); + playbackInfo.trackSelectorResult, + playbackInfo.staticMetadata); playbackInfo.bufferedPositionUs = maskedBufferedPositionUs; } return playbackInfo; @@ -1313,23 +1415,6 @@ private Pair getPeriodPositionOrMaskWindowPosition( return timeline.getPeriodPosition(window, period, windowIndex, C.msToUs(windowPositionMs)); } - private void notifyListeners(ListenerInvocation listenerInvocation) { - CopyOnWriteArrayList listenerSnapshot = new CopyOnWriteArrayList<>(listeners); - notifyListeners(() -> invokeAll(listenerSnapshot, listenerInvocation)); - } - - private void notifyListeners(Runnable listenerNotificationRunnable) { - boolean isRunningRecursiveListenerNotification = !pendingListenerNotifications.isEmpty(); - pendingListenerNotifications.addLast(listenerNotificationRunnable); - if (isRunningRecursiveListenerNotification) { - return; - } - while (!pendingListenerNotifications.isEmpty()) { - pendingListenerNotifications.peekFirst().run(); - pendingListenerNotifications.removeFirst(); - } - } - private long periodPositionUsToWindowPositionMs(MediaPeriodId periodId, long positionUs) { long positionMs = C.usToMs(positionUs); playbackInfo.timeline.getPeriodByUid(periodId.periodUid, period); @@ -1337,166 +1422,10 @@ private long periodPositionUsToWindowPositionMs(MediaPeriodId periodId, long pos return positionMs; } - private static final class PlaybackInfoUpdate implements Runnable { - - private final PlaybackInfo playbackInfo; - private final CopyOnWriteArrayList listenerSnapshot; - private final TrackSelector trackSelector; - private final boolean positionDiscontinuity; - @DiscontinuityReason private final int positionDiscontinuityReason; - @TimelineChangeReason private final int timelineChangeReason; - private final boolean mediaItemTransitioned; - private final int mediaItemTransitionReason; - @Nullable private final MediaItem mediaItem; - @PlayWhenReadyChangeReason private final int playWhenReadyChangeReason; - private final boolean seekProcessed; - private final boolean playbackStateChanged; - private final boolean playbackErrorChanged; - private final boolean isLoadingChanged; - private final boolean timelineChanged; - private final boolean trackSelectorResultChanged; - private final boolean playWhenReadyChanged; - private final boolean playbackSuppressionReasonChanged; - private final boolean isPlayingChanged; - private final boolean playbackParametersChanged; - private final boolean offloadSchedulingEnabledChanged; - - public PlaybackInfoUpdate( - PlaybackInfo playbackInfo, - PlaybackInfo previousPlaybackInfo, - CopyOnWriteArrayList listeners, - TrackSelector trackSelector, - boolean positionDiscontinuity, - @DiscontinuityReason int positionDiscontinuityReason, - @TimelineChangeReason int timelineChangeReason, - boolean mediaItemTransitioned, - @MediaItemTransitionReason int mediaItemTransitionReason, - @Nullable MediaItem mediaItem, - @PlayWhenReadyChangeReason int playWhenReadyChangeReason, - boolean seekProcessed) { - this.playbackInfo = playbackInfo; - this.listenerSnapshot = new CopyOnWriteArrayList<>(listeners); - this.trackSelector = trackSelector; - this.positionDiscontinuity = positionDiscontinuity; - this.positionDiscontinuityReason = positionDiscontinuityReason; - this.timelineChangeReason = timelineChangeReason; - this.mediaItemTransitioned = mediaItemTransitioned; - this.mediaItemTransitionReason = mediaItemTransitionReason; - this.mediaItem = mediaItem; - this.playWhenReadyChangeReason = playWhenReadyChangeReason; - this.seekProcessed = seekProcessed; - playbackStateChanged = previousPlaybackInfo.playbackState != playbackInfo.playbackState; - playbackErrorChanged = - previousPlaybackInfo.playbackError != playbackInfo.playbackError - && playbackInfo.playbackError != null; - isLoadingChanged = previousPlaybackInfo.isLoading != playbackInfo.isLoading; - timelineChanged = !previousPlaybackInfo.timeline.equals(playbackInfo.timeline); - trackSelectorResultChanged = - previousPlaybackInfo.trackSelectorResult != playbackInfo.trackSelectorResult; - playWhenReadyChanged = previousPlaybackInfo.playWhenReady != playbackInfo.playWhenReady; - playbackSuppressionReasonChanged = - previousPlaybackInfo.playbackSuppressionReason != playbackInfo.playbackSuppressionReason; - isPlayingChanged = isPlaying(previousPlaybackInfo) != isPlaying(playbackInfo); - playbackParametersChanged = - !previousPlaybackInfo.playbackParameters.equals(playbackInfo.playbackParameters); - offloadSchedulingEnabledChanged = - previousPlaybackInfo.offloadSchedulingEnabled != playbackInfo.offloadSchedulingEnabled; - } - - @SuppressWarnings("deprecation") - @Override - public void run() { - if (timelineChanged) { - invokeAll( - listenerSnapshot, - listener -> listener.onTimelineChanged(playbackInfo.timeline, timelineChangeReason)); - } - if (positionDiscontinuity) { - invokeAll( - listenerSnapshot, - listener -> listener.onPositionDiscontinuity(positionDiscontinuityReason)); - } - if (mediaItemTransitioned) { - invokeAll( - listenerSnapshot, - listener -> listener.onMediaItemTransition(mediaItem, mediaItemTransitionReason)); - } - if (playbackErrorChanged) { - invokeAll(listenerSnapshot, listener -> listener.onPlayerError(playbackInfo.playbackError)); - } - if (trackSelectorResultChanged) { - trackSelector.onSelectionActivated(playbackInfo.trackSelectorResult.info); - invokeAll( - listenerSnapshot, - listener -> - listener.onTracksChanged( - playbackInfo.trackGroups, playbackInfo.trackSelectorResult.selections)); - } - if (isLoadingChanged) { - invokeAll( - listenerSnapshot, listener -> listener.onIsLoadingChanged(playbackInfo.isLoading)); - } - if (playbackStateChanged || playWhenReadyChanged) { - invokeAll( - listenerSnapshot, - listener -> - listener.onPlayerStateChanged( - playbackInfo.playWhenReady, playbackInfo.playbackState)); - } - if (playbackStateChanged) { - invokeAll( - listenerSnapshot, - listener -> listener.onPlaybackStateChanged(playbackInfo.playbackState)); - } - if (playWhenReadyChanged) { - invokeAll( - listenerSnapshot, - listener -> - listener.onPlayWhenReadyChanged( - playbackInfo.playWhenReady, playWhenReadyChangeReason)); - } - if (playbackSuppressionReasonChanged) { - invokeAll( - listenerSnapshot, - listener -> - listener.onPlaybackSuppressionReasonChanged( - playbackInfo.playbackSuppressionReason)); - } - if (isPlayingChanged) { - invokeAll( - listenerSnapshot, listener -> listener.onIsPlayingChanged(isPlaying(playbackInfo))); - } - if (playbackParametersChanged) { - invokeAll( - listenerSnapshot, - listener -> { - listener.onPlaybackParametersChanged(playbackInfo.playbackParameters); - }); - } - if (seekProcessed) { - invokeAll(listenerSnapshot, EventListener::onSeekProcessed); - } - if (offloadSchedulingEnabledChanged) { - invokeAll( - listenerSnapshot, - listener -> - listener.onExperimentalOffloadSchedulingEnabledChanged( - playbackInfo.offloadSchedulingEnabled)); - } - } - - private static boolean isPlaying(PlaybackInfo playbackInfo) { - return playbackInfo.playbackState == Player.STATE_READY - && playbackInfo.playWhenReady - && playbackInfo.playbackSuppressionReason == PLAYBACK_SUPPRESSION_REASON_NONE; - } - } - - private static void invokeAll( - CopyOnWriteArrayList listeners, ListenerInvocation listenerInvocation) { - for (ListenerHolder listenerHolder : listeners) { - listenerHolder.invoke(listenerInvocation); - } + private static boolean isPlaying(PlaybackInfo playbackInfo) { + return playbackInfo.playbackState == Player.STATE_READY + && playbackInfo.playWhenReady + && playbackInfo.playbackSuppressionReason == PLAYBACK_SUPPRESSION_REASON_NONE; } private static final class MediaSourceHolderSnapshot implements MediaSourceInfoHolder { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerImplInternal.java b/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerImplInternal.java index 38d57ac0bcc..755d7511c4e 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerImplInternal.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/ExoPlayerImplInternal.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2; +import static com.google.android.exoplayer2.util.Util.castNonNull; import static java.lang.Math.max; import static java.lang.Math.min; @@ -33,12 +34,13 @@ import com.google.android.exoplayer2.Player.PlaybackSuppressionReason; import com.google.android.exoplayer2.Player.RepeatMode; import com.google.android.exoplayer2.analytics.AnalyticsCollector; +import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.source.MediaPeriod; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.SampleStream; import com.google.android.exoplayer2.source.ShuffleOrder; import com.google.android.exoplayer2.source.TrackGroupArray; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelector; import com.google.android.exoplayer2.trackselection.TrackSelectorResult; import com.google.android.exoplayer2.upstream.BandwidthMeter; @@ -49,6 +51,7 @@ import com.google.android.exoplayer2.util.TraceUtil; import com.google.android.exoplayer2.util.Util; import com.google.common.base.Supplier; +import com.google.common.collect.ImmutableList; import java.io.IOException; import java.util.ArrayList; import java.util.Collections; @@ -142,6 +145,7 @@ public interface PlaybackInfoUpdateListener { private static final int MSG_PLAYLIST_UPDATE_REQUESTED = 22; private static final int MSG_SET_PAUSE_AT_END_OF_WINDOW = 23; private static final int MSG_SET_OFFLOAD_SCHEDULING_ENABLED = 24; + private static final int MSG_ATTEMPT_ERROR_RECOVERY = 25; private static final int ACTIVE_INTERVAL_MS = 10; private static final int IDLE_INTERVAL_MS = 1000; @@ -174,6 +178,8 @@ public interface PlaybackInfoUpdateListener { private final PlaybackInfoUpdateListener playbackInfoUpdateListener; private final MediaPeriodQueue queue; private final MediaSourceList mediaSourceList; + private final LivePlaybackSpeedControl livePlaybackSpeedControl; + private final long releaseTimeoutMs; @SuppressWarnings("unused") private SeekParameters seekParameters; @@ -183,7 +189,7 @@ public interface PlaybackInfoUpdateListener { private boolean released; private boolean pauseAtEndOfWindow; private boolean pendingPauseAtEndOfPeriod; - private boolean rebuffering; + private boolean isRebuffering; private boolean shouldContinueLoading; @Player.RepeatMode private int repeatMode; private boolean shuffleModeEnabled; @@ -196,9 +202,9 @@ public interface PlaybackInfoUpdateListener { private long rendererPositionUs; private int nextPendingMessageIndexHint; private boolean deliverPendingMessageAtStartPositionRequired; + @Nullable private ExoPlaybackException pendingRecoverableError; - private long releaseTimeoutMs; - private boolean throwWhenStuckBuffering; + private long setForegroundModeTimeoutMs; public ExoPlayerImplInternal( Renderer[] renderers, @@ -210,6 +216,8 @@ public ExoPlayerImplInternal( boolean shuffleModeEnabled, @Nullable AnalyticsCollector analyticsCollector, SeekParameters seekParameters, + LivePlaybackSpeedControl livePlaybackSpeedControl, + long releaseTimeoutMs, boolean pauseAtEndOfWindow, Looper applicationLooper, Clock clock, @@ -223,10 +231,12 @@ public ExoPlayerImplInternal( this.repeatMode = repeatMode; this.shuffleModeEnabled = shuffleModeEnabled; this.seekParameters = seekParameters; + this.livePlaybackSpeedControl = livePlaybackSpeedControl; + this.releaseTimeoutMs = releaseTimeoutMs; + this.setForegroundModeTimeoutMs = releaseTimeoutMs; this.pauseAtEndOfWindow = pauseAtEndOfWindow; this.clock = clock; - throwWhenStuckBuffering = true; backBufferDurationUs = loadControl.getBackBufferDurationUs(); retainBackBufferFromKeyframe = loadControl.retainBackBufferFromKeyframe(); @@ -257,12 +267,8 @@ public ExoPlayerImplInternal( handler = clock.createHandler(playbackLooper, this); } - public void experimentalSetReleaseTimeoutMs(long releaseTimeoutMs) { - this.releaseTimeoutMs = releaseTimeoutMs; - } - - public void experimentalDisableThrowWhenStuckBuffering() { - throwWhenStuckBuffering = false; + public void experimentalSetForegroundModeTimeoutMs(long setForegroundModeTimeoutMs) { + this.setForegroundModeTimeoutMs = setForegroundModeTimeoutMs; } public void experimentalSetOffloadSchedulingEnabled(boolean offloadSchedulingEnabled) { @@ -369,6 +375,12 @@ public synchronized void sendMessage(PlayerMessage message) { handler.obtainMessage(MSG_SEND_MESSAGE, message).sendToTarget(); } + /** + * Sets the foreground mode. + * + * @param foregroundMode Whether foreground mode should be enabled. + * @return Whether the operations succeeded. If false, the operation timed out. + */ public synchronized boolean setForegroundMode(boolean foregroundMode) { if (released || !internalPlaybackThread.isAlive()) { return true; @@ -381,26 +393,22 @@ public synchronized boolean setForegroundMode(boolean foregroundMode) { handler .obtainMessage(MSG_SET_FOREGROUND_MODE, /* foregroundMode */ 0, 0, processedFlag) .sendToTarget(); - if (releaseTimeoutMs > 0) { - waitUninterruptibly(/* condition= */ processedFlag::get, releaseTimeoutMs); - } else { - waitUninterruptibly(/* condition= */ processedFlag::get); - } + waitUninterruptibly(/* condition= */ processedFlag::get, setForegroundModeTimeoutMs); return processedFlag.get(); } } + /** + * Releases the player. + * + * @return Whether the release succeeded. If false, the release timed out. + */ public synchronized boolean release() { if (released || !internalPlaybackThread.isAlive()) { return true; } - handler.sendEmptyMessage(MSG_RELEASE); - if (releaseTimeoutMs > 0) { - waitUninterruptibly(/* condition= */ () -> released, releaseTimeoutMs); - } else { - waitUninterruptibly(/* condition= */ () -> released); - } + waitUninterruptibly(/* condition= */ () -> released, releaseTimeoutMs); return released; } @@ -438,7 +446,9 @@ public void onTrackSelectionsInvalidated() { @Override public void onPlaybackParametersChanged(PlaybackParameters newPlaybackParameters) { - sendPlaybackParametersChangedInternal(newPlaybackParameters, /* acknowledgeCommand= */ false); + handler + .obtainMessage(MSG_PLAYBACK_PARAMETERS_CHANGED_INTERNAL, newPlaybackParameters) + .sendToTarget(); } // Handler.Callback implementation. @@ -492,8 +502,7 @@ public boolean handleMessage(Message msg) { reselectTracksInternal(); break; case MSG_PLAYBACK_PARAMETERS_CHANGED_INTERNAL: - handlePlaybackParameters( - (PlaybackParameters) msg.obj, /* acknowledgeCommand= */ msg.arg1 != 0); + handlePlaybackParameters((PlaybackParameters) msg.obj, /* acknowledgeCommand= */ false); break; case MSG_SEND_MESSAGE: sendMessageInternal((PlayerMessage) msg.obj); @@ -525,6 +534,9 @@ public boolean handleMessage(Message msg) { case MSG_SET_OFFLOAD_SCHEDULING_ENABLED: setOffloadSchedulingEnabledInternal(msg.arg1 == 1); break; + case MSG_ATTEMPT_ERROR_RECOVERY: + attemptErrorRecovery((ExoPlaybackException) msg.obj); + break; case MSG_RELEASE: releaseInternal(); // Return immediately to not send playback info updates after release. @@ -542,9 +554,22 @@ public boolean handleMessage(Message msg) { e = e.copyWithMediaPeriodId(readingPeriod.info.id); } } - Log.e(TAG, "Playback error", e); - stopInternal(/* forceResetRenderers= */ true, /* acknowledgeStop= */ false); - playbackInfo = playbackInfo.copyWithPlaybackError(e); + if (e.isRecoverable && pendingRecoverableError == null) { + Log.w(TAG, "Recoverable playback error", e); + pendingRecoverableError = e; + Message message = handler.obtainMessage(MSG_ATTEMPT_ERROR_RECOVERY, e); + // Given that the player is now in an unhandled exception state, the error needs to be + // recovered or the player stopped before any other message is handled. + message.getTarget().sendMessageAtFrontOfQueue(message); + } else { + if (pendingRecoverableError != null) { + e.addSuppressed(pendingRecoverableError); + pendingRecoverableError = null; + } + Log.e(TAG, "Playback error", e); + stopInternal(/* forceResetRenderers= */ true, /* acknowledgeStop= */ false); + playbackInfo = playbackInfo.copyWithPlaybackError(e); + } maybeNotifyPlaybackInfoChanged(); } catch (IOException e) { ExoPlaybackException error = ExoPlaybackException.createForSource(e); @@ -557,11 +582,8 @@ public boolean handleMessage(Message msg) { stopInternal(/* forceResetRenderers= */ false, /* acknowledgeStop= */ false); playbackInfo = playbackInfo.copyWithPlaybackError(error); maybeNotifyPlaybackInfoChanged(); - } catch (RuntimeException | OutOfMemoryError e) { - ExoPlaybackException error = - e instanceof OutOfMemoryError - ? ExoPlaybackException.createForOutOfMemory((OutOfMemoryError) e) - : ExoPlaybackException.createForUnexpected((RuntimeException) e); + } catch (RuntimeException e) { + ExoPlaybackException error = ExoPlaybackException.createForUnexpected(e); Log.e(TAG, "Playback error", error); stopInternal(/* forceResetRenderers= */ true, /* acknowledgeStop= */ false); playbackInfo = playbackInfo.copyWithPlaybackError(error); @@ -572,26 +594,16 @@ public boolean handleMessage(Message msg) { // Private methods. - /** - * Blocks the current thread until a condition becomes true. - * - *

    If the current thread is interrupted while waiting for the condition to become true, this - * method will restore the interrupt after the condition became true. - * - * @param condition The condition. - */ - private synchronized void waitUninterruptibly(Supplier condition) { - boolean wasInterrupted = false; - while (!condition.get()) { - try { - wait(); - } catch (InterruptedException e) { - wasInterrupted = true; - } - } - if (wasInterrupted) { - // Restore the interrupted status. - Thread.currentThread().interrupt(); + private void attemptErrorRecovery(ExoPlaybackException exceptionToRecoverFrom) + throws ExoPlaybackException { + Assertions.checkArgument( + exceptionToRecoverFrom.isRecoverable + && exceptionToRecoverFrom.type == ExoPlaybackException.TYPE_RENDERER); + try { + seekToCurrentPosition(/* sendDiscontinuity= */ true); + } catch (Exception e) { + exceptionToRecoverFrom.addSuppressed(e); + throw exceptionToRecoverFrom; } } @@ -710,6 +722,18 @@ private void setShuffleOrderInternal(ShuffleOrder shuffleOrder) throws ExoPlayba handleMediaSourceListInfoRefreshed(timeline); } + private void notifyTrackSelectionPlayWhenReadyChanged(boolean playWhenReady) { + MediaPeriodHolder periodHolder = queue.getPlayingPeriod(); + while (periodHolder != null) { + for (ExoTrackSelection trackSelection : periodHolder.getTrackSelectorResult().selections) { + if (trackSelection != null) { + trackSelection.onPlayWhenReadyChanged(playWhenReady); + } + } + periodHolder = periodHolder.getNext(); + } + } + private void setPlayWhenReadyInternal( boolean playWhenReady, @PlaybackSuppressionReason int playbackSuppressionReason, @@ -719,7 +743,8 @@ private void setPlayWhenReadyInternal( playbackInfoUpdate.incrementPendingOperationAcks(operationAck ? 1 : 0); playbackInfoUpdate.setPlayWhenReadyChangeReason(reason); playbackInfo = playbackInfo.copyWithPlayWhenReady(playWhenReady, playbackSuppressionReason); - rebuffering = false; + isRebuffering = false; + notifyTrackSelectionPlayWhenReadyChanged(playWhenReady); if (!shouldPlayWhenReady()) { stopRenderers(); updatePlaybackPositions(); @@ -797,7 +822,7 @@ private void seekToCurrentPosition(boolean sendDiscontinuity) throws ExoPlayback } private void startRenderers() throws ExoPlaybackException { - rebuffering = false; + isRebuffering = false; mediaClock.start(); for (Renderer renderer : renderers) { if (isRendererEnabled(renderer)) { @@ -851,6 +876,36 @@ private void updatePlaybackPositions() throws ExoPlaybackException { MediaPeriodHolder loadingPeriod = queue.getLoadingPeriod(); playbackInfo.bufferedPositionUs = loadingPeriod.getBufferedPositionUs(); playbackInfo.totalBufferedDurationUs = getTotalBufferedDurationUs(); + + // Adjust live playback speed to new position. + if (playbackInfo.playWhenReady + && playbackInfo.playbackState == Player.STATE_READY + && isCurrentPeriodInMovingLiveWindow() + && playbackInfo.playbackParameters.speed == 1f) { + float adjustedSpeed = + livePlaybackSpeedControl.getAdjustedPlaybackSpeed( + getCurrentLiveOffsetUs(), getTotalBufferedDurationUs()); + if (mediaClock.getPlaybackParameters().speed != adjustedSpeed) { + mediaClock.setPlaybackParameters(playbackInfo.playbackParameters.withSpeed(adjustedSpeed)); + handlePlaybackParameters( + playbackInfo.playbackParameters, + /* currentPlaybackSpeed= */ mediaClock.getPlaybackParameters().speed, + /* updatePlaybackInfo= */ false, + /* acknowledgeCommand= */ false); + } + } + } + + private void notifyTrackSelectionRebuffer() { + MediaPeriodHolder periodHolder = queue.getPlayingPeriod(); + while (periodHolder != null) { + for (ExoTrackSelection trackSelection : periodHolder.getTrackSelectorResult().selections) { + if (trackSelection != null) { + trackSelection.onRebuffer(); + } + } + periodHolder = periodHolder.getNext(); + } } private void doSomeWork() throws ExoPlaybackException, IOException { @@ -929,13 +984,18 @@ private void doSomeWork() throws ExoPlaybackException, IOException { } else if (playbackInfo.playbackState == Player.STATE_BUFFERING && shouldTransitionToReadyState(renderersAllowPlayback)) { setState(Player.STATE_READY); + pendingRecoverableError = null; // Any pending error was successfully recovered from. if (shouldPlayWhenReady()) { startRenderers(); } } else if (playbackInfo.playbackState == Player.STATE_READY && !(enabledRendererCount == 0 ? isTimelineReady() : renderersAllowPlayback)) { - rebuffering = shouldPlayWhenReady(); + isRebuffering = shouldPlayWhenReady(); setState(Player.STATE_BUFFERING); + if (isRebuffering) { + notifyTrackSelectionRebuffer(); + livePlaybackSpeedControl.notifyRebuffer(); + } stopRenderers(); } @@ -946,8 +1006,7 @@ && shouldTransitionToReadyState(renderersAllowPlayback)) { renderers[i].maybeThrowStreamError(); } } - if (throwWhenStuckBuffering - && !playbackInfo.isLoading + if (!playbackInfo.isLoading && playbackInfo.totalBufferedDurationUs < 500_000 && isLoadingPossible()) { // Throw if the LoadControl prevents loading even if the buffer is empty or almost empty. We @@ -960,30 +1019,63 @@ && isLoadingPossible()) { playbackInfo = playbackInfo.copyWithOffloadSchedulingEnabled(offloadSchedulingEnabled); } + boolean sleepingForOffload = false; if ((shouldPlayWhenReady() && playbackInfo.playbackState == Player.STATE_READY) || playbackInfo.playbackState == Player.STATE_BUFFERING) { - maybeScheduleWakeup(operationStartTimeMs, ACTIVE_INTERVAL_MS); + sleepingForOffload = !maybeScheduleWakeup(operationStartTimeMs, ACTIVE_INTERVAL_MS); } else if (enabledRendererCount != 0 && playbackInfo.playbackState != Player.STATE_ENDED) { scheduleNextWork(operationStartTimeMs, IDLE_INTERVAL_MS); } else { handler.removeMessages(MSG_DO_SOME_WORK); } + if (playbackInfo.sleepingForOffload != sleepingForOffload) { + playbackInfo = playbackInfo.copyWithSleepingForOffload(sleepingForOffload); + } requestForRendererSleep = false; // A sleep request is only valid for the current doSomeWork. TraceUtil.endSection(); } + private long getCurrentLiveOffsetUs() { + return getLiveOffsetUs( + playbackInfo.timeline, playbackInfo.periodId.periodUid, playbackInfo.positionUs); + } + + private long getLiveOffsetUs(Timeline timeline, Object periodUid, long periodPositionUs) { + int windowIndex = timeline.getPeriodByUid(periodUid, period).windowIndex; + timeline.getWindow(windowIndex, window); + if (window.windowStartTimeMs == C.TIME_UNSET || !window.isLive() || !window.isDynamic) { + return C.TIME_UNSET; + } + return C.msToUs(window.getCurrentUnixTimeMs() - window.windowStartTimeMs) + - (periodPositionUs + period.getPositionInWindowUs()); + } + + private boolean isCurrentPeriodInMovingLiveWindow() { + return isInMovingLiveWindow(playbackInfo.timeline, playbackInfo.periodId); + } + + private boolean isInMovingLiveWindow(Timeline timeline, MediaPeriodId mediaPeriodId) { + if (mediaPeriodId.isAd() || timeline.isEmpty()) { + return false; + } + int windowIndex = timeline.getPeriodByUid(mediaPeriodId.periodUid, period).windowIndex; + timeline.getWindow(windowIndex, window); + return window.isLive() && window.isDynamic; + } + private void scheduleNextWork(long thisOperationStartTimeMs, long intervalMs) { handler.removeMessages(MSG_DO_SOME_WORK); handler.sendEmptyMessageAtTime(MSG_DO_SOME_WORK, thisOperationStartTimeMs + intervalMs); } - private void maybeScheduleWakeup(long operationStartTimeMs, long intervalMs) { + private boolean maybeScheduleWakeup(long operationStartTimeMs, long intervalMs) { if (offloadSchedulingEnabled && requestForRendererSleep) { - return; + return false; } scheduleNextWork(operationStartTimeMs, intervalMs); + return true; } private void seekToInternal(SeekPosition seekPosition) throws ExoPlaybackException { @@ -991,7 +1083,7 @@ private void seekToInternal(SeekPosition seekPosition) throws ExoPlaybackExcepti MediaPeriodId periodId; long periodPositionUs; - long requestedContentPosition; + long requestedContentPositionUs; boolean seekPositionAdjusted; @Nullable Pair resolvedSeekPosition = @@ -1010,17 +1102,17 @@ private void seekToInternal(SeekPosition seekPosition) throws ExoPlaybackExcepti getPlaceholderFirstMediaPeriodPosition(playbackInfo.timeline); periodId = firstPeriodAndPosition.first; periodPositionUs = firstPeriodAndPosition.second; - requestedContentPosition = C.TIME_UNSET; + requestedContentPositionUs = C.TIME_UNSET; seekPositionAdjusted = !playbackInfo.timeline.isEmpty(); } else { // Update the resolved seek position to take ads into account. Object periodUid = resolvedSeekPosition.first; - long resolvedContentPosition = resolvedSeekPosition.second; - requestedContentPosition = - seekPosition.windowPositionUs == C.TIME_UNSET ? C.TIME_UNSET : resolvedContentPosition; + long resolvedContentPositionUs = resolvedSeekPosition.second; + requestedContentPositionUs = + seekPosition.windowPositionUs == C.TIME_UNSET ? C.TIME_UNSET : resolvedContentPositionUs; periodId = queue.resolveMediaPeriodIdForAds( - playbackInfo.timeline, periodUid, resolvedContentPosition); + playbackInfo.timeline, periodUid, resolvedContentPositionUs); if (periodId.isAd()) { playbackInfo.timeline.getPeriodByUid(periodId.periodUid, period); periodPositionUs = @@ -1029,7 +1121,7 @@ private void seekToInternal(SeekPosition seekPosition) throws ExoPlaybackExcepti : 0; seekPositionAdjusted = true; } else { - periodPositionUs = resolvedContentPosition; + periodPositionUs = resolvedContentPositionUs; seekPositionAdjusted = seekPosition.windowPositionUs == C.TIME_UNSET; } } @@ -1075,10 +1167,16 @@ private void seekToInternal(SeekPosition seekPosition) throws ExoPlaybackExcepti /* forceBufferingState= */ playbackInfo.playbackState == Player.STATE_ENDED); seekPositionAdjusted |= periodPositionUs != newPeriodPositionUs; periodPositionUs = newPeriodPositionUs; + updateLivePlaybackSpeedControl( + /* newTimeline= */ playbackInfo.timeline, + /* newPeriodId= */ periodId, + /* oldTimeline= */ playbackInfo.timeline, + /* oldPeriodId= */ playbackInfo.periodId, + /* positionForTargetOffsetOverrideUs= */ requestedContentPositionUs); } } finally { playbackInfo = - handlePositionDiscontinuity(periodId, periodPositionUs, requestedContentPosition); + handlePositionDiscontinuity(periodId, periodPositionUs, requestedContentPositionUs); if (seekPositionAdjusted) { playbackInfoUpdate.setPositionDiscontinuity(Player.DISCONTINUITY_REASON_SEEK_ADJUSTMENT); } @@ -1103,7 +1201,7 @@ private long seekToPeriodPosition( boolean forceBufferingState) throws ExoPlaybackException { stopRenderers(); - rebuffering = false; + isRebuffering = false; if (forceBufferingState || playbackInfo.playbackState == Player.STATE_READY) { setState(Player.STATE_BUFFERING); } @@ -1184,10 +1282,10 @@ private void resetRendererPosition(long periodPositionUs) throws ExoPlaybackExce notifyTrackSelectionDiscontinuity(); } - private void setPlaybackParametersInternal(PlaybackParameters playbackParameters) { + private void setPlaybackParametersInternal(PlaybackParameters playbackParameters) + throws ExoPlaybackException { mediaClock.setPlaybackParameters(playbackParameters); - sendPlaybackParametersChangedInternal( - mediaClock.getPlaybackParameters(), /* acknowledgeCommand= */ true); + handlePlaybackParameters(mediaClock.getPlaybackParameters(), /* acknowledgeCommand= */ true); } private void setSeekParametersInternal(SeekParameters seekParameters) { @@ -1246,7 +1344,7 @@ private void resetInternal( boolean releaseMediaSourceList, boolean resetError) { handler.removeMessages(MSG_DO_SOME_WORK); - rebuffering = false; + isRebuffering = false; mediaClock.stop(); rendererPositionUs = 0; for (Renderer renderer : renderers) { @@ -1301,6 +1399,7 @@ private void resetInternal( /* isLoading= */ false, resetTrackInfo ? TrackGroupArray.EMPTY : playbackInfo.trackGroups, resetTrackInfo ? emptyTrackSelectorResult : playbackInfo.trackSelectorResult, + resetTrackInfo ? ImmutableList.of() : playbackInfo.staticMetadata, mediaPeriodId, playbackInfo.playWhenReady, playbackInfo.playbackSuppressionReason, @@ -1308,10 +1407,12 @@ private void resetInternal( startPositionUs, /* totalBufferedDurationUs= */ 0, startPositionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + /* sleepingForOffload= */ false); if (releaseMediaSourceList) { mediaSourceList.release(); } + pendingRecoverableError = null; } private Pair getPlaceholderFirstMediaPeriodPosition(Timeline timeline) { @@ -1364,7 +1465,7 @@ private void sendMessageInternal(PlayerMessage message) throws ExoPlaybackExcept } private void sendMessageToTarget(PlayerMessage message) throws ExoPlaybackException { - if (message.getHandler().getLooper() == playbackLooper) { + if (message.getLooper() == playbackLooper) { deliverMessage(message); if (playbackInfo.playbackState == Player.STATE_READY || playbackInfo.playbackState == Player.STATE_BUFFERING) { @@ -1377,21 +1478,23 @@ private void sendMessageToTarget(PlayerMessage message) throws ExoPlaybackExcept } private void sendMessageToTargetThread(final PlayerMessage message) { - Handler handler = message.getHandler(); - if (!handler.getLooper().getThread().isAlive()) { + Looper looper = message.getLooper(); + if (!looper.getThread().isAlive()) { Log.w("TAG", "Trying to send message on a dead thread."); message.markAsProcessed(/* isDelivered= */ false); return; } - handler.post( - () -> { - try { - deliverMessage(message); - } catch (ExoPlaybackException e) { - Log.e(TAG, "Unexpected error delivering message on external thread.", e); - throw new RuntimeException(e); - } - }); + clock + .createHandler(looper, /* callback= */ null) + .post( + () -> { + try { + deliverMessage(message); + } catch (ExoPlaybackException e) { + Log.e(TAG, "Unexpected error delivering message on external thread.", e); + throw new RuntimeException(e); + } + }); } private void deliverMessage(PlayerMessage message) throws ExoPlaybackException { @@ -1586,8 +1689,7 @@ private void reselectTracksInternal() throws ExoPlaybackException { private void updateTrackSelectionPlaybackSpeed(float playbackSpeed) { MediaPeriodHolder periodHolder = queue.getPlayingPeriod(); while (periodHolder != null) { - TrackSelection[] trackSelections = periodHolder.getTrackSelectorResult().selections.getAll(); - for (TrackSelection trackSelection : trackSelections) { + for (ExoTrackSelection trackSelection : periodHolder.getTrackSelectorResult().selections) { if (trackSelection != null) { trackSelection.onPlaybackSpeed(playbackSpeed); } @@ -1599,8 +1701,7 @@ private void updateTrackSelectionPlaybackSpeed(float playbackSpeed) { private void notifyTrackSelectionDiscontinuity() { MediaPeriodHolder periodHolder = queue.getPlayingPeriod(); while (periodHolder != null) { - TrackSelection[] trackSelections = periodHolder.getTrackSelectorResult().selections.getAll(); - for (TrackSelection trackSelection : trackSelections) { + for (ExoTrackSelection trackSelection : periodHolder.getTrackSelectorResult().selections) { if (trackSelection != null) { trackSelection.onDiscontinuity(); } @@ -1623,11 +1724,23 @@ private boolean shouldTransitionToReadyState(boolean renderersReadyOrEnded) { return true; } // Renderers are ready and we're loading. Ask the LoadControl whether to transition. + long targetLiveOffsetUs = + isInMovingLiveWindow(playbackInfo.timeline, queue.getPlayingPeriod().info.id) + ? livePlaybackSpeedControl.getTargetLiveOffsetUs() + : C.TIME_UNSET; MediaPeriodHolder loadingHolder = queue.getLoadingPeriod(); - boolean bufferedToEnd = loadingHolder.isFullyBuffered() && loadingHolder.info.isFinal; - return bufferedToEnd + boolean isBufferedToEnd = loadingHolder.isFullyBuffered() && loadingHolder.info.isFinal; + // Ad loader implementations may only load ad media once playback has nearly reached the ad, but + // it is possible for playback to be stuck buffering waiting for this. Therefore, we start + // playback regardless of buffered duration if we are waiting for an ad media period to prepare. + boolean isAdPendingPreparation = loadingHolder.info.id.isAd() && !loadingHolder.prepared; + return isBufferedToEnd + || isAdPendingPreparation || loadControl.shouldStartPlayback( - getTotalBufferedDurationUs(), mediaClock.getPlaybackParameters().speed, rebuffering); + getTotalBufferedDurationUs(), + mediaClock.getPlaybackParameters().speed, + isRebuffering, + targetLiveOffsetUs); } private boolean isTimelineReady() { @@ -1687,6 +1800,14 @@ timeline, rendererPositionUs, getMaxRendererReadPositionUs())) { newPositionUs = seekToPeriodPosition(newPeriodId, newPositionUs, forceBufferingState); } } finally { + updateLivePlaybackSpeedControl( + /* newTimeline= */ timeline, + newPeriodId, + /* oldTimeline= */ playbackInfo.timeline, + /* oldPeriodId= */ playbackInfo.periodId, + /* positionForTargetOffsetOverrideUs */ positionUpdate.setTargetLiveOffset + ? newPositionUs + : C.TIME_UNSET); if (periodPositionChanged || newRequestedContentPositionUs != playbackInfo.requestedContentPositionUs) { playbackInfo = @@ -1704,6 +1825,36 @@ timeline, rendererPositionUs, getMaxRendererReadPositionUs())) { } } + private void updateLivePlaybackSpeedControl( + Timeline newTimeline, + MediaPeriodId newPeriodId, + Timeline oldTimeline, + MediaPeriodId oldPeriodId, + long positionForTargetOffsetOverrideUs) { + if (newTimeline.isEmpty() || !isInMovingLiveWindow(newTimeline, newPeriodId)) { + // Live playback speed control is unused. + return; + } + int windowIndex = newTimeline.getPeriodByUid(newPeriodId.periodUid, period).windowIndex; + newTimeline.getWindow(windowIndex, window); + livePlaybackSpeedControl.setLiveConfiguration(castNonNull(window.liveConfiguration)); + if (positionForTargetOffsetOverrideUs != C.TIME_UNSET) { + livePlaybackSpeedControl.setTargetLiveOffsetOverrideUs( + getLiveOffsetUs(newTimeline, newPeriodId.periodUid, positionForTargetOffsetOverrideUs)); + } else { + Object windowUid = window.uid; + @Nullable Object oldWindowUid = null; + if (!oldTimeline.isEmpty()) { + int oldWindowIndex = oldTimeline.getPeriodByUid(oldPeriodId.periodUid, period).windowIndex; + oldWindowUid = oldTimeline.getWindow(oldWindowIndex, window).uid; + } + if (!Util.areEqual(oldWindowUid, windowUid)) { + // Reset overridden target live offset to media values if window changes. + livePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(C.TIME_UNSET); + } + } + } + private long getMaxRendererReadPositionUs() { MediaPeriodHolder readingHolder = queue.getReadingPeriod(); if (readingHolder == null) { @@ -1867,7 +2018,7 @@ private boolean replaceStreamsOrDisableRendererForTransition() throws ExoPlaybac } if (!renderer.isCurrentStreamFinal()) { // The renderer stream is not final, so we can replace the sample streams immediately. - Format[] formats = getFormats(newTrackSelectorResult.selections.get(i)); + Format[] formats = getFormats(newTrackSelectorResult.selections[i]); renderer.replaceStream( formats, readingPeriodHolder.sampleStreams[i], @@ -1903,6 +2054,12 @@ private void maybeUpdatePlayingPeriod() throws ExoPlaybackException { ? Player.DISCONTINUITY_REASON_PERIOD_TRANSITION : Player.DISCONTINUITY_REASON_AD_INSERTION; playbackInfoUpdate.setPositionDiscontinuity(discontinuityReason); + updateLivePlaybackSpeedControl( + /* newTimeline= */ playbackInfo.timeline, + /* newPeriodId= */ newPlayingPeriodHolder.info.id, + /* oldTimeline= */ playbackInfo.timeline, + /* oldPeriodId= */ oldPlayingPeriodHolder.info.id, + /* positionForTargetOffsetOverrideUs= */ C.TIME_UNSET); resetPendingPauseAtEndOfPeriod(); updatePlaybackPositions(); advancedPlayingPeriod = true; @@ -1992,12 +2149,30 @@ private void handleContinueLoadingRequested(MediaPeriod mediaPeriod) { private void handlePlaybackParameters( PlaybackParameters playbackParameters, boolean acknowledgeCommand) throws ExoPlaybackException { - playbackInfoUpdate.incrementPendingOperationAcks(acknowledgeCommand ? 1 : 0); - playbackInfo = playbackInfo.copyWithPlaybackParameters(playbackParameters); + handlePlaybackParameters( + playbackParameters, + playbackParameters.speed, + /* updatePlaybackInfo= */ true, + acknowledgeCommand); + } + + private void handlePlaybackParameters( + PlaybackParameters playbackParameters, + float currentPlaybackSpeed, + boolean updatePlaybackInfo, + boolean acknowledgeCommand) + throws ExoPlaybackException { + if (updatePlaybackInfo) { + if (acknowledgeCommand) { + playbackInfoUpdate.incrementPendingOperationAcks(1); + } + playbackInfo = playbackInfo.copyWithPlaybackParameters(playbackParameters); + } updateTrackSelectionPlaybackSpeed(playbackParameters.speed); for (Renderer renderer : renderers) { if (renderer != null) { - renderer.setOperatingRate(playbackParameters.speed); + renderer.setPlaybackSpeed( + currentPlaybackSpeed, /* targetPlaybackSpeed= */ playbackParameters.speed); } } } @@ -2057,6 +2232,7 @@ private PlaybackInfo handlePositionDiscontinuity( resetPendingPauseAtEndOfPeriod(); TrackGroupArray trackGroupArray = playbackInfo.trackGroups; TrackSelectorResult trackSelectorResult = playbackInfo.trackSelectorResult; + List staticMetadata = playbackInfo.staticMetadata; if (mediaSourceList.isPrepared()) { @Nullable MediaPeriodHolder playingPeriodHolder = queue.getPlayingPeriod(); trackGroupArray = @@ -2067,18 +2243,46 @@ private PlaybackInfo handlePositionDiscontinuity( playingPeriodHolder == null ? emptyTrackSelectorResult : playingPeriodHolder.getTrackSelectorResult(); + staticMetadata = extractMetadataFromTrackSelectionArray(trackSelectorResult.selections); + // Ensure the media period queue requested content position matches the new playback info. + if (playingPeriodHolder != null + && playingPeriodHolder.info.requestedContentPositionUs != contentPositionUs) { + playingPeriodHolder.info = + playingPeriodHolder.info.copyWithRequestedContentPositionUs(contentPositionUs); + } } else if (!mediaPeriodId.equals(playbackInfo.periodId)) { // Reset previously kept track info if unprepared and the period changes. trackGroupArray = TrackGroupArray.EMPTY; trackSelectorResult = emptyTrackSelectorResult; + staticMetadata = ImmutableList.of(); } + return playbackInfo.copyWithNewPosition( mediaPeriodId, positionUs, contentPositionUs, getTotalBufferedDurationUs(), trackGroupArray, - trackSelectorResult); + trackSelectorResult, + staticMetadata); + } + + private ImmutableList extractMetadataFromTrackSelectionArray( + ExoTrackSelection[] trackSelections) { + ImmutableList.Builder result = new ImmutableList.Builder<>(); + boolean seenNonEmptyMetadata = false; + for (ExoTrackSelection trackSelection : trackSelections) { + if (trackSelection != null) { + Format format = trackSelection.getFormat(/* index= */ 0); + if (format.metadata == null) { + result.add(new Metadata()); + } else { + result.add(format.metadata); + seenNonEmptyMetadata = true; + } + } + } + return seenNonEmptyMetadata ? result.build() : ImmutableList.of(); } private void enableRenderers() throws ExoPlaybackException { @@ -2115,7 +2319,7 @@ private void enableRenderer(int rendererIndex, boolean wasRendererEnabled) TrackSelectorResult trackSelectorResult = periodHolder.getTrackSelectorResult(); RendererConfiguration rendererConfiguration = trackSelectorResult.rendererConfigurations[rendererIndex]; - TrackSelection newSelection = trackSelectorResult.selections.get(rendererIndex); + ExoTrackSelection newSelection = trackSelectorResult.selections[rendererIndex]; Format[] formats = getFormats(newSelection); // The renderer needs enabling with its new track selection. boolean playing = shouldPlayWhenReady() && playbackInfo.playbackState == Player.STATE_READY; @@ -2199,17 +2403,6 @@ private void updateLoadControlTrackSelection( loadControl.onTracksSelected(renderers, trackGroups, trackSelectorResult.selections); } - private void sendPlaybackParametersChangedInternal( - PlaybackParameters playbackParameters, boolean acknowledgeCommand) { - handler - .obtainMessage( - MSG_PLAYBACK_PARAMETERS_CHANGED_INTERNAL, - acknowledgeCommand ? 1 : 0, - 0, - playbackParameters) - .sendToTarget(); - } - private boolean shouldPlayWhenReady() { return playbackInfo.playWhenReady && playbackInfo.playbackSuppressionReason == Player.PLAYBACK_SUPPRESSION_REASON_NONE; @@ -2230,7 +2423,8 @@ private static PositionUpdateForPlaylistChange resolvePositionForPlaylistChange( /* periodPositionUs= */ 0, /* requestedContentPositionUs= */ C.TIME_UNSET, /* forceBufferingState= */ false, - /* endPlayback= */ true); + /* endPlayback= */ true, + /* setTargetLiveOffset= */ false); } MediaPeriodId oldPeriodId = playbackInfo.periodId; Object newPeriodUid = oldPeriodId.periodUid; @@ -2244,6 +2438,7 @@ private static PositionUpdateForPlaylistChange resolvePositionForPlaylistChange( int startAtDefaultPositionWindowIndex = C.INDEX_UNSET; boolean forceBufferingState = false; boolean endPlayback = false; + boolean setTargetLiveOffset = false; if (pendingInitialSeekPosition != null) { // Resolve initial seek position. @Nullable @@ -2268,6 +2463,8 @@ private static PositionUpdateForPlaylistChange resolvePositionForPlaylistChange( } else { newPeriodUid = periodPosition.first; newContentPositionUs = periodPosition.second; + // Use explicit initial seek as new target live offset. + setTargetLiveOffset = true; } forceBufferingState = playbackInfo.playbackState == Player.STATE_ENDED; } @@ -2311,6 +2508,8 @@ private static PositionUpdateForPlaylistChange resolvePositionForPlaylistChange( timeline.getPeriodPosition(window, period, windowIndex, windowPositionUs); newPeriodUid = periodPosition.first; newContentPositionUs = periodPosition.second; + // Use an explicitly requested content position as new target live offset. + setTargetLiveOffset = true; } } @@ -2359,7 +2558,12 @@ private static PositionUpdateForPlaylistChange resolvePositionForPlaylistChange( } return new PositionUpdateForPlaylistChange( - newPeriodId, periodPositionUs, newContentPositionUs, forceBufferingState, endPlayback); + newPeriodId, + periodPositionUs, + newContentPositionUs, + forceBufferingState, + endPlayback, + setTargetLiveOffset); } private static boolean shouldUseRequestedContentPosition( @@ -2589,7 +2793,7 @@ private static Pair resolveSeekPosition( return newPeriodIndex == C.INDEX_UNSET ? null : newTimeline.getUidOfPeriod(newPeriodIndex); } - private static Format[] getFormats(TrackSelection newSelection) { + private static Format[] getFormats(ExoTrackSelection newSelection) { // Build an array of formats contained by the selection. int length = newSelection != null ? newSelection.length() : 0; Format[] formats = new Format[length]; @@ -2622,18 +2826,21 @@ private static final class PositionUpdateForPlaylistChange { public final long requestedContentPositionUs; public final boolean forceBufferingState; public final boolean endPlayback; + public final boolean setTargetLiveOffset; public PositionUpdateForPlaylistChange( MediaPeriodId periodId, long periodPositionUs, long requestedContentPositionUs, boolean forceBufferingState, - boolean endPlayback) { + boolean endPlayback, + boolean setTargetLiveOffset) { this.periodId = periodId; this.periodPositionUs = periodPositionUs; this.requestedContentPositionUs = requestedContentPositionUs; this.forceBufferingState = forceBufferingState; this.endPlayback = endPlayback; + this.setTargetLiveOffset = setTargetLiveOffset; } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/ExoTimeoutException.java b/library/core/src/main/java/com/google/android/exoplayer2/ExoTimeoutException.java new file mode 100644 index 00000000000..c35f6fa95e6 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/ExoTimeoutException.java @@ -0,0 +1,77 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2; + +import androidx.annotation.IntDef; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; + +/** A timeout of an operation on the ExoPlayer playback thread. */ +public final class ExoTimeoutException extends Exception { + + /** + * The operation which produced the timeout error. One of {@link #TIMEOUT_OPERATION_RELEASE}, + * {@link #TIMEOUT_OPERATION_SET_FOREGROUND_MODE}, {@link #TIMEOUT_OPERATION_DETACH_SURFACE} or + * {@link #TIMEOUT_OPERATION_UNDEFINED}. Note that new operations may be added in the future and + * error handling should handle unknown operation values. + */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + TIMEOUT_OPERATION_UNDEFINED, + TIMEOUT_OPERATION_RELEASE, + TIMEOUT_OPERATION_SET_FOREGROUND_MODE, + TIMEOUT_OPERATION_DETACH_SURFACE + }) + public @interface TimeoutOperation {} + + /** The operation where this error occurred is not defined. */ + public static final int TIMEOUT_OPERATION_UNDEFINED = 0; + /** The error occurred in {@link Player#release}. */ + public static final int TIMEOUT_OPERATION_RELEASE = 1; + /** The error occurred in {@link ExoPlayer#setForegroundMode}. */ + public static final int TIMEOUT_OPERATION_SET_FOREGROUND_MODE = 2; + /** The error occurred while detaching a surface from the player. */ + public static final int TIMEOUT_OPERATION_DETACH_SURFACE = 3; + + /** The operation on the ExoPlayer playback thread that timed out. */ + @TimeoutOperation public final int timeoutOperation; + + /** + * Creates the timeout exception. + * + * @param timeoutOperation The {@link TimeoutOperation operation} that produced the timeout. + */ + public ExoTimeoutException(@TimeoutOperation int timeoutOperation) { + super(getErrorMessage(timeoutOperation)); + this.timeoutOperation = timeoutOperation; + } + + private static String getErrorMessage(@TimeoutOperation int timeoutOperation) { + switch (timeoutOperation) { + case TIMEOUT_OPERATION_RELEASE: + return "Player release timed out."; + case TIMEOUT_OPERATION_SET_FOREGROUND_MODE: + return "Setting foreground mode timed out."; + case TIMEOUT_OPERATION_DETACH_SURFACE: + return "Detaching surface timed out."; + case TIMEOUT_OPERATION_UNDEFINED: + default: + return "Undefined timeout."; + } + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/LivePlaybackSpeedControl.java b/library/core/src/main/java/com/google/android/exoplayer2/LivePlaybackSpeedControl.java new file mode 100644 index 00000000000..57f85486ab7 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/LivePlaybackSpeedControl.java @@ -0,0 +1,67 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2; + +import com.google.android.exoplayer2.MediaItem.LiveConfiguration; + +/** + * Controls the playback speed while playing live content in order to maintain a steady target live + * offset. + */ +public interface LivePlaybackSpeedControl { + + /** + * Sets the live configuration defined by the media. + * + * @param liveConfiguration The {@link LiveConfiguration} as defined by the media. + */ + void setLiveConfiguration(LiveConfiguration liveConfiguration); + + /** + * Sets the target live offset in microseconds that overrides the live offset {@link + * #setLiveConfiguration configured} by the media. Passing {@code C.TIME_UNSET} deletes a previous + * override. + * + *

    If no target live offset is configured by {@link #setLiveConfiguration}, this override has + * no effect. + */ + void setTargetLiveOffsetOverrideUs(long liveOffsetUs); + + /** + * Notifies the live playback speed control that a rebuffer occurred. + * + *

    A rebuffer is defined to be caused by buffer depletion rather than a user action. Hence this + * method is not called during initial buffering or when buffering as a result of a seek + * operation. + */ + void notifyRebuffer(); + + /** + * Returns the adjusted playback speed in order get closer towards the {@link + * #getTargetLiveOffsetUs() target live offset}. + * + * @param liveOffsetUs The current live offset, in microseconds. + * @param bufferedDurationUs The duration of media that's currently buffered, in microseconds. + * @return The adjusted factor by which playback should be sped up. + */ + float getAdjustedPlaybackSpeed(long liveOffsetUs, long bufferedDurationUs); + + /** + * Returns the current target live offset, in microseconds, or {@link C#TIME_UNSET} if no target + * live offset is defined for the current media. + */ + long getTargetLiveOffsetUs(); +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/LoadControl.java b/library/core/src/main/java/com/google/android/exoplayer2/LoadControl.java index 94f61bb618c..66fa7a7f17b 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/LoadControl.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/LoadControl.java @@ -17,17 +17,13 @@ import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.TrackGroupArray; -import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; -/** - * Controls buffering of media. - */ +/** Controls buffering of media. */ public interface LoadControl { - /** - * Called by the player when prepared with a new source. - */ + /** Called by the player when prepared with a new source. */ void onPrepared(); /** @@ -37,33 +33,27 @@ public interface LoadControl { * @param trackGroups The {@link TrackGroup}s from which the selection was made. * @param trackSelections The track selections that were made. */ - void onTracksSelected(Renderer[] renderers, TrackGroupArray trackGroups, - TrackSelectionArray trackSelections); + void onTracksSelected( + Renderer[] renderers, TrackGroupArray trackGroups, ExoTrackSelection[] trackSelections); - /** - * Called by the player when stopped. - */ + /** Called by the player when stopped. */ void onStopped(); - /** - * Called by the player when released. - */ + /** Called by the player when released. */ void onReleased(); - /** - * Returns the {@link Allocator} that should be used to obtain media buffer allocations. - */ + /** Returns the {@link Allocator} that should be used to obtain media buffer allocations. */ Allocator getAllocator(); /** * Returns the duration of media to retain in the buffer prior to the current playback position, * for fast backward seeking. - *

    - * Note: If {@link #retainBackBufferFromKeyframe()} is false then seeking in the back-buffer will - * only be fast if the back-buffer contains a keyframe prior to the seek position. - *

    - * Note: Implementations should return a single value. Dynamic changes to the back-buffer are not - * currently supported. + * + *

    Note: If {@link #retainBackBufferFromKeyframe()} is false then seeking in the back-buffer + * will only be fast if the back-buffer contains a keyframe prior to the seek position. + * + *

    Note: Implementations should return a single value. Dynamic changes to the back-buffer are + * not currently supported. * * @return The duration of media to retain in the buffer prior to the current playback position, * in microseconds. @@ -73,17 +63,19 @@ void onTracksSelected(Renderer[] renderers, TrackGroupArray trackGroups, /** * Returns whether media should be retained from the keyframe before the current playback position * minus {@link #getBackBufferDurationUs()}, rather than any sample before or at that position. - *

    - * Warning: Returning true will cause the back-buffer size to depend on the spacing of keyframes - * in the media being played. Returning true is not recommended unless you control the media and - * are comfortable with the back-buffer size exceeding {@link #getBackBufferDurationUs()} by as - * much as the maximum duration between adjacent keyframes in the media. - *

    - * Note: Implementations should return a single value. Dynamic changes to the back-buffer are not - * currently supported. + * + *

    Warning: Returning true will cause the back-buffer size to depend on the spacing of + * keyframes in the media being played. Returning true is not recommended unless you control the + * media and are comfortable with the back-buffer size exceeding {@link + * #getBackBufferDurationUs()} by as much as the maximum duration between adjacent keyframes in + * the media. + * + *

    Note: Implementations should return a single value. Dynamic changes to the back-buffer are + * not currently supported. * * @return Whether media should be retained from the keyframe before the current playback position - * minus {@link #getBackBufferDurationUs()}, rather than any sample before or at that position. + * minus {@link #getBackBufferDurationUs()}, rather than any sample before or at that + * position. */ boolean retainBackBufferFromKeyframe(); @@ -96,7 +88,7 @@ void onTracksSelected(Renderer[] renderers, TrackGroupArray trackGroups, * negative and equal in magnitude to the duration of any media in previous periods still to * be played. * @param bufferedDurationUs The duration of media that's currently buffered. - * @param playbackSpeed The current playback speed. + * @param playbackSpeed The current factor by which playback is sped up. * @return Whether the loading should continue. */ boolean shouldContinueLoading( @@ -109,11 +101,15 @@ boolean shouldContinueLoading( * false} until some condition has been met (e.g. a certain amount of media is buffered). * * @param bufferedDurationUs The duration of media that's currently buffered. - * @param playbackSpeed The current playback speed. + * @param playbackSpeed The current factor by which playback is sped up. * @param rebuffering Whether the player is rebuffering. A rebuffer is defined to be caused by * buffer depletion rather than a user action. Hence this parameter is false during initial * buffering and when buffering as a result of a seek operation. + * @param targetLiveOffsetUs The desired playback position offset to the live edge in + * microseconds, or {@link C#TIME_UNSET} if the media is not a live stream or no offset is + * configured. * @return Whether playback should be allowed to start or resume. */ - boolean shouldStartPlayback(long bufferedDurationUs, float playbackSpeed, boolean rebuffering); + boolean shouldStartPlayback( + long bufferedDurationUs, float playbackSpeed, boolean rebuffering, long targetLiveOffsetUs); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/MediaPeriodHolder.java b/library/core/src/main/java/com/google/android/exoplayer2/MediaPeriodHolder.java index 65f40e9c619..e8639e1f9a3 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/MediaPeriodHolder.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/MediaPeriodHolder.java @@ -24,8 +24,7 @@ import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.SampleStream; import com.google.android.exoplayer2.source.TrackGroupArray; -import com.google.android.exoplayer2.trackselection.TrackSelection; -import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelector; import com.google.android.exoplayer2.trackselection.TrackSelectorResult; import com.google.android.exoplayer2.upstream.Allocator; @@ -173,7 +172,7 @@ public long getNextLoadPositionUs() { /** * Handles period preparation. * - * @param playbackSpeed The current playback speed. + * @param playbackSpeed The current factor by which playback is sped up. * @param timeline The current {@link Timeline}. * @throws ExoPlaybackException If an error occurs during track selection. */ @@ -224,7 +223,7 @@ public void continueLoading(long rendererPositionUs) { *

    The new track selection needs to be applied with {@link * #applyTrackSelection(TrackSelectorResult, long, boolean)} before taking effect. * - * @param playbackSpeed The current playback speed. + * @param playbackSpeed The current factor by which playback is sped up. * @param timeline The current {@link Timeline}. * @return The {@link TrackSelectorResult}. * @throws ExoPlaybackException If an error occurs during track selection. @@ -233,7 +232,7 @@ public TrackSelectorResult selectTracks(float playbackSpeed, Timeline timeline) throws ExoPlaybackException { TrackSelectorResult selectorResult = trackSelector.selectTracks(rendererCapabilities, getTrackGroups(), info.id, timeline); - for (TrackSelection trackSelection : selectorResult.selections.getAll()) { + for (ExoTrackSelection trackSelection : selectorResult.selections) { if (trackSelection != null) { trackSelection.onPlaybackSpeed(playbackSpeed); } @@ -289,10 +288,9 @@ public long applyTrackSelection( trackSelectorResult = newTrackSelectorResult; enableTrackSelectionsInResult(); // Disable streams on the period and get new streams for updated/newly-enabled tracks. - TrackSelectionArray trackSelections = newTrackSelectorResult.selections; positionUs = mediaPeriod.selectTracks( - trackSelections.getAll(), + newTrackSelectorResult.selections, mayRetainStreamFlags, sampleStreams, streamResetFlags, @@ -309,7 +307,7 @@ public long applyTrackSelection( hasEnabledTracks = true; } } else { - Assertions.checkState(trackSelections.get(i) == null); + Assertions.checkState(newTrackSelectorResult.selections[i] == null); } } return positionUs; @@ -361,7 +359,7 @@ private void enableTrackSelectionsInResult() { } for (int i = 0; i < trackSelectorResult.length; i++) { boolean rendererEnabled = trackSelectorResult.isRendererEnabled(i); - TrackSelection trackSelection = trackSelectorResult.selections.get(i); + ExoTrackSelection trackSelection = trackSelectorResult.selections[i]; if (rendererEnabled && trackSelection != null) { trackSelection.enable(); } @@ -374,7 +372,7 @@ private void disableTrackSelectionsInResult() { } for (int i = 0; i < trackSelectorResult.length; i++) { boolean rendererEnabled = trackSelectorResult.isRendererEnabled(i); - TrackSelection trackSelection = trackSelectorResult.selections.get(i); + ExoTrackSelection trackSelection = trackSelectorResult.selections[i]; if (rendererEnabled && trackSelection != null) { trackSelection.disable(); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/MetadataRetriever.java b/library/core/src/main/java/com/google/android/exoplayer2/MetadataRetriever.java index 72f6957865e..4c48cd31412 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/MetadataRetriever.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/MetadataRetriever.java @@ -22,6 +22,10 @@ import android.os.Handler; import android.os.HandlerThread; import android.os.Message; +import androidx.annotation.VisibleForTesting; +import com.google.android.exoplayer2.extractor.DefaultExtractorsFactory; +import com.google.android.exoplayer2.extractor.ExtractorsFactory; +import com.google.android.exoplayer2.extractor.mp4.Mp4Extractor; import com.google.android.exoplayer2.source.DefaultMediaSourceFactory; import com.google.android.exoplayer2.source.MediaPeriod; import com.google.android.exoplayer2.source.MediaSource; @@ -29,7 +33,8 @@ import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.DefaultAllocator; -import com.google.android.exoplayer2.util.Util; +import com.google.android.exoplayer2.util.Clock; +import com.google.android.exoplayer2.util.HandlerWrapper; import com.google.common.util.concurrent.ListenableFuture; import com.google.common.util.concurrent.SettableFuture; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @@ -43,8 +48,9 @@ private MetadataRetriever() {} /** * Retrieves the {@link TrackGroupArray} corresponding to a {@link MediaItem}. * - *

    This is equivalent to using {@code retrieveMetadata(new DefaultMediaSourceFactory(context), - * mediaItem)}. + *

    This is equivalent to using {@link #retrieveMetadata(MediaSourceFactory, MediaItem)} with a + * {@link DefaultMediaSourceFactory} and a {@link DefaultExtractorsFactory} with {@link + * Mp4Extractor#FLAG_READ_MOTION_PHOTO_METADATA} and {@link Mp4Extractor#FLAG_READ_SEF_DATA} set. * * @param context The {@link Context}. * @param mediaItem The {@link MediaItem} whose metadata should be retrieved. @@ -52,7 +58,7 @@ private MetadataRetriever() {} */ public static ListenableFuture retrieveMetadata( Context context, MediaItem mediaItem) { - return retrieveMetadata(new DefaultMediaSourceFactory(context), mediaItem); + return retrieveMetadata(context, mediaItem, Clock.DEFAULT); } /** @@ -67,9 +73,26 @@ public static ListenableFuture retrieveMetadata( */ public static ListenableFuture retrieveMetadata( MediaSourceFactory mediaSourceFactory, MediaItem mediaItem) { + return retrieveMetadata(mediaSourceFactory, mediaItem, Clock.DEFAULT); + } + + @VisibleForTesting + /* package */ static ListenableFuture retrieveMetadata( + Context context, MediaItem mediaItem, Clock clock) { + ExtractorsFactory extractorsFactory = + new DefaultExtractorsFactory() + .setMp4ExtractorFlags( + Mp4Extractor.FLAG_READ_MOTION_PHOTO_METADATA | Mp4Extractor.FLAG_READ_SEF_DATA); + MediaSourceFactory mediaSourceFactory = + new DefaultMediaSourceFactory(context, extractorsFactory); + return retrieveMetadata(mediaSourceFactory, mediaItem, clock); + } + + private static ListenableFuture retrieveMetadata( + MediaSourceFactory mediaSourceFactory, MediaItem mediaItem, Clock clock) { // Recreate thread and handler every time this method is called so that it can be used // concurrently. - return new MetadataRetrieverInternal(mediaSourceFactory).retrieveMetadata(mediaItem); + return new MetadataRetrieverInternal(mediaSourceFactory, clock).retrieveMetadata(mediaItem); } private static final class MetadataRetrieverInternal { @@ -81,15 +104,15 @@ private static final class MetadataRetrieverInternal { private final MediaSourceFactory mediaSourceFactory; private final HandlerThread mediaSourceThread; - private final Handler mediaSourceHandler; + private final HandlerWrapper mediaSourceHandler; private final SettableFuture trackGroupsFuture; - public MetadataRetrieverInternal(MediaSourceFactory mediaSourceFactory) { + public MetadataRetrieverInternal(MediaSourceFactory mediaSourceFactory, Clock clock) { this.mediaSourceFactory = mediaSourceFactory; mediaSourceThread = new HandlerThread("ExoPlayer:MetadataRetriever"); mediaSourceThread.start(); mediaSourceHandler = - Util.createHandler(mediaSourceThread.getLooper(), new MediaSourceHandlerCallback()); + clock.createHandler(mediaSourceThread.getLooper(), new MediaSourceHandlerCallback()); trackGroupsFuture = SettableFuture.create(); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/NoSampleRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/NoSampleRenderer.java index fd5f0431e1a..f3329b8ea77 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/NoSampleRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/NoSampleRenderer.java @@ -168,7 +168,7 @@ public boolean isEnded() { @Override @Capabilities public int supportsFormat(Format format) throws ExoPlaybackException { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } @Override diff --git a/library/core/src/main/java/com/google/android/exoplayer2/PlaybackInfo.java b/library/core/src/main/java/com/google/android/exoplayer2/PlaybackInfo.java index 9fb65630058..96d14d0239e 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/PlaybackInfo.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/PlaybackInfo.java @@ -18,9 +18,12 @@ import androidx.annotation.CheckResult; import androidx.annotation.Nullable; import com.google.android.exoplayer2.Player.PlaybackSuppressionReason; +import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.trackselection.TrackSelectorResult; +import com.google.common.collect.ImmutableList; +import java.util.List; /** * Information about an ongoing playback. @@ -57,6 +60,8 @@ public final TrackGroupArray trackGroups; /** The result of the current track selection. */ public final TrackSelectorResult trackSelectorResult; + /** The current static metadata of the track selections. */ + public final List staticMetadata; /** The {@link MediaPeriodId} of the currently loading media period in the {@link #timeline}. */ public final MediaPeriodId loadingMediaPeriodId; /** Whether playback should proceed when {@link #playbackState} == {@link Player#STATE_READY}. */ @@ -67,6 +72,8 @@ public final PlaybackParameters playbackParameters; /** Whether offload scheduling is enabled for the main player loop. */ public final boolean offloadSchedulingEnabled; + /** Whether the main player loop is sleeping, while using offload scheduling. */ + public final boolean sleepingForOffload; /** * Position up to which media is buffered in {@link #loadingMediaPeriodId) relative to the start @@ -102,6 +109,7 @@ public static PlaybackInfo createDummy(TrackSelectorResult emptyTrackSelectorRes /* isLoading= */ false, TrackGroupArray.EMPTY, emptyTrackSelectorResult, + /* staticMetadata= */ ImmutableList.of(), PLACEHOLDER_MEDIA_PERIOD_ID, /* playWhenReady= */ false, Player.PLAYBACK_SUPPRESSION_REASON_NONE, @@ -109,7 +117,8 @@ public static PlaybackInfo createDummy(TrackSelectorResult emptyTrackSelectorRes /* bufferedPositionUs= */ 0, /* totalBufferedDurationUs= */ 0, /* positionUs= */ 0, - /* offloadSchedulingEnabled= */ false); + /* offloadSchedulingEnabled= */ false, + /* sleepingForOffload= */ false); } /** @@ -123,6 +132,7 @@ public static PlaybackInfo createDummy(TrackSelectorResult emptyTrackSelectorRes * @param isLoading See {@link #isLoading}. * @param trackGroups See {@link #trackGroups}. * @param trackSelectorResult See {@link #trackSelectorResult}. + * @param staticMetadata See {@link #staticMetadata}. * @param loadingMediaPeriodId See {@link #loadingMediaPeriodId}. * @param playWhenReady See {@link #playWhenReady}. * @param playbackSuppressionReason See {@link #playbackSuppressionReason}. @@ -131,6 +141,7 @@ public static PlaybackInfo createDummy(TrackSelectorResult emptyTrackSelectorRes * @param totalBufferedDurationUs See {@link #totalBufferedDurationUs}. * @param positionUs See {@link #positionUs}. * @param offloadSchedulingEnabled See {@link #offloadSchedulingEnabled}. + * @param sleepingForOffload See {@link #sleepingForOffload}. */ public PlaybackInfo( Timeline timeline, @@ -141,6 +152,7 @@ public PlaybackInfo( boolean isLoading, TrackGroupArray trackGroups, TrackSelectorResult trackSelectorResult, + List staticMetadata, MediaPeriodId loadingMediaPeriodId, boolean playWhenReady, @PlaybackSuppressionReason int playbackSuppressionReason, @@ -148,7 +160,8 @@ public PlaybackInfo( long bufferedPositionUs, long totalBufferedDurationUs, long positionUs, - boolean offloadSchedulingEnabled) { + boolean offloadSchedulingEnabled, + boolean sleepingForOffload) { this.timeline = timeline; this.periodId = periodId; this.requestedContentPositionUs = requestedContentPositionUs; @@ -157,6 +170,7 @@ public PlaybackInfo( this.isLoading = isLoading; this.trackGroups = trackGroups; this.trackSelectorResult = trackSelectorResult; + this.staticMetadata = staticMetadata; this.loadingMediaPeriodId = loadingMediaPeriodId; this.playWhenReady = playWhenReady; this.playbackSuppressionReason = playbackSuppressionReason; @@ -165,6 +179,7 @@ public PlaybackInfo( this.totalBufferedDurationUs = totalBufferedDurationUs; this.positionUs = positionUs; this.offloadSchedulingEnabled = offloadSchedulingEnabled; + this.sleepingForOffload = sleepingForOffload; } /** Returns a placeholder period id for an empty timeline. */ @@ -183,6 +198,8 @@ public static MediaPeriodId getDummyPeriodForEmptyTimeline() { * @param trackGroups The track groups for the new position. See {@link #trackGroups}. * @param trackSelectorResult The track selector result for the new position. See {@link * #trackSelectorResult}. + * @param staticMetadata The static metadata for the track selections. See {@link + * #staticMetadata}. * @return Copied playback info with new playing position. */ @CheckResult @@ -192,7 +209,8 @@ public PlaybackInfo copyWithNewPosition( long requestedContentPositionUs, long totalBufferedDurationUs, TrackGroupArray trackGroups, - TrackSelectorResult trackSelectorResult) { + TrackSelectorResult trackSelectorResult, + List staticMetadata) { return new PlaybackInfo( timeline, periodId, @@ -202,6 +220,7 @@ public PlaybackInfo copyWithNewPosition( isLoading, trackGroups, trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -209,7 +228,8 @@ public PlaybackInfo copyWithNewPosition( bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } /** @@ -229,6 +249,7 @@ public PlaybackInfo copyWithTimeline(Timeline timeline) { isLoading, trackGroups, trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -236,7 +257,8 @@ public PlaybackInfo copyWithTimeline(Timeline timeline) { bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } /** @@ -256,6 +278,7 @@ public PlaybackInfo copyWithPlaybackState(int playbackState) { isLoading, trackGroups, trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -263,7 +286,8 @@ public PlaybackInfo copyWithPlaybackState(int playbackState) { bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } /** @@ -283,6 +307,7 @@ public PlaybackInfo copyWithPlaybackError(@Nullable ExoPlaybackException playbac isLoading, trackGroups, trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -290,7 +315,8 @@ public PlaybackInfo copyWithPlaybackError(@Nullable ExoPlaybackException playbac bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } /** @@ -310,6 +336,7 @@ public PlaybackInfo copyWithIsLoading(boolean isLoading) { isLoading, trackGroups, trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -317,7 +344,8 @@ public PlaybackInfo copyWithIsLoading(boolean isLoading) { bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } /** @@ -337,6 +365,7 @@ public PlaybackInfo copyWithLoadingMediaPeriodId(MediaPeriodId loadingMediaPerio isLoading, trackGroups, trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -344,7 +373,8 @@ public PlaybackInfo copyWithLoadingMediaPeriodId(MediaPeriodId loadingMediaPerio bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } /** @@ -368,6 +398,7 @@ public PlaybackInfo copyWithPlayWhenReady( isLoading, trackGroups, trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -375,7 +406,8 @@ public PlaybackInfo copyWithPlayWhenReady( bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } /** @@ -395,6 +427,7 @@ public PlaybackInfo copyWithPlaybackParameters(PlaybackParameters playbackParame isLoading, trackGroups, trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -402,7 +435,8 @@ public PlaybackInfo copyWithPlaybackParameters(PlaybackParameters playbackParame bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } /** @@ -423,6 +457,36 @@ public PlaybackInfo copyWithOffloadSchedulingEnabled(boolean offloadSchedulingEn isLoading, trackGroups, trackSelectorResult, + staticMetadata, + loadingMediaPeriodId, + playWhenReady, + playbackSuppressionReason, + playbackParameters, + bufferedPositionUs, + totalBufferedDurationUs, + positionUs, + offloadSchedulingEnabled, + sleepingForOffload); + } + + /** + * Copies playback info with new sleepingForOffload. + * + * @param sleepingForOffload New main player loop sleeping state. See {@link #sleepingForOffload}. + * @return Copied playback info with new main player loop sleeping state. + */ + @CheckResult + public PlaybackInfo copyWithSleepingForOffload(boolean sleepingForOffload) { + return new PlaybackInfo( + timeline, + periodId, + requestedContentPositionUs, + playbackState, + playbackError, + isLoading, + trackGroups, + trackSelectorResult, + staticMetadata, loadingMediaPeriodId, playWhenReady, playbackSuppressionReason, @@ -430,6 +494,7 @@ public PlaybackInfo copyWithOffloadSchedulingEnabled(boolean offloadSchedulingEn bufferedPositionUs, totalBufferedDurationUs, positionUs, - offloadSchedulingEnabled); + offloadSchedulingEnabled, + sleepingForOffload); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/PlayerMessage.java b/library/core/src/main/java/com/google/android/exoplayer2/PlayerMessage.java index 7e2cb69bc69..36f562f7cb2 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/PlayerMessage.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/PlayerMessage.java @@ -16,8 +16,8 @@ package com.google.android.exoplayer2; import android.os.Handler; +import android.os.Looper; import androidx.annotation.Nullable; -import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Clock; import java.util.concurrent.TimeoutException; @@ -55,11 +55,12 @@ public interface Sender { private final Target target; private final Sender sender; + private final Clock clock; private final Timeline timeline; private int type; @Nullable private Object payload; - private Handler handler; + private Looper looper; private int windowIndex; private long positionMs; private boolean deleteAfterDelivery; @@ -77,7 +78,8 @@ public interface Sender { * set to {@link Timeline#EMPTY}, any position can be specified. * @param defaultWindowIndex The default window index in the {@code timeline} when no other window * index is specified. - * @param defaultHandler The default handler to send the message on when no other handler is + * @param clock The {@link Clock}. + * @param defaultLooper The default {@link Looper} to send the message on when no other looper is * specified. */ public PlayerMessage( @@ -85,11 +87,13 @@ public PlayerMessage( Target target, Timeline timeline, int defaultWindowIndex, - Handler defaultHandler) { + Clock clock, + Looper defaultLooper) { this.sender = sender; this.target = target; this.timeline = timeline; - this.handler = defaultHandler; + this.looper = defaultLooper; + this.clock = clock; this.windowIndex = defaultWindowIndex; this.positionMs = C.TIME_UNSET; this.deleteAfterDelivery = true; @@ -142,22 +146,28 @@ public Object getPayload() { return payload; } + /** @deprecated Use {@link #setLooper(Looper)} instead. */ + @Deprecated + public PlayerMessage setHandler(Handler handler) { + return setLooper(handler.getLooper()); + } + /** - * Sets the handler the message is delivered on. + * Sets the {@link Looper} the message is delivered on. * - * @param handler A {@link Handler}. + * @param looper A {@link Looper}. * @return This message. * @throws IllegalStateException If {@link #send()} has already been called. */ - public PlayerMessage setHandler(Handler handler) { + public PlayerMessage setLooper(Looper looper) { Assertions.checkState(!isSent); - this.handler = handler; + this.looper = looper; return this; } - /** Returns the handler the message is delivered on. */ - public Handler getHandler() { - return handler; + /** Returns the {@link Looper} the message is delivered on. */ + public Looper getLooper() { + return looper; } /** @@ -269,70 +279,64 @@ public synchronized boolean isCanceled() { return isCanceled; } + /** + * Marks the message as processed. Should only be called by a {@link Sender} and may be called + * multiple times. + * + * @param isDelivered Whether the message has been delivered to its target. The message is + * considered as being delivered when this method has been called with {@code isDelivered} set + * to true at least once. + */ + public synchronized void markAsProcessed(boolean isDelivered) { + this.isDelivered |= isDelivered; + isProcessed = true; + notifyAll(); + } + /** * Blocks until after the message has been delivered or the player is no longer able to deliver * the message. * - *

    Note that this method can't be called if the current thread is the same thread used by the - * message handler set with {@link #setHandler(Handler)} as it would cause a deadlock. + *

    Note that this method must not be called if the current thread is the same thread used by + * the message {@link #getLooper() looper} as it would cause a deadlock. * * @return Whether the message was delivered successfully. * @throws IllegalStateException If this method is called before {@link #send()}. * @throws IllegalStateException If this method is called on the same thread used by the message - * handler set with {@link #setHandler(Handler)}. + * {@link #getLooper() looper}. * @throws InterruptedException If the current thread is interrupted while waiting for the message * to be delivered. */ public synchronized boolean blockUntilDelivered() throws InterruptedException { Assertions.checkState(isSent); - Assertions.checkState(handler.getLooper().getThread() != Thread.currentThread()); + Assertions.checkState(looper.getThread() != Thread.currentThread()); while (!isProcessed) { wait(); } return isDelivered; } - /** - * Marks the message as processed. Should only be called by a {@link Sender} and may be called - * multiple times. - * - * @param isDelivered Whether the message has been delivered to its target. The message is - * considered as being delivered when this method has been called with {@code isDelivered} set - * to true at least once. - */ - public synchronized void markAsProcessed(boolean isDelivered) { - this.isDelivered |= isDelivered; - isProcessed = true; - notifyAll(); - } - /** * Blocks until after the message has been delivered or the player is no longer able to deliver - * the message or the specified waiting time elapses. + * the message or the specified timeout elapsed. * - *

    Note that this method can't be called if the current thread is the same thread used by the - * message handler set with {@link #setHandler(Handler)} as it would cause a deadlock. + *

    Note that this method must not be called if the current thread is the same thread used by + * the message {@link #getLooper() looper} as it would cause a deadlock. * - * @param timeoutMs the maximum time to wait in milliseconds. + * @param timeoutMs The timeout in milliseconds. * @return Whether the message was delivered successfully. * @throws IllegalStateException If this method is called before {@link #send()}. * @throws IllegalStateException If this method is called on the same thread used by the message - * handler set with {@link #setHandler(Handler)}. - * @throws TimeoutException If the waiting time elapsed and this message has not been delivered - * and the player is still able to deliver the message. + * {@link #getLooper() looper}. + * @throws TimeoutException If the {@code timeoutMs} elapsed and this message has not been + * delivered and the player is still able to deliver the message. * @throws InterruptedException If the current thread is interrupted while waiting for the message * to be delivered. */ - public synchronized boolean experimentalBlockUntilDelivered(long timeoutMs) - throws InterruptedException, TimeoutException { - return experimentalBlockUntilDelivered(timeoutMs, Clock.DEFAULT); - } - - @VisibleForTesting() - /* package */ synchronized boolean experimentalBlockUntilDelivered(long timeoutMs, Clock clock) + public synchronized boolean blockUntilDelivered(long timeoutMs) throws InterruptedException, TimeoutException { Assertions.checkState(isSent); - Assertions.checkState(handler.getLooper().getThread() != Thread.currentThread()); + Assertions.checkState(looper.getThread() != Thread.currentThread()); long deadlineMs = clock.elapsedRealtime() + timeoutMs; long remainingMs = timeoutMs; @@ -340,11 +344,9 @@ public synchronized boolean experimentalBlockUntilDelivered(long timeoutMs) wait(remainingMs); remainingMs = deadlineMs - clock.elapsedRealtime(); } - if (!isProcessed) { throw new TimeoutException("Message delivery timed out."); } - return isDelivered; } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/Renderer.java b/library/core/src/main/java/com/google/android/exoplayer2/Renderer.java index 10ffcc9f9f3..8578a239298 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/Renderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/Renderer.java @@ -114,7 +114,7 @@ interface WakeupListener { /** * The type of a message that can be passed to a {@link MediaCodec}-based video renderer via * {@link ExoPlayer#createMessage(Target)}. The message payload should be one of the integer - * scaling modes in {@link VideoScalingMode}. + * scaling modes in {@link C.VideoScalingMode}. * *

    Note that the scaling mode only applies if the {@link Surface} targeted by the renderer is * owned by a {@link android.view.SurfaceView}. @@ -160,13 +160,14 @@ interface WakeupListener { */ int MSG_SET_SKIP_SILENCE_ENABLED = 101; /** - * A type of a message that can be passed to an audio renderer via {@link + * The type of a message that can be passed to audio and video renderers via {@link * ExoPlayer#createMessage(Target)}. The message payload should be an {@link Integer} instance - * representing the audio session ID that will be attached to the underlying audio track. + * representing the audio session ID that will be attached to the underlying audio track. Video + * renderers that support tunneling will use the audio session ID when tunneling is enabled. */ int MSG_SET_AUDIO_SESSION_ID = 102; /** - * A type of a message that can be passed to a {@link Renderer} via {@link + * The type of a message that can be passed to a {@link Renderer} via {@link * ExoPlayer#createMessage(Target)}, to inform the renderer that it can schedule waking up another * component. * @@ -180,12 +181,9 @@ interface WakeupListener { @SuppressWarnings("deprecation") int MSG_CUSTOM_BASE = C.MSG_CUSTOM_BASE; - /** - * Video scaling modes for {@link MediaCodec}-based renderers. One of {@link - * #VIDEO_SCALING_MODE_SCALE_TO_FIT} or {@link #VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING}. - */ + /** @deprecated Use {@link C.VideoScalingMode}. */ // VIDEO_SCALING_MODE_DEFAULT is an intentionally duplicated constant. - @SuppressWarnings("UniqueConstants") + @SuppressWarnings({"UniqueConstants", "Deprecation"}) @Documented @Retention(RetentionPolicy.SOURCE) @IntDef( @@ -194,17 +192,16 @@ interface WakeupListener { VIDEO_SCALING_MODE_SCALE_TO_FIT, VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING }) + @Deprecated @interface VideoScalingMode {} - /** See {@link MediaCodec#VIDEO_SCALING_MODE_SCALE_TO_FIT}. */ - @SuppressWarnings("deprecation") - int VIDEO_SCALING_MODE_SCALE_TO_FIT = C.VIDEO_SCALING_MODE_SCALE_TO_FIT; - /** See {@link MediaCodec#VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING}. */ - @SuppressWarnings("deprecation") + /** @deprecated Use {@link C#VIDEO_SCALING_MODE_SCALE_TO_FIT}. */ + @Deprecated int VIDEO_SCALING_MODE_SCALE_TO_FIT = C.VIDEO_SCALING_MODE_SCALE_TO_FIT; + /** @deprecated Use {@link C#VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING}. */ + @Deprecated int VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING = C.VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING; - /** A default video scaling mode for {@link MediaCodec}-based renderers. */ - @SuppressWarnings("deprecation") - int VIDEO_SCALING_MODE_DEFAULT = C.VIDEO_SCALING_MODE_DEFAULT; + /** @deprecated Use {@code C.VIDEO_SCALING_MODE_DEFAULT}. */ + @Deprecated int VIDEO_SCALING_MODE_DEFAULT = C.VIDEO_SCALING_MODE_DEFAULT; /** * The renderer states. One of {@link #STATE_DISABLED}, {@link #STATE_ENABLED} or {@link @@ -215,9 +212,9 @@ interface WakeupListener { @IntDef({STATE_DISABLED, STATE_ENABLED, STATE_STARTED}) @interface State {} /** - * The renderer is disabled. A renderer in this state may hold resources that it requires for - * rendering (e.g. media decoders), for use if it's subsequently enabled. {@link #reset()} can be - * called to force the renderer to release these resources. + * The renderer is disabled. A renderer in this state will not proactively acquire resources that + * it requires for rendering (e.g., media decoders), but may continue to hold any that it already + * has. {@link #reset()} can be called to force the renderer to release such resources. */ int STATE_DISABLED = 0; /** @@ -241,10 +238,9 @@ interface WakeupListener { String getName(); /** - * Returns the track type that the renderer handles. For example, a video renderer will return - * {@link C#TRACK_TYPE_VIDEO}, an audio renderer will return {@link C#TRACK_TYPE_AUDIO}, a text - * renderer will return {@link C#TRACK_TYPE_TEXT}, and so on. + * Returns the track type that the renderer handles. * + * @see Player#getRendererType(int) * @return One of the {@code TRACK_TYPE_*} constants defined in {@link C}. */ int getTrackType(); @@ -403,16 +399,18 @@ void replaceStream(Format[] formats, SampleStream stream, long startPositionUs, void resetPosition(long positionUs) throws ExoPlaybackException; /** - * Sets the operating rate of this renderer, where 1 is the default rate, 2 is twice the default - * rate, 0.5 is half the default rate and so on. The operating rate is a hint to the renderer of - * the speed at which playback will proceed, and may be used for resource planning. + * Indicates the playback speed to this renderer. * *

    The default implementation is a no-op. * - * @param operatingRate The operating rate. - * @throws ExoPlaybackException If an error occurs handling the operating rate. + * @param currentPlaybackSpeed The factor by which playback is currently sped up. + * @param targetPlaybackSpeed The target factor by which playback should be sped up. This may be + * different from {@code currentPlaybackSpeed}, for example, if the speed is temporarily + * adjusted for live playback. + * @throws ExoPlaybackException If an error occurs handling the playback speed. */ - default void setOperatingRate(float operatingRate) throws ExoPlaybackException {} + default void setPlaybackSpeed(float currentPlaybackSpeed, float targetPlaybackSpeed) + throws ExoPlaybackException {} /** * Incrementally renders the {@link SampleStream}. diff --git a/library/core/src/main/java/com/google/android/exoplayer2/RendererCapabilities.java b/library/core/src/main/java/com/google/android/exoplayer2/RendererCapabilities.java index 882c0d11414..657e1174e8a 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/RendererCapabilities.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/RendererCapabilities.java @@ -17,7 +17,6 @@ import android.annotation.SuppressLint; import androidx.annotation.IntDef; -import com.google.android.exoplayer2.util.MimeTypes; import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; @@ -27,11 +26,8 @@ */ public interface RendererCapabilities { - /** - * Level of renderer support for a format. One of {@link #FORMAT_HANDLED}, {@link - * #FORMAT_EXCEEDS_CAPABILITIES}, {@link #FORMAT_UNSUPPORTED_DRM}, {@link - * #FORMAT_UNSUPPORTED_SUBTYPE} or {@link #FORMAT_UNSUPPORTED_TYPE}. - */ + /** @deprecated Use {@link C.FormatSupport} instead. */ + @SuppressWarnings("Deprecation") @Documented @Retention(RetentionPolicy.SOURCE) @IntDef({ @@ -41,52 +37,20 @@ public interface RendererCapabilities { FORMAT_UNSUPPORTED_SUBTYPE, FORMAT_UNSUPPORTED_TYPE }) + @Deprecated @interface FormatSupport {} - - /** A mask to apply to {@link Capabilities} to obtain the {@link FormatSupport} only. */ + /** A mask to apply to {@link Capabilities} to obtain the {@link C.FormatSupport} only. */ int FORMAT_SUPPORT_MASK = 0b111; - /** - * The {@link Renderer} is capable of rendering the format. - */ - int FORMAT_HANDLED = 0b100; - /** - * The {@link Renderer} is capable of rendering formats with the same mime type, but the - * properties of the format exceed the renderer's capabilities. There is a chance the renderer - * will be able to play the format in practice because some renderers report their capabilities - * conservatively, but the expected outcome is that playback will fail. - *

    - * Example: The {@link Renderer} is capable of rendering H264 and the format's mime type is - * {@link MimeTypes#VIDEO_H264}, but the format's resolution exceeds the maximum limit supported - * by the underlying H264 decoder. - */ - int FORMAT_EXCEEDS_CAPABILITIES = 0b011; - /** - * The {@link Renderer} is capable of rendering formats with the same mime type, but is not - * capable of rendering the format because the format's drm protection is not supported. - *

    - * Example: The {@link Renderer} is capable of rendering H264 and the format's mime type is - * {@link MimeTypes#VIDEO_H264}, but the format indicates PlayReady drm protection where-as the - * renderer only supports Widevine. - */ - int FORMAT_UNSUPPORTED_DRM = 0b010; - /** - * The {@link Renderer} is a general purpose renderer for formats of the same top-level type, - * but is not capable of rendering the format or any other format with the same mime type because - * the sub-type is not supported. - *

    - * Example: The {@link Renderer} is a general purpose audio renderer and the format's - * mime type matches audio/[subtype], but there does not exist a suitable decoder for [subtype]. - */ - int FORMAT_UNSUPPORTED_SUBTYPE = 0b001; - /** - * The {@link Renderer} is not capable of rendering the format, either because it does not - * support the format's top-level type, or because it's a specialized renderer for a different - * mime type. - *

    - * Example: The {@link Renderer} is a general purpose video renderer, but the format has an - * audio mime type. - */ - int FORMAT_UNSUPPORTED_TYPE = 0b000; + /** @deprecated Use {@link C#FORMAT_HANDLED} instead. */ + @Deprecated int FORMAT_HANDLED = C.FORMAT_HANDLED; + /** @deprecated Use {@link C#FORMAT_EXCEEDS_CAPABILITIES} instead. */ + @Deprecated int FORMAT_EXCEEDS_CAPABILITIES = C.FORMAT_EXCEEDS_CAPABILITIES; + /** @deprecated Use {@link C#FORMAT_UNSUPPORTED_DRM} instead. */ + @Deprecated int FORMAT_UNSUPPORTED_DRM = C.FORMAT_UNSUPPORTED_DRM; + /** @deprecated Use {@link C#FORMAT_UNSUPPORTED_SUBTYPE} instead. */ + @Deprecated int FORMAT_UNSUPPORTED_SUBTYPE = C.FORMAT_UNSUPPORTED_SUBTYPE; + /** @deprecated Use {@link C#FORMAT_UNSUPPORTED_TYPE} instead. */ + @Deprecated int FORMAT_UNSUPPORTED_TYPE = C.FORMAT_UNSUPPORTED_TYPE; /** * Level of renderer support for adaptive format switches. One of {@link #ADAPTIVE_SEAMLESS}, @@ -136,7 +100,7 @@ public interface RendererCapabilities { /** * Combined renderer capabilities. * - *

    This is a bitwise OR of {@link FormatSupport}, {@link AdaptiveSupport} and {@link + *

    This is a bitwise OR of {@link C.FormatSupport}, {@link AdaptiveSupport} and {@link * TunnelingSupport}. Use {@link #getFormatSupport(int)}, {@link #getAdaptiveSupport(int)} or * {@link #getTunnelingSupport(int)} to obtain the individual flags. And use {@link #create(int)} * or {@link #create(int, int, int)} to create the combined capabilities. @@ -144,18 +108,19 @@ public interface RendererCapabilities { *

    Possible values: * *

      - *
    • {@link FormatSupport}: The level of support for the format itself. One of {@link - * #FORMAT_HANDLED}, {@link #FORMAT_EXCEEDS_CAPABILITIES}, {@link #FORMAT_UNSUPPORTED_DRM}, - * {@link #FORMAT_UNSUPPORTED_SUBTYPE} and {@link #FORMAT_UNSUPPORTED_TYPE}. + *
    • {@link C.FormatSupport}: The level of support for the format itself. One of {@link + * C#FORMAT_HANDLED}, {@link C#FORMAT_EXCEEDS_CAPABILITIES}, {@link + * C#FORMAT_UNSUPPORTED_DRM}, {@link C#FORMAT_UNSUPPORTED_SUBTYPE} and {@link + * C#FORMAT_UNSUPPORTED_TYPE}. *
    • {@link AdaptiveSupport}: The level of support for adapting from the format to another * format of the same mime type. One of {@link #ADAPTIVE_SEAMLESS}, {@link * #ADAPTIVE_NOT_SEAMLESS} and {@link #ADAPTIVE_NOT_SUPPORTED}. Only set if the level of - * support for the format itself is {@link #FORMAT_HANDLED} or {@link - * #FORMAT_EXCEEDS_CAPABILITIES}. + * support for the format itself is {@link C#FORMAT_HANDLED} or {@link + * C#FORMAT_EXCEEDS_CAPABILITIES}. *
    • {@link TunnelingSupport}: The level of support for tunneling. One of {@link * #TUNNELING_SUPPORTED} and {@link #TUNNELING_NOT_SUPPORTED}. Only set if the level of - * support for the format itself is {@link #FORMAT_HANDLED} or {@link - * #FORMAT_EXCEEDS_CAPABILITIES}. + * support for the format itself is {@link C#FORMAT_HANDLED} or {@link + * C#FORMAT_EXCEEDS_CAPABILITIES}. *
    */ @Documented @@ -165,25 +130,25 @@ public interface RendererCapabilities { @interface Capabilities {} /** - * Returns {@link Capabilities} for the given {@link FormatSupport}. + * Returns {@link Capabilities} for the given {@link C.FormatSupport}. * *

    The {@link AdaptiveSupport} is set to {@link #ADAPTIVE_NOT_SUPPORTED} and {{@link * TunnelingSupport} is set to {@link #TUNNELING_NOT_SUPPORTED}. * - * @param formatSupport The {@link FormatSupport}. - * @return The combined {@link Capabilities} of the given {@link FormatSupport}, {@link + * @param formatSupport The {@link C.FormatSupport}. + * @return The combined {@link Capabilities} of the given {@link C.FormatSupport}, {@link * #ADAPTIVE_NOT_SUPPORTED} and {@link #TUNNELING_NOT_SUPPORTED}. */ @Capabilities - static int create(@FormatSupport int formatSupport) { + static int create(@C.FormatSupport int formatSupport) { return create(formatSupport, ADAPTIVE_NOT_SUPPORTED, TUNNELING_NOT_SUPPORTED); } /** - * Returns {@link Capabilities} combining the given {@link FormatSupport}, {@link AdaptiveSupport} - * and {@link TunnelingSupport}. + * Returns {@link Capabilities} combining the given {@link C.FormatSupport}, {@link + * AdaptiveSupport} and {@link TunnelingSupport}. * - * @param formatSupport The {@link FormatSupport}. + * @param formatSupport The {@link C.FormatSupport}. * @param adaptiveSupport The {@link AdaptiveSupport}. * @param tunnelingSupport The {@link TunnelingSupport}. * @return The combined {@link Capabilities}. @@ -192,21 +157,21 @@ static int create(@FormatSupport int formatSupport) { @SuppressLint("WrongConstant") @Capabilities static int create( - @FormatSupport int formatSupport, + @C.FormatSupport int formatSupport, @AdaptiveSupport int adaptiveSupport, @TunnelingSupport int tunnelingSupport) { return formatSupport | adaptiveSupport | tunnelingSupport; } /** - * Returns the {@link FormatSupport} from the combined {@link Capabilities}. + * Returns the {@link C.FormatSupport} from the combined {@link Capabilities}. * * @param supportFlags The combined {@link Capabilities}. - * @return The {@link FormatSupport} only. + * @return The {@link C.FormatSupport} only. */ // Suppression needed for IntDef casting. @SuppressLint("WrongConstant") - @FormatSupport + @C.FormatSupport static int getFormatSupport(@Capabilities int supportFlags) { return supportFlags & FORMAT_SUPPORT_MASK; } @@ -237,29 +202,6 @@ static int getTunnelingSupport(@Capabilities int supportFlags) { return supportFlags & TUNNELING_SUPPORT_MASK; } - /** - * Returns string representation of a {@link FormatSupport} flag. - * - * @param formatSupport A {@link FormatSupport} flag. - * @return A string representation of the flag. - */ - static String getFormatSupportString(@FormatSupport int formatSupport) { - switch (formatSupport) { - case RendererCapabilities.FORMAT_HANDLED: - return "YES"; - case RendererCapabilities.FORMAT_EXCEEDS_CAPABILITIES: - return "NO_EXCEEDS_CAPABILITIES"; - case RendererCapabilities.FORMAT_UNSUPPORTED_DRM: - return "NO_UNSUPPORTED_DRM"; - case RendererCapabilities.FORMAT_UNSUPPORTED_SUBTYPE: - return "NO_UNSUPPORTED_TYPE"; - case RendererCapabilities.FORMAT_UNSUPPORTED_TYPE: - return "NO"; - default: - throw new IllegalStateException(); - } - } - /** Returns the name of the {@link Renderer}. */ String getName(); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/RendererConfiguration.java b/library/core/src/main/java/com/google/android/exoplayer2/RendererConfiguration.java index bc8c6ff6333..e36cb0c71d8 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/RendererConfiguration.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/RendererConfiguration.java @@ -22,24 +22,16 @@ */ public final class RendererConfiguration { - /** - * The default configuration. - */ + /** The default configuration. */ public static final RendererConfiguration DEFAULT = - new RendererConfiguration(C.AUDIO_SESSION_ID_UNSET); + new RendererConfiguration(/* tunneling= */ false); - /** - * The audio session id to use for tunneling, or {@link C#AUDIO_SESSION_ID_UNSET} if tunneling - * should not be enabled. - */ - public final int tunnelingAudioSessionId; + /** Whether to enable tunneling. */ + public final boolean tunneling; - /** - * @param tunnelingAudioSessionId The audio session id to use for tunneling, or - * {@link C#AUDIO_SESSION_ID_UNSET} if tunneling should not be enabled. - */ - public RendererConfiguration(int tunnelingAudioSessionId) { - this.tunnelingAudioSessionId = tunnelingAudioSessionId; + /** @param tunneling Whether to enable tunneling. */ + public RendererConfiguration(boolean tunneling) { + this.tunneling = tunneling; } @Override @@ -51,12 +43,11 @@ public boolean equals(@Nullable Object obj) { return false; } RendererConfiguration other = (RendererConfiguration) obj; - return tunnelingAudioSessionId == other.tunnelingAudioSessionId; + return tunneling == other.tunneling; } @Override public int hashCode() { - return tunnelingAudioSessionId; + return tunneling ? 0 : 1; } - } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/SimpleExoPlayer.java b/library/core/src/main/java/com/google/android/exoplayer2/SimpleExoPlayer.java index 6652cbb03d0..e89e05eb646 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/SimpleExoPlayer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/SimpleExoPlayer.java @@ -18,8 +18,9 @@ import android.content.Context; import android.graphics.Rect; import android.graphics.SurfaceTexture; +import android.media.AudioFormat; +import android.media.AudioTrack; import android.media.MediaCodec; -import android.media.PlaybackParams; import android.os.Handler; import android.os.Looper; import android.view.Surface; @@ -27,7 +28,6 @@ import android.view.SurfaceView; import android.view.TextureView; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.analytics.AnalyticsCollector; import com.google.android.exoplayer2.analytics.AnalyticsListener; @@ -36,6 +36,7 @@ import com.google.android.exoplayer2.audio.AudioRendererEventListener; import com.google.android.exoplayer2.audio.AuxEffectInfo; import com.google.android.exoplayer2.decoder.DecoderCounters; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.device.DeviceInfo; import com.google.android.exoplayer2.device.DeviceListener; import com.google.android.exoplayer2.extractor.DefaultExtractorsFactory; @@ -59,14 +60,17 @@ import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.PriorityTaskManager; import com.google.android.exoplayer2.util.Util; +import com.google.android.exoplayer2.video.VideoDecoderGLSurfaceView; import com.google.android.exoplayer2.video.VideoDecoderOutputBufferRenderer; import com.google.android.exoplayer2.video.VideoFrameMetadataListener; +import com.google.android.exoplayer2.video.VideoListener; import com.google.android.exoplayer2.video.VideoRendererEventListener; import com.google.android.exoplayer2.video.spherical.CameraMotionListener; import java.util.ArrayList; import java.util.Collections; import java.util.List; import java.util.concurrent.CopyOnWriteArraySet; +import java.util.concurrent.TimeoutException; /** * An {@link ExoPlayer} implementation that uses default {@link Renderer} components. Instances can @@ -80,9 +84,8 @@ public class SimpleExoPlayer extends BasePlayer Player.MetadataComponent, Player.DeviceComponent { - /** @deprecated Use {@link com.google.android.exoplayer2.video.VideoListener}. */ - @Deprecated - public interface VideoListener extends com.google.android.exoplayer2.video.VideoListener {} + /** The default timeout for detaching a surface from the player, in milliseconds. */ + public static final long DEFAULT_DETACH_SURFACE_TIMEOUT_MS = 2_000; /** * A builder for {@link SimpleExoPlayer} instances. @@ -107,11 +110,13 @@ public static final class Builder { @C.WakeMode private int wakeMode; private boolean handleAudioBecomingNoisy; private boolean skipSilenceEnabled; - @Renderer.VideoScalingMode private int videoScalingMode; + @C.VideoScalingMode private int videoScalingMode; private boolean useLazyPreparation; private SeekParameters seekParameters; + private LivePlaybackSpeedControl livePlaybackSpeedControl; + private long releaseTimeoutMs; + private long detachSurfaceTimeoutMs; private boolean pauseAtEndOfMediaItems; - private boolean throwWhenStuckBuffering; private boolean buildCalled; /** @@ -131,6 +136,7 @@ public static final class Builder { *

  • {@link MediaSourceFactory}: {@link DefaultMediaSourceFactory} *
  • {@link LoadControl}: {@link DefaultLoadControl} *
  • {@link BandwidthMeter}: {@link DefaultBandwidthMeter#getSingletonInstance(Context)} + *
  • {@link LivePlaybackSpeedControl}: {@link DefaultLivePlaybackSpeedControl} *
  • {@link Looper}: The {@link Looper} associated with the current thread, or the {@link * Looper} of the application's main thread if the current thread doesn't have a {@link * Looper} @@ -140,9 +146,11 @@ public static final class Builder { *
  • {@link C.WakeMode}: {@link C#WAKE_MODE_NONE} *
  • {@code handleAudioBecomingNoisy}: {@code true} *
  • {@code skipSilenceEnabled}: {@code false} - *
  • {@link Renderer.VideoScalingMode}: {@link Renderer#VIDEO_SCALING_MODE_DEFAULT} + *
  • {@link C.VideoScalingMode}: {@link C#VIDEO_SCALING_MODE_DEFAULT} *
  • {@code useLazyPreparation}: {@code true} *
  • {@link SeekParameters}: {@link SeekParameters#DEFAULT} + *
  • {@code releaseTimeoutMs}: {@link ExoPlayer#DEFAULT_RELEASE_TIMEOUT_MS} + *
  • {@code detachSurfaceTimeoutMs}: {@link #DEFAULT_DETACH_SURFACE_TIMEOUT_MS} *
  • {@code pauseAtEndOfMediaItems}: {@code false} *
  • {@link Clock}: {@link Clock#DEFAULT} * @@ -235,11 +243,13 @@ public Builder( looper = Util.getCurrentOrMainLooper(); audioAttributes = AudioAttributes.DEFAULT; wakeMode = C.WAKE_MODE_NONE; - videoScalingMode = Renderer.VIDEO_SCALING_MODE_DEFAULT; + videoScalingMode = C.VIDEO_SCALING_MODE_DEFAULT; useLazyPreparation = true; seekParameters = SeekParameters.DEFAULT; + livePlaybackSpeedControl = new DefaultLivePlaybackSpeedControl.Builder().build(); clock = Clock.DEFAULT; - throwWhenStuckBuffering = true; + releaseTimeoutMs = ExoPlayer.DEFAULT_RELEASE_TIMEOUT_MS; + detachSurfaceTimeoutMs = DEFAULT_DETACH_SURFACE_TIMEOUT_MS; } /** @@ -410,17 +420,17 @@ public Builder setSkipSilenceEnabled(boolean skipSilenceEnabled) { } /** - * Sets the {@link Renderer.VideoScalingMode} that will be used by the player. + * Sets the {@link C.VideoScalingMode} that will be used by the player. * *

    Note that the scaling mode only applies if a {@link MediaCodec}-based video {@link * Renderer} is enabled and if the output surface is owned by a {@link * android.view.SurfaceView}. * - * @param videoScalingMode A {@link Renderer.VideoScalingMode}. + * @param videoScalingMode A {@link C.VideoScalingMode}. * @return This builder. * @throws IllegalStateException If {@link #build()} has already been called. */ - public Builder setVideoScalingMode(@Renderer.VideoScalingMode int videoScalingMode) { + public Builder setVideoScalingMode(@C.VideoScalingMode int videoScalingMode) { Assertions.checkState(!buildCalled); this.videoScalingMode = videoScalingMode; return this; @@ -456,6 +466,40 @@ public Builder setSeekParameters(SeekParameters seekParameters) { return this; } + /** + * Sets a timeout for calls to {@link #release} and {@link #setForegroundMode}. + * + *

    If a call to {@link #release} or {@link #setForegroundMode} takes more than {@code + * timeoutMs} to complete, the player will report an error via {@link + * Player.EventListener#onPlayerError}. + * + * @param releaseTimeoutMs The release timeout, in milliseconds. + * @return This builder. + * @throws IllegalStateException If {@link #build()} has already been called. + */ + public Builder setReleaseTimeoutMs(long releaseTimeoutMs) { + Assertions.checkState(!buildCalled); + this.releaseTimeoutMs = releaseTimeoutMs; + return this; + } + + /** + * Sets a timeout for detaching a surface from the player. + * + *

    If detaching a surface or replacing a surface takes more than {@code + * detachSurfaceTimeoutMs} to complete, the player will report an error via {@link + * Player.EventListener#onPlayerError}. + * + * @param detachSurfaceTimeoutMs The timeout for detaching a surface, in milliseconds. + * @return This builder. + * @throws IllegalStateException If {@link #build()} has already been called. + */ + public Builder setDetachSurfaceTimeoutMs(long detachSurfaceTimeoutMs) { + Assertions.checkState(!buildCalled); + this.detachSurfaceTimeoutMs = detachSurfaceTimeoutMs; + return this; + } + /** * Sets whether to pause playback at the end of each media item. * @@ -475,15 +519,16 @@ public Builder setPauseAtEndOfMediaItems(boolean pauseAtEndOfMediaItems) { } /** - * Sets whether the player should throw when it detects it's stuck buffering. + * Sets the {@link LivePlaybackSpeedControl} that will control the playback speed when playing + * live streams, in order to maintain a steady target offset from the live stream edge. * - *

    This method is experimental, and will be renamed or removed in a future release. - * - * @param throwWhenStuckBuffering Whether to throw when the player detects it's stuck buffering. + * @param livePlaybackSpeedControl The {@link LivePlaybackSpeedControl}. * @return This builder. + * @throws IllegalStateException If {@link #build()} has already been called. */ - public Builder experimentalSetThrowWhenStuckBuffering(boolean throwWhenStuckBuffering) { - this.throwWhenStuckBuffering = throwWhenStuckBuffering; + public Builder setLivePlaybackSpeedControl(LivePlaybackSpeedControl livePlaybackSpeedControl) { + Assertions.checkState(!buildCalled); + this.livePlaybackSpeedControl = livePlaybackSpeedControl; return this; } @@ -521,30 +566,28 @@ public SimpleExoPlayer build() { protected final Renderer[] renderers; + private final Context applicationContext; private final ExoPlayerImpl player; private final ComponentListener componentListener; - private final CopyOnWriteArraySet - videoListeners; + private final CopyOnWriteArraySet videoListeners; private final CopyOnWriteArraySet audioListeners; private final CopyOnWriteArraySet textOutputs; private final CopyOnWriteArraySet metadataOutputs; private final CopyOnWriteArraySet deviceListeners; - private final CopyOnWriteArraySet videoDebugListeners; - private final CopyOnWriteArraySet audioDebugListeners; private final AnalyticsCollector analyticsCollector; private final AudioBecomingNoisyManager audioBecomingNoisyManager; private final AudioFocusManager audioFocusManager; private final StreamVolumeManager streamVolumeManager; private final WakeLockManager wakeLockManager; private final WifiLockManager wifiLockManager; + private final long detachSurfaceTimeoutMs; @Nullable private Format videoFormat; @Nullable private Format audioFormat; - - @Nullable private VideoDecoderOutputBufferRenderer videoDecoderOutputBufferRenderer; + @Nullable private AudioTrack keepSessionIdAudioTrack; @Nullable private Surface surface; private boolean ownsSurface; - @Renderer.VideoScalingMode private int videoScalingMode; + @C.VideoScalingMode private int videoScalingMode; @Nullable private SurfaceHolder surfaceHolder; @Nullable private TextureView textureView; private int surfaceWidth; @@ -592,19 +635,19 @@ protected SimpleExoPlayer( /** @param builder The {@link Builder} to obtain all construction parameters. */ protected SimpleExoPlayer(Builder builder) { + applicationContext = builder.context.getApplicationContext(); analyticsCollector = builder.analyticsCollector; priorityTaskManager = builder.priorityTaskManager; audioAttributes = builder.audioAttributes; videoScalingMode = builder.videoScalingMode; skipSilenceEnabled = builder.skipSilenceEnabled; + detachSurfaceTimeoutMs = builder.detachSurfaceTimeoutMs; componentListener = new ComponentListener(); videoListeners = new CopyOnWriteArraySet<>(); audioListeners = new CopyOnWriteArraySet<>(); textOutputs = new CopyOnWriteArraySet<>(); metadataOutputs = new CopyOnWriteArraySet<>(); deviceListeners = new CopyOnWriteArraySet<>(); - videoDebugListeners = new CopyOnWriteArraySet<>(); - audioDebugListeners = new CopyOnWriteArraySet<>(); Handler eventHandler = new Handler(builder.looper); renderers = builder.renderersFactory.createRenderers( @@ -616,8 +659,13 @@ protected SimpleExoPlayer(Builder builder) { // Set initial values. audioVolume = 1; - audioSessionId = C.AUDIO_SESSION_ID_UNSET; + if (Util.SDK_INT < 21) { + audioSessionId = initializeKeepSessionIdAudioTrack(C.AUDIO_SESSION_ID_UNSET); + } else { + audioSessionId = C.generateAudioSessionIdV21(applicationContext); + } currentCues = Collections.emptyList(); + throwsWhenUsingWrongThread = true; // Build the player and associated objects. player = @@ -630,15 +678,13 @@ protected SimpleExoPlayer(Builder builder) { analyticsCollector, builder.useLazyPreparation, builder.seekParameters, + builder.livePlaybackSpeedControl, + builder.releaseTimeoutMs, builder.pauseAtEndOfMediaItems, builder.clock, - builder.looper); + builder.looper, + /* wrappingPlayer= */ this); player.addListener(componentListener); - videoDebugListeners.add(analyticsCollector); - videoListeners.add(analyticsCollector); - audioDebugListeners.add(analyticsCollector); - audioListeners.add(analyticsCollector); - addMetadataOutput(analyticsCollector); audioBecomingNoisyManager = new AudioBecomingNoisyManager(builder.context, eventHandler, componentListener); @@ -652,10 +698,9 @@ protected SimpleExoPlayer(Builder builder) { wifiLockManager = new WifiLockManager(builder.context); wifiLockManager.setEnabled(builder.wakeMode == C.WAKE_MODE_NETWORK); deviceInfo = createDeviceInfo(streamVolumeManager); - if (!builder.throwWhenStuckBuffering) { - player.experimentalDisableThrowWhenStuckBuffering(); - } + sendRendererMessage(C.TRACK_TYPE_AUDIO, Renderer.MSG_SET_AUDIO_SESSION_ID, audioSessionId); + sendRendererMessage(C.TRACK_TYPE_VIDEO, Renderer.MSG_SET_AUDIO_SESSION_ID, audioSessionId); sendRendererMessage(C.TRACK_TYPE_AUDIO, Renderer.MSG_SET_AUDIO_ATTRIBUTES, audioAttributes); sendRendererMessage(C.TRACK_TYPE_VIDEO, Renderer.MSG_SET_SCALING_MODE, videoScalingMode); sendRendererMessage( @@ -664,9 +709,16 @@ protected SimpleExoPlayer(Builder builder) { @Override public void experimentalSetOffloadSchedulingEnabled(boolean offloadSchedulingEnabled) { + verifyApplicationThread(); player.experimentalSetOffloadSchedulingEnabled(offloadSchedulingEnabled); } + @Override + public boolean experimentalIsSleepingForOffload() { + verifyApplicationThread(); + return player.experimentalIsSleepingForOffload(); + } + @Override @Nullable public AudioComponent getAudioComponent() { @@ -703,17 +755,17 @@ public DeviceComponent getDeviceComponent() { *

    Note that the scaling mode only applies if a {@link MediaCodec}-based video {@link Renderer} * is enabled and if the output surface is owned by a {@link android.view.SurfaceView}. * - * @param videoScalingMode The {@link Renderer.VideoScalingMode}. + * @param videoScalingMode The {@link C.VideoScalingMode}. */ @Override - public void setVideoScalingMode(@Renderer.VideoScalingMode int videoScalingMode) { + public void setVideoScalingMode(@C.VideoScalingMode int videoScalingMode) { verifyApplicationThread(); this.videoScalingMode = videoScalingMode; sendRendererMessage(C.TRACK_TYPE_VIDEO, Renderer.MSG_SET_SCALING_MODE, videoScalingMode); } @Override - @Renderer.VideoScalingMode + @C.VideoScalingMode public int getVideoScalingMode() { return videoScalingMode; } @@ -739,7 +791,7 @@ public void setVideoSurface(@Nullable Surface surface) { verifyApplicationThread(); removeSurfaceCallbacks(); if (surface != null) { - clearVideoDecoderOutputBufferRenderer(); + setVideoDecoderOutputBufferRenderer(/* videoDecoderOutputBufferRenderer= */ null); } setVideoSurfaceInternal(surface, /* ownsSurface= */ false); int newSurfaceSize = surface == null ? 0 : C.LENGTH_UNSET; @@ -751,7 +803,7 @@ public void setVideoSurfaceHolder(@Nullable SurfaceHolder surfaceHolder) { verifyApplicationThread(); removeSurfaceCallbacks(); if (surfaceHolder != null) { - clearVideoDecoderOutputBufferRenderer(); + setVideoDecoderOutputBufferRenderer(/* videoDecoderOutputBufferRenderer= */ null); } this.surfaceHolder = surfaceHolder; if (surfaceHolder == null) { @@ -781,12 +833,29 @@ public void clearVideoSurfaceHolder(@Nullable SurfaceHolder surfaceHolder) { @Override public void setVideoSurfaceView(@Nullable SurfaceView surfaceView) { - setVideoSurfaceHolder(surfaceView == null ? null : surfaceView.getHolder()); + verifyApplicationThread(); + if (surfaceView instanceof VideoDecoderGLSurfaceView) { + VideoDecoderOutputBufferRenderer videoDecoderOutputBufferRenderer = + ((VideoDecoderGLSurfaceView) surfaceView).getVideoDecoderOutputBufferRenderer(); + clearVideoSurface(); + surfaceHolder = surfaceView.getHolder(); + setVideoDecoderOutputBufferRenderer(videoDecoderOutputBufferRenderer); + } else { + setVideoSurfaceHolder(surfaceView == null ? null : surfaceView.getHolder()); + } } @Override public void clearVideoSurfaceView(@Nullable SurfaceView surfaceView) { - clearVideoSurfaceHolder(surfaceView == null ? null : surfaceView.getHolder()); + verifyApplicationThread(); + if (surfaceView instanceof VideoDecoderGLSurfaceView) { + if (surfaceView.getHolder() == surfaceHolder) { + setVideoDecoderOutputBufferRenderer(null); + surfaceHolder = null; + } + } else { + clearVideoSurfaceHolder(surfaceView == null ? null : surfaceView.getHolder()); + } } @Override @@ -794,7 +863,7 @@ public void setVideoTextureView(@Nullable TextureView textureView) { verifyApplicationThread(); removeSurfaceCallbacks(); if (textureView != null) { - clearVideoDecoderOutputBufferRenderer(); + setVideoDecoderOutputBufferRenderer(/* videoDecoderOutputBufferRenderer= */ null); } this.textureView = textureView; if (textureView == null) { @@ -825,32 +894,6 @@ public void clearVideoTextureView(@Nullable TextureView textureView) { } } - @Override - public void setVideoDecoderOutputBufferRenderer( - @Nullable VideoDecoderOutputBufferRenderer videoDecoderOutputBufferRenderer) { - verifyApplicationThread(); - if (videoDecoderOutputBufferRenderer != null) { - clearVideoSurface(); - } - setVideoDecoderOutputBufferRendererInternal(videoDecoderOutputBufferRenderer); - } - - @Override - public void clearVideoDecoderOutputBufferRenderer() { - verifyApplicationThread(); - setVideoDecoderOutputBufferRendererInternal(/* videoDecoderOutputBufferRenderer= */ null); - } - - @Override - public void clearVideoDecoderOutputBufferRenderer( - @Nullable VideoDecoderOutputBufferRenderer videoDecoderOutputBufferRenderer) { - verifyApplicationThread(); - if (videoDecoderOutputBufferRenderer != null - && videoDecoderOutputBufferRenderer == this.videoDecoderOutputBufferRenderer) { - clearVideoDecoderOutputBufferRenderer(); - } - } - @Override public void addAudioListener(AudioListener listener) { // Don't verify application thread. We allow calls to this method from any thread. @@ -864,11 +907,6 @@ public void removeAudioListener(AudioListener listener) { audioListeners.remove(listener); } - @Override - public void setAudioAttributes(AudioAttributes audioAttributes) { - setAudioAttributes(audioAttributes, /* handleAudioFocus= */ false); - } - @Override public void setAudioAttributes(AudioAttributes audioAttributes, boolean handleAudioFocus) { verifyApplicationThread(); @@ -879,6 +917,7 @@ public void setAudioAttributes(AudioAttributes audioAttributes, boolean handleAu this.audioAttributes = audioAttributes; sendRendererMessage(C.TRACK_TYPE_AUDIO, Renderer.MSG_SET_AUDIO_ATTRIBUTES, audioAttributes); streamVolumeManager.setStreamType(Util.getStreamTypeForAudioUsage(audioAttributes.usage)); + analyticsCollector.onAudioAttributesChanged(audioAttributes); for (AudioListener audioListener : audioListeners) { audioListener.onAudioAttributesChanged(audioAttributes); } @@ -903,10 +942,23 @@ public void setAudioSessionId(int audioSessionId) { if (this.audioSessionId == audioSessionId) { return; } + if (audioSessionId == C.AUDIO_SESSION_ID_UNSET) { + if (Util.SDK_INT < 21) { + audioSessionId = initializeKeepSessionIdAudioTrack(C.AUDIO_SESSION_ID_UNSET); + } else { + audioSessionId = C.generateAudioSessionIdV21(applicationContext); + } + } else if (Util.SDK_INT < 21) { + // We need to re-initialize keepSessionIdAudioTrack to make sure the session is kept alive for + // as long as the player is using it. + initializeKeepSessionIdAudioTrack(audioSessionId); + } this.audioSessionId = audioSessionId; sendRendererMessage(C.TRACK_TYPE_AUDIO, Renderer.MSG_SET_AUDIO_SESSION_ID, audioSessionId); - if (audioSessionId != C.AUDIO_SESSION_ID_UNSET) { - notifyAudioSessionIdSet(); + sendRendererMessage(C.TRACK_TYPE_VIDEO, Renderer.MSG_SET_AUDIO_SESSION_ID, audioSessionId); + analyticsCollector.onAudioSessionIdChanged(audioSessionId); + for (AudioListener audioListener : audioListeners) { + audioListener.onAudioSessionIdChanged(audioSessionId); } } @@ -935,6 +987,7 @@ public void setVolume(float audioVolume) { } this.audioVolume = audioVolume; sendVolumeToRenderers(); + analyticsCollector.onVolumeChanged(audioVolume); for (AudioListener audioListener : audioListeners) { audioListener.onVolumeChanged(audioVolume); } @@ -962,37 +1015,6 @@ public void setSkipSilenceEnabled(boolean skipSilenceEnabled) { notifySkipSilenceEnabledChanged(); } - /** - * Sets the stream type for audio playback, used by the underlying audio track. - * - *

    Setting the stream type during playback may introduce a short gap in audio output as the - * audio track is recreated. A new audio session id will also be generated. - * - *

    Calling this method overwrites any attributes set previously by calling {@link - * #setAudioAttributes(AudioAttributes)}. - * - * @deprecated Use {@link #setAudioAttributes(AudioAttributes)}. - * @param streamType The stream type for audio playback. - */ - @Deprecated - public void setAudioStreamType(@C.StreamType int streamType) { - @C.AudioUsage int usage = Util.getAudioUsageForStreamType(streamType); - @C.AudioContentType int contentType = Util.getAudioContentTypeForStreamType(streamType); - AudioAttributes audioAttributes = - new AudioAttributes.Builder().setUsage(usage).setContentType(contentType).build(); - setAudioAttributes(audioAttributes); - } - - /** - * Returns the stream type for audio playback. - * - * @deprecated Use {@link #getAudioAttributes()}. - */ - @Deprecated - public @C.StreamType int getAudioStreamType() { - return Util.getStreamTypeForAudioUsage(audioAttributes.usage); - } - /** Returns the {@link AnalyticsCollector} used for collecting analytics events. */ public AnalyticsCollector getAnalyticsCollector() { return analyticsCollector; @@ -1063,25 +1085,6 @@ public void setPriorityTaskManager(@Nullable PriorityTaskManager priorityTaskMan this.priorityTaskManager = priorityTaskManager; } - /** - * Sets the {@link PlaybackParams} governing audio playback. - * - * @param params The {@link PlaybackParams}, or null to clear any previously set parameters. - * @deprecated Use {@link #setPlaybackParameters(PlaybackParameters)}. - */ - @Deprecated - @RequiresApi(23) - public void setPlaybackParams(@Nullable PlaybackParams params) { - PlaybackParameters playbackParameters; - if (params != null) { - params.allowDefaults(); - playbackParameters = new PlaybackParameters(params.getSpeed(), params.getPitch()); - } else { - playbackParameters = null; - } - setPlaybackParameters(playbackParameters); - } - /** Returns the video format currently being played, or null if no video is being played. */ @Nullable public Format getVideoFormat() { @@ -1107,14 +1110,14 @@ public DecoderCounters getAudioDecoderCounters() { } @Override - public void addVideoListener(com.google.android.exoplayer2.video.VideoListener listener) { + public void addVideoListener(VideoListener listener) { // Don't verify application thread. We allow calls to this method from any thread. Assertions.checkNotNull(listener); videoListeners.add(listener); } @Override - public void removeVideoListener(com.google.android.exoplayer2.video.VideoListener listener) { + public void removeVideoListener(VideoListener listener) { // Don't verify application thread. We allow calls to this method from any thread. videoListeners.remove(listener); } @@ -1155,34 +1158,6 @@ public void clearCameraMotionListener(CameraMotionListener listener) { C.TRACK_TYPE_CAMERA_MOTION, Renderer.MSG_SET_CAMERA_MOTION_LISTENER, /* payload= */ null); } - /** - * Sets a listener to receive video events, removing all existing listeners. - * - * @param listener The listener. - * @deprecated Use {@link #addVideoListener(com.google.android.exoplayer2.video.VideoListener)}. - */ - @Deprecated - @SuppressWarnings("deprecation") - public void setVideoListener(@Nullable VideoListener listener) { - videoListeners.clear(); - if (listener != null) { - addVideoListener(listener); - } - } - - /** - * Equivalent to {@link #removeVideoListener(com.google.android.exoplayer2.video.VideoListener)}. - * - * @param listener The listener to clear. - * @deprecated Use {@link - * #removeVideoListener(com.google.android.exoplayer2.video.VideoListener)}. - */ - @Deprecated - @SuppressWarnings("deprecation") - public void clearVideoListener(VideoListener listener) { - removeVideoListener(listener); - } - @Override public void addTextOutput(TextOutput listener) { // Don't verify application thread. We allow calls to this method from any thread. @@ -1202,31 +1177,6 @@ public List getCurrentCues() { return currentCues; } - /** - * Sets an output to receive text events, removing all existing outputs. - * - * @param output The output. - * @deprecated Use {@link #addTextOutput(TextOutput)}. - */ - @Deprecated - public void setTextOutput(TextOutput output) { - textOutputs.clear(); - if (output != null) { - addTextOutput(output); - } - } - - /** - * Equivalent to {@link #removeTextOutput(TextOutput)}. - * - * @param output The output to clear. - * @deprecated Use {@link #removeTextOutput(TextOutput)}. - */ - @Deprecated - public void clearTextOutput(TextOutput output) { - removeTextOutput(output); - } - @Override public void addMetadataOutput(MetadataOutput listener) { // Don't verify application thread. We allow calls to this method from any thread. @@ -1240,95 +1190,6 @@ public void removeMetadataOutput(MetadataOutput listener) { metadataOutputs.remove(listener); } - /** - * Sets an output to receive metadata events, removing all existing outputs. - * - * @param output The output. - * @deprecated Use {@link #addMetadataOutput(MetadataOutput)}. - */ - @Deprecated - public void setMetadataOutput(MetadataOutput output) { - metadataOutputs.retainAll(Collections.singleton(analyticsCollector)); - if (output != null) { - addMetadataOutput(output); - } - } - - /** - * Equivalent to {@link #removeMetadataOutput(MetadataOutput)}. - * - * @param output The output to clear. - * @deprecated Use {@link #removeMetadataOutput(MetadataOutput)}. - */ - @Deprecated - public void clearMetadataOutput(MetadataOutput output) { - removeMetadataOutput(output); - } - - /** - * @deprecated Use {@link #addAnalyticsListener(AnalyticsListener)} to get more detailed debug - * information. - */ - @Deprecated - @SuppressWarnings("deprecation") - public void setVideoDebugListener(@Nullable VideoRendererEventListener listener) { - videoDebugListeners.retainAll(Collections.singleton(analyticsCollector)); - if (listener != null) { - addVideoDebugListener(listener); - } - } - - /** - * @deprecated Use {@link #addAnalyticsListener(AnalyticsListener)} to get more detailed debug - * information. - */ - @Deprecated - public void addVideoDebugListener(VideoRendererEventListener listener) { - Assertions.checkNotNull(listener); - videoDebugListeners.add(listener); - } - - /** - * @deprecated Use {@link #addAnalyticsListener(AnalyticsListener)} and {@link - * #removeAnalyticsListener(AnalyticsListener)} to get more detailed debug information. - */ - @Deprecated - public void removeVideoDebugListener(VideoRendererEventListener listener) { - videoDebugListeners.remove(listener); - } - - /** - * @deprecated Use {@link #addAnalyticsListener(AnalyticsListener)} to get more detailed debug - * information. - */ - @Deprecated - @SuppressWarnings("deprecation") - public void setAudioDebugListener(@Nullable AudioRendererEventListener listener) { - audioDebugListeners.retainAll(Collections.singleton(analyticsCollector)); - if (listener != null) { - addAudioDebugListener(listener); - } - } - - /** - * @deprecated Use {@link #addAnalyticsListener(AnalyticsListener)} to get more detailed debug - * information. - */ - @Deprecated - public void addAudioDebugListener(AudioRendererEventListener listener) { - Assertions.checkNotNull(listener); - audioDebugListeners.add(listener); - } - - /** - * @deprecated Use {@link #addAnalyticsListener(AnalyticsListener)} and {@link - * #removeAnalyticsListener(AnalyticsListener)} to get more detailed debug information. - */ - @Deprecated - public void removeAudioDebugListener(AudioRendererEventListener listener) { - audioDebugListeners.remove(listener); - } - // ExoPlayer implementation @Override @@ -1341,6 +1202,11 @@ public Looper getApplicationLooper() { return player.getApplicationLooper(); } + @Override + public Clock getClock() { + return player.getClock(); + } + @Override public void addListener(Player.EventListener listener) { // Don't verify application thread. We allow calls to this method from any thread. @@ -1702,12 +1568,17 @@ public void stop(boolean reset) { @Override public void release() { verifyApplicationThread(); + if (Util.SDK_INT < 21 && keepSessionIdAudioTrack != null) { + keepSessionIdAudioTrack.release(); + keepSessionIdAudioTrack = null; + } audioBecomingNoisyManager.setEnabled(false); streamVolumeManager.release(); wakeLockManager.setStayAwake(false); wifiLockManager.setStayAwake(false); audioFocusManager.release(); player.release(); + analyticsCollector.release(); removeSurfaceCallbacks(); if (surface != null) { if (ownsSurface) { @@ -1760,6 +1631,12 @@ public TrackSelectionArray getCurrentTrackSelections() { return player.getCurrentTrackSelections(); } + @Override + public List getCurrentStaticMetadata() { + verifyApplicationThread(); + return player.getCurrentStaticMetadata(); + } + @Override public Timeline getCurrentTimeline() { verifyApplicationThread(); @@ -1948,7 +1825,7 @@ public void setDeviceMuted(boolean muted) { * Sets whether the player should throw an {@link IllegalStateException} when methods are called * from a thread other than the one associated with {@link #getApplicationLooper()}. * - *

    The default is {@code false}, but will change to {@code true} in the future. + *

    The default is {@code true} and this method will be removed in the future. * * @param throwsWhenUsingWrongThread Whether to throw when methods are called from a wrong thread. */ @@ -1991,10 +1868,16 @@ private void setVideoSurfaceInternal(@Nullable Surface surface, boolean ownsSurf // We're replacing a surface. Block to ensure that it's not accessed after the method returns. try { for (PlayerMessage message : messages) { - message.blockUntilDelivered(); + message.blockUntilDelivered(detachSurfaceTimeoutMs); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); + } catch (TimeoutException e) { + // One of the renderers timed out releasing its resources. + player.stop( + /* reset= */ false, + ExoPlaybackException.createForRenderer( + new ExoTimeoutException(ExoTimeoutException.TIMEOUT_OPERATION_DETACH_SURFACE))); } // If we created the previous surface, we are responsible for releasing it. if (this.ownsSurface) { @@ -2005,20 +1888,20 @@ private void setVideoSurfaceInternal(@Nullable Surface surface, boolean ownsSurf this.ownsSurface = ownsSurface; } - private void setVideoDecoderOutputBufferRendererInternal( + private void setVideoDecoderOutputBufferRenderer( @Nullable VideoDecoderOutputBufferRenderer videoDecoderOutputBufferRenderer) { sendRendererMessage( C.TRACK_TYPE_VIDEO, Renderer.MSG_SET_VIDEO_DECODER_OUTPUT_BUFFER_RENDERER, videoDecoderOutputBufferRenderer); - this.videoDecoderOutputBufferRenderer = videoDecoderOutputBufferRenderer; } private void maybeNotifySurfaceSizeChanged(int width, int height) { if (width != surfaceWidth || height != surfaceHeight) { surfaceWidth = width; surfaceHeight = height; - for (com.google.android.exoplayer2.video.VideoListener videoListener : videoListeners) { + analyticsCollector.onSurfaceSizeChanged(width, height); + for (VideoListener videoListener : videoListeners) { videoListener.onSurfaceSizeChanged(width, height); } } @@ -2029,29 +1912,10 @@ private void sendVolumeToRenderers() { sendRendererMessage(C.TRACK_TYPE_AUDIO, Renderer.MSG_SET_VOLUME, scaledVolume); } - private void notifyAudioSessionIdSet() { - for (AudioListener audioListener : audioListeners) { - // Prevent duplicate notification if a listener is both a AudioRendererEventListener and - // a AudioListener, as they have the same method signature. - if (!audioDebugListeners.contains(audioListener)) { - audioListener.onAudioSessionId(audioSessionId); - } - } - for (AudioRendererEventListener audioDebugListener : audioDebugListeners) { - audioDebugListener.onAudioSessionId(audioSessionId); - } - } - @SuppressWarnings("SuspiciousMethodCalls") private void notifySkipSilenceEnabledChanged() { + analyticsCollector.onSkipSilenceEnabledChanged(skipSilenceEnabled); for (AudioListener listener : audioListeners) { - // Prevent duplicate notification if a listener is both a AudioRendererEventListener and - // a AudioListener, as they have the same method signature. - if (!audioDebugListeners.contains(listener)) { - listener.onSkipSilenceEnabledChanged(skipSilenceEnabled); - } - } - for (AudioRendererEventListener listener : audioDebugListeners) { listener.onSkipSilenceEnabledChanged(skipSilenceEnabled); } } @@ -2074,7 +1938,9 @@ private void updateWakeAndWifiLock() { switch (playbackState) { case Player.STATE_READY: case Player.STATE_BUFFERING: - wakeLockManager.setStayAwake(getPlayWhenReady()); + boolean isSleeping = experimentalIsSleepingForOffload(); + wakeLockManager.setStayAwake(getPlayWhenReady() && !isSleeping); + // The wifi lock is not released while sleeping to avoid interrupting downloads. wifiLockManager.setStayAwake(getPlayWhenReady()); break; case Player.STATE_ENDED: @@ -2108,6 +1974,40 @@ private void sendRendererMessage(int trackType, int messageType, @Nullable Objec } } + /** + * Initializes {@link #keepSessionIdAudioTrack} to keep an audio session ID alive. If the audio + * session ID is {@link C#AUDIO_SESSION_ID_UNSET} then a new audio session ID is generated. + * + *

    Use of this method is only required on API level 21 and earlier. + * + * @param audioSessionId The audio session ID, or {@link C#AUDIO_SESSION_ID_UNSET} to generate a + * new one. + * @return The audio session ID. + */ + private int initializeKeepSessionIdAudioTrack(int audioSessionId) { + if (keepSessionIdAudioTrack != null + && keepSessionIdAudioTrack.getAudioSessionId() != audioSessionId) { + keepSessionIdAudioTrack.release(); + keepSessionIdAudioTrack = null; + } + if (keepSessionIdAudioTrack == null) { + int sampleRate = 4000; // Minimum sample rate supported by the platform. + int channelConfig = AudioFormat.CHANNEL_OUT_MONO; + @C.PcmEncoding int encoding = C.ENCODING_PCM_16BIT; + int bufferSize = 2; // Use a two byte buffer, as it is not actually used for playback. + keepSessionIdAudioTrack = + new AudioTrack( + C.STREAM_TYPE_DEFAULT, + sampleRate, + channelConfig, + encoding, + bufferSize, + AudioTrack.MODE_STATIC, + audioSessionId); + } + return keepSessionIdAudioTrack.getAudioSessionId(); + } + private static DeviceInfo createDeviceInfo(StreamVolumeManager streamVolumeManager) { return new DeviceInfo( DeviceInfo.PLAYBACK_TYPE_LOCAL, @@ -2138,78 +2038,64 @@ private final class ComponentListener @Override public void onVideoEnabled(DecoderCounters counters) { videoDecoderCounters = counters; - for (VideoRendererEventListener videoDebugListener : videoDebugListeners) { - videoDebugListener.onVideoEnabled(counters); - } + analyticsCollector.onVideoEnabled(counters); } @Override public void onVideoDecoderInitialized( String decoderName, long initializedTimestampMs, long initializationDurationMs) { - for (VideoRendererEventListener videoDebugListener : videoDebugListeners) { - videoDebugListener.onVideoDecoderInitialized( - decoderName, initializedTimestampMs, initializationDurationMs); - } + analyticsCollector.onVideoDecoderInitialized( + decoderName, initializedTimestampMs, initializationDurationMs); } @Override - public void onVideoInputFormatChanged(Format format) { + public void onVideoInputFormatChanged( + Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { videoFormat = format; - for (VideoRendererEventListener videoDebugListener : videoDebugListeners) { - videoDebugListener.onVideoInputFormatChanged(format); - } + analyticsCollector.onVideoInputFormatChanged(format, decoderReuseEvaluation); } @Override public void onDroppedFrames(int count, long elapsed) { - for (VideoRendererEventListener videoDebugListener : videoDebugListeners) { - videoDebugListener.onDroppedFrames(count, elapsed); - } + analyticsCollector.onDroppedFrames(count, elapsed); } @Override public void onVideoSizeChanged( int width, int height, int unappliedRotationDegrees, float pixelWidthHeightRatio) { - for (com.google.android.exoplayer2.video.VideoListener videoListener : videoListeners) { - // Prevent duplicate notification if a listener is both a VideoRendererEventListener and - // a VideoListener, as they have the same method signature. - if (!videoDebugListeners.contains(videoListener)) { - videoListener.onVideoSizeChanged( - width, height, unappliedRotationDegrees, pixelWidthHeightRatio); - } - } - for (VideoRendererEventListener videoDebugListener : videoDebugListeners) { - videoDebugListener.onVideoSizeChanged( + analyticsCollector.onVideoSizeChanged( + width, height, unappliedRotationDegrees, pixelWidthHeightRatio); + for (VideoListener videoListener : videoListeners) { + videoListener.onVideoSizeChanged( width, height, unappliedRotationDegrees, pixelWidthHeightRatio); } } @Override public void onRenderedFirstFrame(Surface surface) { + analyticsCollector.onRenderedFirstFrame(surface); if (SimpleExoPlayer.this.surface == surface) { - for (com.google.android.exoplayer2.video.VideoListener videoListener : videoListeners) { + for (VideoListener videoListener : videoListeners) { videoListener.onRenderedFirstFrame(); } } - for (VideoRendererEventListener videoDebugListener : videoDebugListeners) { - videoDebugListener.onRenderedFirstFrame(surface); - } + } + + @Override + public void onVideoDecoderReleased(String decoderName) { + analyticsCollector.onVideoDecoderReleased(decoderName); } @Override public void onVideoDisabled(DecoderCounters counters) { - for (VideoRendererEventListener videoDebugListener : videoDebugListeners) { - videoDebugListener.onVideoDisabled(counters); - } + analyticsCollector.onVideoDisabled(counters); videoFormat = null; videoDecoderCounters = null; } @Override public void onVideoFrameProcessingOffset(long totalProcessingOffsetUs, int frameCount) { - for (VideoRendererEventListener videoDebugListener : videoDebugListeners) { - videoDebugListener.onVideoFrameProcessingOffset(totalProcessingOffsetUs, frameCount); - } + analyticsCollector.onVideoFrameProcessingOffset(totalProcessingOffsetUs, frameCount); } // AudioRendererEventListener implementation @@ -2217,56 +2103,41 @@ public void onVideoFrameProcessingOffset(long totalProcessingOffsetUs, int frame @Override public void onAudioEnabled(DecoderCounters counters) { audioDecoderCounters = counters; - for (AudioRendererEventListener audioDebugListener : audioDebugListeners) { - audioDebugListener.onAudioEnabled(counters); - } - } - - @Override - public void onAudioSessionId(int sessionId) { - if (audioSessionId == sessionId) { - return; - } - audioSessionId = sessionId; - notifyAudioSessionIdSet(); + analyticsCollector.onAudioEnabled(counters); } @Override public void onAudioDecoderInitialized( String decoderName, long initializedTimestampMs, long initializationDurationMs) { - for (AudioRendererEventListener audioDebugListener : audioDebugListeners) { - audioDebugListener.onAudioDecoderInitialized( - decoderName, initializedTimestampMs, initializationDurationMs); - } + analyticsCollector.onAudioDecoderInitialized( + decoderName, initializedTimestampMs, initializationDurationMs); } @Override - public void onAudioInputFormatChanged(Format format) { + public void onAudioInputFormatChanged( + Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { audioFormat = format; - for (AudioRendererEventListener audioDebugListener : audioDebugListeners) { - audioDebugListener.onAudioInputFormatChanged(format); - } + analyticsCollector.onAudioInputFormatChanged(format, decoderReuseEvaluation); } @Override public void onAudioPositionAdvancing(long playoutStartSystemTimeMs) { - for (AudioRendererEventListener audioDebugListener : audioDebugListeners) { - audioDebugListener.onAudioPositionAdvancing(playoutStartSystemTimeMs); - } + analyticsCollector.onAudioPositionAdvancing(playoutStartSystemTimeMs); } @Override public void onAudioUnderrun(int bufferSize, long bufferSizeMs, long elapsedSinceLastFeedMs) { - for (AudioRendererEventListener audioDebugListener : audioDebugListeners) { - audioDebugListener.onAudioUnderrun(bufferSize, bufferSizeMs, elapsedSinceLastFeedMs); - } + analyticsCollector.onAudioUnderrun(bufferSize, bufferSizeMs, elapsedSinceLastFeedMs); + } + + @Override + public void onAudioDecoderReleased(String decoderName) { + analyticsCollector.onAudioDecoderReleased(decoderName); } @Override public void onAudioDisabled(DecoderCounters counters) { - for (AudioRendererEventListener audioDebugListener : audioDebugListeners) { - audioDebugListener.onAudioDisabled(counters); - } + analyticsCollector.onAudioDisabled(counters); audioFormat = null; audioDecoderCounters = null; audioSessionId = C.AUDIO_SESSION_ID_UNSET; @@ -2281,6 +2152,11 @@ public void onSkipSilenceEnabledChanged(boolean skipSilenceEnabled) { notifySkipSilenceEnabledChanged(); } + @Override + public void onAudioSinkError(Exception audioSinkError) { + analyticsCollector.onAudioSinkError(audioSinkError); + } + // TextOutput implementation @Override @@ -2295,6 +2171,7 @@ public void onCues(List cues) { @Override public void onMetadata(Metadata metadata) { + analyticsCollector.onMetadata(metadata); for (MetadataOutput metadataOutput : metadataOutputs) { metadataOutput.onMetadata(metadata); } @@ -2412,5 +2289,10 @@ public void onPlayWhenReadyChanged( boolean playWhenReady, @PlayWhenReadyChangeReason int reason) { updateWakeAndWifiLock(); } + + @Override + public void onExperimentalSleepingForOffloadChanged(boolean sleepingForOffload) { + updateWakeAndWifiLock(); + } } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/analytics/AnalyticsCollector.java b/library/core/src/main/java/com/google/android/exoplayer2/analytics/AnalyticsCollector.java index 35f3099dc94..20bb920c573 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/analytics/AnalyticsCollector.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/analytics/AnalyticsCollector.java @@ -17,7 +17,10 @@ import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import android.os.Looper; +import android.util.SparseArray; import android.view.Surface; +import androidx.annotation.CallSuper; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlaybackException; @@ -31,12 +34,11 @@ import com.google.android.exoplayer2.Timeline.Window; import com.google.android.exoplayer2.analytics.AnalyticsListener.EventTime; import com.google.android.exoplayer2.audio.AudioAttributes; -import com.google.android.exoplayer2.audio.AudioListener; import com.google.android.exoplayer2.audio.AudioRendererEventListener; import com.google.android.exoplayer2.decoder.DecoderCounters; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.drm.DrmSessionEventListener; import com.google.android.exoplayer2.metadata.Metadata; -import com.google.android.exoplayer2.metadata.MetadataOutput; import com.google.android.exoplayer2.source.LoadEventInfo; import com.google.android.exoplayer2.source.MediaLoadData; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; @@ -46,7 +48,8 @@ import com.google.android.exoplayer2.upstream.BandwidthMeter; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Clock; -import com.google.android.exoplayer2.video.VideoListener; +import com.google.android.exoplayer2.util.ListenerSet; +import com.google.android.exoplayer2.util.Util; import com.google.android.exoplayer2.video.VideoRendererEventListener; import com.google.common.base.Objects; import com.google.common.collect.ImmutableList; @@ -54,31 +57,27 @@ import com.google.common.collect.Iterables; import java.io.IOException; import java.util.List; -import java.util.concurrent.CopyOnWriteArraySet; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; import org.checkerframework.checker.nullness.qual.RequiresNonNull; /** - * Data collector which is able to forward analytics events to {@link AnalyticsListener}s by - * listening to all available ExoPlayer listeners. + * Data collector that forwards analytics events to {@link AnalyticsListener AnalyticsListeners}. */ public class AnalyticsCollector implements Player.EventListener, - MetadataOutput, AudioRendererEventListener, VideoRendererEventListener, MediaSourceEventListener, BandwidthMeter.EventListener, - DrmSessionEventListener, - VideoListener, - AudioListener { + DrmSessionEventListener { - private final CopyOnWriteArraySet listeners; private final Clock clock; private final Period period; private final Window window; private final MediaPeriodQueueTracker mediaPeriodQueueTracker; + private final SparseArray eventTimes; + private ListenerSet listeners; private @MonotonicNonNull Player player; private boolean isSeeking; @@ -89,10 +88,16 @@ public class AnalyticsCollector */ public AnalyticsCollector(Clock clock) { this.clock = checkNotNull(clock); - listeners = new CopyOnWriteArraySet<>(); + listeners = + new ListenerSet<>( + Util.getCurrentOrMainLooper(), + clock, + AnalyticsListener.Events::new, + (listener, eventFlags) -> {}); period = new Period(); window = new Window(); mediaPeriodQueueTracker = new MediaPeriodQueueTracker(period); + eventTimes = new SparseArray<>(); } /** @@ -100,6 +105,7 @@ public AnalyticsCollector(Clock clock) { * * @param listener The listener to add. */ + @CallSuper public void addListener(AnalyticsListener listener) { Assertions.checkNotNull(listener); listeners.add(listener); @@ -110,6 +116,7 @@ public void addListener(AnalyticsListener listener) { * * @param listener The listener to remove. */ + @CallSuper public void removeListener(AnalyticsListener listener) { listeners.remove(listener); } @@ -119,11 +126,34 @@ public void removeListener(AnalyticsListener listener) { * yet or the current player is idle. * * @param player The {@link Player} for which data will be collected. + * @param looper The {@link Looper} used for listener callbacks. */ - public void setPlayer(Player player) { + @CallSuper + public void setPlayer(Player player, Looper looper) { Assertions.checkState( this.player == null || mediaPeriodQueueTracker.mediaPeriodQueue.isEmpty()); this.player = checkNotNull(player); + listeners = + listeners.copy( + looper, + (listener, events) -> { + events.setEventTimes(eventTimes); + listener.onEvents(player, events); + }); + } + + /** + * Releases the collector. Must be called after the player for which data is collected has been + * released. + */ + @CallSuper + public void release() { + EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); + eventTimes.put(AnalyticsListener.EVENT_PLAYER_RELEASED, eventTime); + // Release listeners lazily so that all events that got triggered as part of player.release() + // are still delivered to all listeners. + listeners.lazyRelease( + AnalyticsListener.EVENT_PLAYER_RELEASED, listener -> listener.onPlayerReleased(eventTime)); } /** @@ -135,7 +165,7 @@ public void setPlayer(Player player) { * @param readingPeriod The media period in the queue that is currently being read by renderers, * or null if the queue is empty. */ - public void updateMediaPeriodQueueInfo( + public final void updateMediaPeriodQueueInfo( List queue, @Nullable MediaPeriodId readingPeriod) { mediaPeriodQueueTracker.onQueueUpdated(queue, readingPeriod, checkNotNull(player)); } @@ -150,9 +180,8 @@ public final void notifySeekStarted() { if (!isSeeking) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); isSeeking = true; - for (AnalyticsListener listener : listeners) { - listener.onSeekStarted(eventTime); - } + sendEvent( + eventTime, /* eventFlag= */ C.INDEX_UNSET, listener -> listener.onSeekStarted(eventTime)); } } @@ -161,14 +190,19 @@ public final void resetForNewPlaylist() { // TODO: remove method. } - // MetadataOutput implementation. + // MetadataOutput events. - @Override + /** + * Called when there is metadata associated with current playback time. + * + * @param metadata The metadata. + */ public final void onMetadata(Metadata metadata) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onMetadata(eventTime, metadata); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_METADATA, + listener -> listener.onMetadata(eventTime, metadata)); } // AudioRendererEventListener implementation. @@ -177,10 +211,13 @@ public final void onMetadata(Metadata metadata) { @Override public final void onAudioEnabled(DecoderCounters counters) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onAudioEnabled(eventTime, counters); - listener.onDecoderEnabled(eventTime, C.TRACK_TYPE_AUDIO, counters); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_ENABLED, + listener -> { + listener.onAudioEnabled(eventTime, counters); + listener.onDecoderEnabled(eventTime, C.TRACK_TYPE_AUDIO, counters); + }); } @SuppressWarnings("deprecation") @@ -188,82 +225,129 @@ public final void onAudioEnabled(DecoderCounters counters) { public final void onAudioDecoderInitialized( String decoderName, long initializedTimestampMs, long initializationDurationMs) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onAudioDecoderInitialized(eventTime, decoderName, initializationDurationMs); - listener.onDecoderInitialized( - eventTime, C.TRACK_TYPE_AUDIO, decoderName, initializationDurationMs); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_DECODER_INITIALIZED, + listener -> { + listener.onAudioDecoderInitialized(eventTime, decoderName, initializationDurationMs); + listener.onDecoderInitialized( + eventTime, C.TRACK_TYPE_AUDIO, decoderName, initializationDurationMs); + }); } @SuppressWarnings("deprecation") @Override - public final void onAudioInputFormatChanged(Format format) { + public final void onAudioInputFormatChanged( + Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onAudioInputFormatChanged(eventTime, format); - listener.onDecoderInputFormatChanged(eventTime, C.TRACK_TYPE_AUDIO, format); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_INPUT_FORMAT_CHANGED, + listener -> { + listener.onAudioInputFormatChanged(eventTime, format, decoderReuseEvaluation); + listener.onDecoderInputFormatChanged(eventTime, C.TRACK_TYPE_AUDIO, format); + }); } @Override public final void onAudioPositionAdvancing(long playoutStartSystemTimeMs) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onAudioPositionAdvancing(eventTime, playoutStartSystemTimeMs); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_POSITION_ADVANCING, + listener -> listener.onAudioPositionAdvancing(eventTime, playoutStartSystemTimeMs)); } @Override public final void onAudioUnderrun( int bufferSize, long bufferSizeMs, long elapsedSinceLastFeedMs) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onAudioUnderrun(eventTime, bufferSize, bufferSizeMs, elapsedSinceLastFeedMs); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_UNDERRUN, + listener -> + listener.onAudioUnderrun(eventTime, bufferSize, bufferSizeMs, elapsedSinceLastFeedMs)); + } + + @Override + public final void onAudioDecoderReleased(String decoderName) { + EventTime eventTime = generateReadingMediaPeriodEventTime(); + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_DECODER_RELEASED, + listener -> listener.onAudioDecoderReleased(eventTime, decoderName)); } @SuppressWarnings("deprecation") @Override public final void onAudioDisabled(DecoderCounters counters) { EventTime eventTime = generatePlayingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onAudioDisabled(eventTime, counters); - listener.onDecoderDisabled(eventTime, C.TRACK_TYPE_AUDIO, counters); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_DISABLED, + listener -> { + listener.onAudioDisabled(eventTime, counters); + listener.onDecoderDisabled(eventTime, C.TRACK_TYPE_AUDIO, counters); + }); } - // AudioListener implementation. - @Override - public final void onAudioSessionId(int audioSessionId) { + public final void onSkipSilenceEnabledChanged(boolean skipSilenceEnabled) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onAudioSessionId(eventTime, audioSessionId); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_SKIP_SILENCE_ENABLED_CHANGED, + listener -> listener.onSkipSilenceEnabledChanged(eventTime, skipSilenceEnabled)); } @Override - public void onAudioAttributesChanged(AudioAttributes audioAttributes) { + public final void onAudioSinkError(Exception audioSinkError) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onAudioAttributesChanged(eventTime, audioAttributes); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_SINK_ERROR, + listener -> listener.onAudioSinkError(eventTime, audioSinkError)); } - @Override - public void onSkipSilenceEnabledChanged(boolean skipSilenceEnabled) { + // Additional audio events. + + /** + * Called when the audio session ID changes. + * + * @param audioSessionId The audio session ID. + */ + public final void onAudioSessionIdChanged(int audioSessionId) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onSkipSilenceEnabledChanged(eventTime, skipSilenceEnabled); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_SESSION_ID, + listener -> listener.onAudioSessionIdChanged(eventTime, audioSessionId)); } - @Override - public void onVolumeChanged(float audioVolume) { + /** + * Called when the audio attributes change. + * + * @param audioAttributes The audio attributes. + */ + public final void onAudioAttributesChanged(AudioAttributes audioAttributes) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onVolumeChanged(eventTime, audioVolume); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_AUDIO_ATTRIBUTES_CHANGED, + listener -> listener.onAudioAttributesChanged(eventTime, audioAttributes)); + } + + /** + * Called when the volume changes. + * + * @param volume The new volume, with 0 being silence and 1 being unity gain. + */ + public final void onVolumeChanged(float volume) { + EventTime eventTime = generateReadingMediaPeriodEventTime(); + sendEvent( + eventTime, + AnalyticsListener.EVENT_VOLUME_CHANGED, + listener -> listener.onVolumeChanged(eventTime, volume)); } // VideoRendererEventListener implementation. @@ -272,10 +356,13 @@ public void onVolumeChanged(float audioVolume) { @Override public final void onVideoEnabled(DecoderCounters counters) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onVideoEnabled(eventTime, counters); - listener.onDecoderEnabled(eventTime, C.TRACK_TYPE_VIDEO, counters); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_VIDEO_ENABLED, + listener -> { + listener.onVideoEnabled(eventTime, counters); + listener.onDecoderEnabled(eventTime, C.TRACK_TYPE_VIDEO, counters); + }); } @SuppressWarnings("deprecation") @@ -283,80 +370,109 @@ public final void onVideoEnabled(DecoderCounters counters) { public final void onVideoDecoderInitialized( String decoderName, long initializedTimestampMs, long initializationDurationMs) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onVideoDecoderInitialized(eventTime, decoderName, initializationDurationMs); - listener.onDecoderInitialized( - eventTime, C.TRACK_TYPE_VIDEO, decoderName, initializationDurationMs); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_VIDEO_DECODER_INITIALIZED, + listener -> { + listener.onVideoDecoderInitialized(eventTime, decoderName, initializationDurationMs); + listener.onDecoderInitialized( + eventTime, C.TRACK_TYPE_VIDEO, decoderName, initializationDurationMs); + }); } @SuppressWarnings("deprecation") @Override - public final void onVideoInputFormatChanged(Format format) { + public final void onVideoInputFormatChanged( + Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onVideoInputFormatChanged(eventTime, format); - listener.onDecoderInputFormatChanged(eventTime, C.TRACK_TYPE_VIDEO, format); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_VIDEO_INPUT_FORMAT_CHANGED, + listener -> { + listener.onVideoInputFormatChanged(eventTime, format, decoderReuseEvaluation); + listener.onDecoderInputFormatChanged(eventTime, C.TRACK_TYPE_VIDEO, format); + }); } @Override public final void onDroppedFrames(int count, long elapsedMs) { EventTime eventTime = generatePlayingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onDroppedVideoFrames(eventTime, count, elapsedMs); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_DROPPED_VIDEO_FRAMES, + listener -> listener.onDroppedVideoFrames(eventTime, count, elapsedMs)); + } + + @Override + public final void onVideoDecoderReleased(String decoderName) { + EventTime eventTime = generateReadingMediaPeriodEventTime(); + sendEvent( + eventTime, + AnalyticsListener.EVENT_VIDEO_DECODER_RELEASED, + listener -> listener.onVideoDecoderReleased(eventTime, decoderName)); } @SuppressWarnings("deprecation") @Override public final void onVideoDisabled(DecoderCounters counters) { EventTime eventTime = generatePlayingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onVideoDisabled(eventTime, counters); - listener.onDecoderDisabled(eventTime, C.TRACK_TYPE_VIDEO, counters); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_VIDEO_DISABLED, + listener -> { + listener.onVideoDisabled(eventTime, counters); + listener.onDecoderDisabled(eventTime, C.TRACK_TYPE_VIDEO, counters); + }); } @Override - public final void onRenderedFirstFrame(@Nullable Surface surface) { + public final void onVideoSizeChanged( + int width, int height, int unappliedRotationDegrees, float pixelWidthHeightRatio) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onRenderedFirstFrame(eventTime, surface); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_VIDEO_SIZE_CHANGED, + listener -> + listener.onVideoSizeChanged( + eventTime, width, height, unappliedRotationDegrees, pixelWidthHeightRatio)); } @Override - public final void onVideoFrameProcessingOffset(long totalProcessingOffsetUs, int frameCount) { - EventTime eventTime = generatePlayingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onVideoFrameProcessingOffset(eventTime, totalProcessingOffsetUs, frameCount); - } + public final void onRenderedFirstFrame(@Nullable Surface surface) { + EventTime eventTime = generateReadingMediaPeriodEventTime(); + sendEvent( + eventTime, + AnalyticsListener.EVENT_RENDERED_FIRST_FRAME, + listener -> listener.onRenderedFirstFrame(eventTime, surface)); } - // VideoListener implementation. - @Override - public final void onRenderedFirstFrame() { - // Do nothing. Already reported in VideoRendererEventListener.onRenderedFirstFrame. + public final void onVideoFrameProcessingOffset(long totalProcessingOffsetUs, int frameCount) { + EventTime eventTime = generatePlayingMediaPeriodEventTime(); + sendEvent( + eventTime, + AnalyticsListener.EVENT_VIDEO_FRAME_PROCESSING_OFFSET, + listener -> + listener.onVideoFrameProcessingOffset(eventTime, totalProcessingOffsetUs, frameCount)); } - @Override - public final void onVideoSizeChanged( - int width, int height, int unappliedRotationDegrees, float pixelWidthHeightRatio) { - EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onVideoSizeChanged( - eventTime, width, height, unappliedRotationDegrees, pixelWidthHeightRatio); - } - } + // Additional video events. - @Override + /** + * Called each time there's a change in the size of the surface onto which the video is being + * rendered. + * + * @param width The surface width in pixels. May be {@link C#LENGTH_UNSET} if unknown, or 0 if the + * video is not rendered onto a surface. + * @param height The surface height in pixels. May be {@link C#LENGTH_UNSET} if unknown, or 0 if + * the video is not rendered onto a surface. + */ public void onSurfaceSizeChanged(int width, int height) { EventTime eventTime = generateReadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onSurfaceSizeChanged(eventTime, width, height); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_SURFACE_SIZE_CHANGED, + listener -> listener.onSurfaceSizeChanged(eventTime, width, height)); } // MediaSourceEventListener implementation. @@ -368,9 +484,10 @@ public final void onLoadStarted( LoadEventInfo loadEventInfo, MediaLoadData mediaLoadData) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onLoadStarted(eventTime, loadEventInfo, mediaLoadData); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_LOAD_STARTED, + listener -> listener.onLoadStarted(eventTime, loadEventInfo, mediaLoadData)); } @Override @@ -380,9 +497,10 @@ public final void onLoadCompleted( LoadEventInfo loadEventInfo, MediaLoadData mediaLoadData) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onLoadCompleted(eventTime, loadEventInfo, mediaLoadData); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_LOAD_COMPLETED, + listener -> listener.onLoadCompleted(eventTime, loadEventInfo, mediaLoadData)); } @Override @@ -392,9 +510,10 @@ public final void onLoadCanceled( LoadEventInfo loadEventInfo, MediaLoadData mediaLoadData) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onLoadCanceled(eventTime, loadEventInfo, mediaLoadData); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_LOAD_CANCELED, + listener -> listener.onLoadCanceled(eventTime, loadEventInfo, mediaLoadData)); } @Override @@ -406,138 +525,164 @@ public final void onLoadError( IOException error, boolean wasCanceled) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onLoadError(eventTime, loadEventInfo, mediaLoadData, error, wasCanceled); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_LOAD_ERROR, + listener -> + listener.onLoadError(eventTime, loadEventInfo, mediaLoadData, error, wasCanceled)); } @Override public final void onUpstreamDiscarded( int windowIndex, @Nullable MediaPeriodId mediaPeriodId, MediaLoadData mediaLoadData) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onUpstreamDiscarded(eventTime, mediaLoadData); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_UPSTREAM_DISCARDED, + listener -> listener.onUpstreamDiscarded(eventTime, mediaLoadData)); } @Override public final void onDownstreamFormatChanged( int windowIndex, @Nullable MediaPeriodId mediaPeriodId, MediaLoadData mediaLoadData) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onDownstreamFormatChanged(eventTime, mediaLoadData); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_DOWNSTREAM_FORMAT_CHANGED, + listener -> listener.onDownstreamFormatChanged(eventTime, mediaLoadData)); } // Player.EventListener implementation. - // TODO: Add onFinishedReportingChanges to Player.EventListener to know when a set of simultaneous - // callbacks finished. This helps to assign exactly the same EventTime to all of them instead of - // having slightly different real times. + // TODO: Use Player.EventListener.onEvents to know when a set of simultaneous callbacks finished. + // This helps to assign exactly the same EventTime to all of them instead of having slightly + // different real times. @Override public final void onTimelineChanged(Timeline timeline, @Player.TimelineChangeReason int reason) { mediaPeriodQueueTracker.onTimelineChanged(checkNotNull(player)); EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onTimelineChanged(eventTime, reason); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_TIMELINE_CHANGED, + listener -> listener.onTimelineChanged(eventTime, reason)); } @Override public final void onMediaItemTransition( @Nullable MediaItem mediaItem, @Player.MediaItemTransitionReason int reason) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onMediaItemTransition(eventTime, mediaItem, reason); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_MEDIA_ITEM_TRANSITION, + listener -> listener.onMediaItemTransition(eventTime, mediaItem, reason)); } @Override public final void onTracksChanged( TrackGroupArray trackGroups, TrackSelectionArray trackSelections) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onTracksChanged(eventTime, trackGroups, trackSelections); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_TRACKS_CHANGED, + listener -> listener.onTracksChanged(eventTime, trackGroups, trackSelections)); + } + + @Override + public final void onStaticMetadataChanged(List metadataList) { + EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); + sendEvent( + eventTime, + AnalyticsListener.EVENT_STATIC_METADATA_CHANGED, + listener -> listener.onStaticMetadataChanged(eventTime, metadataList)); } @Override public final void onIsLoadingChanged(boolean isLoading) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onIsLoadingChanged(eventTime, isLoading); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_IS_LOADING_CHANGED, + listener -> listener.onIsLoadingChanged(eventTime, isLoading)); } @SuppressWarnings("deprecation") @Override public final void onPlayerStateChanged(boolean playWhenReady, @Player.State int playbackState) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onPlayerStateChanged(eventTime, playWhenReady, playbackState); - } + sendEvent( + eventTime, + /* eventFlag= */ C.INDEX_UNSET, + listener -> listener.onPlayerStateChanged(eventTime, playWhenReady, playbackState)); } @Override public final void onPlaybackStateChanged(@Player.State int state) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onPlaybackStateChanged(eventTime, state); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_PLAYBACK_STATE_CHANGED, + listener -> listener.onPlaybackStateChanged(eventTime, state)); } @Override public final void onPlayWhenReadyChanged( boolean playWhenReady, @Player.PlayWhenReadyChangeReason int reason) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onPlayWhenReadyChanged(eventTime, playWhenReady, reason); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_PLAY_WHEN_READY_CHANGED, + listener -> listener.onPlayWhenReadyChanged(eventTime, playWhenReady, reason)); } @Override - public void onPlaybackSuppressionReasonChanged( + public final void onPlaybackSuppressionReasonChanged( @PlaybackSuppressionReason int playbackSuppressionReason) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onPlaybackSuppressionReasonChanged(eventTime, playbackSuppressionReason); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED, + listener -> + listener.onPlaybackSuppressionReasonChanged(eventTime, playbackSuppressionReason)); } @Override public void onIsPlayingChanged(boolean isPlaying) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onIsPlayingChanged(eventTime, isPlaying); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_IS_PLAYING_CHANGED, + listener -> listener.onIsPlayingChanged(eventTime, isPlaying)); } @Override public final void onRepeatModeChanged(@Player.RepeatMode int repeatMode) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onRepeatModeChanged(eventTime, repeatMode); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_REPEAT_MODE_CHANGED, + listener -> listener.onRepeatModeChanged(eventTime, repeatMode)); } @Override public final void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onShuffleModeChanged(eventTime, shuffleModeEnabled); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_SHUFFLE_MODE_ENABLED_CHANGED, + listener -> listener.onShuffleModeChanged(eventTime, shuffleModeEnabled)); } @Override public final void onPlayerError(ExoPlaybackException error) { EventTime eventTime = error.mediaPeriodId != null - ? generateEventTime(error.mediaPeriodId) + ? generateEventTime(new MediaPeriodId(error.mediaPeriodId)) : generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onPlayerError(eventTime, error); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_PLAYER_ERROR, + listener -> listener.onPlayerError(eventTime, error)); } @Override @@ -547,26 +692,27 @@ public final void onPositionDiscontinuity(@Player.DiscontinuityReason int reason } mediaPeriodQueueTracker.onPositionDiscontinuity(checkNotNull(player)); EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onPositionDiscontinuity(eventTime, reason); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_POSITION_DISCONTINUITY, + listener -> listener.onPositionDiscontinuity(eventTime, reason)); } @Override public final void onPlaybackParametersChanged(PlaybackParameters playbackParameters) { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onPlaybackParametersChanged(eventTime, playbackParameters); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_PLAYBACK_PARAMETERS_CHANGED, + listener -> listener.onPlaybackParametersChanged(eventTime, playbackParameters)); } @SuppressWarnings("deprecation") @Override public final void onSeekProcessed() { EventTime eventTime = generateCurrentPlayerMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onSeekProcessed(eventTime); - } + sendEvent( + eventTime, /* eventFlag= */ C.INDEX_UNSET, listener -> listener.onSeekProcessed(eventTime)); } // BandwidthMeter.Listener implementation. @@ -574,9 +720,10 @@ public final void onSeekProcessed() { @Override public final void onBandwidthSample(int elapsedMs, long bytes, long bitrate) { EventTime eventTime = generateLoadingMediaPeriodEventTime(); - for (AnalyticsListener listener : listeners) { - listener.onBandwidthEstimate(eventTime, elapsedMs, bytes, bitrate); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_BANDWIDTH_ESTIMATE, + listener -> listener.onBandwidthEstimate(eventTime, elapsedMs, bytes, bitrate)); } // DefaultDrmSessionManager.EventListener implementation. @@ -584,58 +731,80 @@ public final void onBandwidthSample(int elapsedMs, long bytes, long bitrate) { @Override public final void onDrmSessionAcquired(int windowIndex, @Nullable MediaPeriodId mediaPeriodId) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onDrmSessionAcquired(eventTime); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_DRM_SESSION_ACQUIRED, + listener -> listener.onDrmSessionAcquired(eventTime)); } @Override public final void onDrmKeysLoaded(int windowIndex, @Nullable MediaPeriodId mediaPeriodId) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onDrmKeysLoaded(eventTime); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_DRM_KEYS_LOADED, + listener -> listener.onDrmKeysLoaded(eventTime)); } @Override public final void onDrmSessionManagerError( int windowIndex, @Nullable MediaPeriodId mediaPeriodId, Exception error) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onDrmSessionManagerError(eventTime, error); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_DRM_SESSION_MANAGER_ERROR, + listener -> listener.onDrmSessionManagerError(eventTime, error)); } @Override public final void onDrmKeysRestored(int windowIndex, @Nullable MediaPeriodId mediaPeriodId) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onDrmKeysRestored(eventTime); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_DRM_KEYS_RESTORED, + listener -> listener.onDrmKeysRestored(eventTime)); } @Override public final void onDrmKeysRemoved(int windowIndex, @Nullable MediaPeriodId mediaPeriodId) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onDrmKeysRemoved(eventTime); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_DRM_KEYS_REMOVED, + listener -> listener.onDrmKeysRemoved(eventTime)); } @Override public final void onDrmSessionReleased(int windowIndex, @Nullable MediaPeriodId mediaPeriodId) { EventTime eventTime = generateMediaPeriodEventTime(windowIndex, mediaPeriodId); - for (AnalyticsListener listener : listeners) { - listener.onDrmSessionReleased(eventTime); - } + sendEvent( + eventTime, + AnalyticsListener.EVENT_DRM_SESSION_RELEASED, + listener -> listener.onDrmSessionReleased(eventTime)); } - // Internal methods. + /** + * Sends an event to registered listeners. + * + * @param eventTime The {@link EventTime} to report. + * @param eventFlag An integer flag indicating the type of the event, or {@link C#INDEX_UNSET} to + * report this event without flag. + * @param eventInvocation The event. + */ + protected final void sendEvent( + EventTime eventTime, int eventFlag, ListenerSet.Event eventInvocation) { + eventTimes.put(eventFlag, eventTime); + listeners.sendEvent(eventFlag, eventInvocation); + } + /** Generates an {@link EventTime} for the currently playing item in the player. */ + protected final EventTime generateCurrentPlayerMediaPeriodEventTime() { + return generateEventTime(mediaPeriodQueueTracker.getCurrentPlayerMediaPeriod()); + } /** Returns a new {@link EventTime} for the specified timeline, window and media period id. */ @RequiresNonNull("player") - protected EventTime generateEventTime( + protected final EventTime generateEventTime( Timeline timeline, int windowIndex, @Nullable MediaPeriodId mediaPeriodId) { if (timeline.isEmpty()) { // Ensure media period id is only reported together with a valid timeline. @@ -676,6 +845,8 @@ protected EventTime generateEventTime( player.getTotalBufferedDuration()); } + // Internal methods. + private EventTime generateEventTime(@Nullable MediaPeriodId mediaPeriodId) { checkNotNull(player); @Nullable @@ -694,10 +865,6 @@ private EventTime generateEventTime(@Nullable MediaPeriodId mediaPeriodId) { return generateEventTime(knownTimeline, windowIndex, mediaPeriodId); } - private EventTime generateCurrentPlayerMediaPeriodEventTime() { - return generateEventTime(mediaPeriodQueueTracker.getCurrentPlayerMediaPeriod()); - } - private EventTime generatePlayingMediaPeriodEventTime() { return generateEventTime(mediaPeriodQueueTracker.getPlayingMediaPeriod()); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/analytics/AnalyticsListener.java b/library/core/src/main/java/com/google/android/exoplayer2/analytics/AnalyticsListener.java index 2e26019541e..7692ec47e2d 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/analytics/AnalyticsListener.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/analytics/AnalyticsListener.java @@ -15,7 +15,12 @@ */ package com.google.android.exoplayer2.analytics; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; + +import android.os.Looper; +import android.util.SparseArray; import android.view.Surface; +import androidx.annotation.IntDef; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlaybackException; @@ -28,15 +33,22 @@ import com.google.android.exoplayer2.Player.TimelineChangeReason; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.audio.AudioAttributes; +import com.google.android.exoplayer2.audio.AudioSink; import com.google.android.exoplayer2.decoder.DecoderCounters; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.source.LoadEventInfo; import com.google.android.exoplayer2.source.MediaLoadData; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.util.MutableFlags; import com.google.common.base.Objects; import java.io.IOException; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.util.List; /** * A listener for analytics events. @@ -45,9 +57,262 @@ * time at the time of the event. * *

    All methods have no-op default implementations to allow selective overrides. + * + *

    Listeners can choose to implement individual events (e.g. {@link + * #onIsPlayingChanged(EventTime, boolean)}) or {@link #onEvents(Player, Events)}, which is called + * after one or more events occurred together. */ public interface AnalyticsListener { + /** A set of {@link EventFlags}. */ + final class Events extends MutableFlags { + + private final SparseArray eventTimes; + + /** Creates the set of event flags. */ + public Events() { + eventTimes = new SparseArray<>(/* initialCapacity= */ 0); + } + + /** + * Returns the {@link EventTime} for the specified event. + * + * @param event The {@link EventFlags event}. + * @return The {@link EventTime} of this event. + */ + public EventTime getEventTime(@EventFlags int event) { + return checkNotNull(eventTimes.get(event)); + } + + /** + * Sets the {@link EventTime} values for events recorded in this set. + * + * @param eventTimes A map from {@link EventFlags} to {@link EventTime}. Must at least contain + * all the events recorded in this set. + */ + public void setEventTimes(SparseArray eventTimes) { + this.eventTimes.clear(); + for (int i = 0; i < size(); i++) { + @EventFlags int eventFlag = get(i); + this.eventTimes.append(eventFlag, checkNotNull(eventTimes.get(eventFlag))); + } + } + + /** + * Returns whether the given event occurred. + * + * @param event The {@link EventFlags event}. + * @return Whether the event occurred. + */ + @Override + public boolean contains(@EventFlags int event) { + // Overridden to add IntDef compiler enforcement and new JavaDoc. + return super.contains(event); + } + + /** + * Returns whether any of the given events occurred. + * + * @param events The {@link EventFlags events}. + * @return Whether any of the events occurred. + */ + @Override + public boolean containsAny(@EventFlags int... events) { + // Overridden to add IntDef compiler enforcement and new JavaDoc. + return super.containsAny(events); + } + + /** + * Returns the {@link EventFlags event} at the given index. + * + *

    Although index-based access is possible, it doesn't imply a particular order of these + * events. + * + * @param index The index. Must be between 0 (inclusive) and {@link #size()} (exclusive). + * @return The {@link EventFlags event} at the given index. + */ + @Override + @EventFlags + public int get(int index) { + // Overridden to add IntDef compiler enforcement and new JavaDoc. + return super.get(index); + } + } + + /** + * Events that can be reported via {@link #onEvents(Player, Events)}. + * + *

    One of the {@link AnalyticsListener}{@code .EVENT_*} flags. + */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + EVENT_TIMELINE_CHANGED, + EVENT_MEDIA_ITEM_TRANSITION, + EVENT_TRACKS_CHANGED, + EVENT_STATIC_METADATA_CHANGED, + EVENT_IS_LOADING_CHANGED, + EVENT_PLAYBACK_STATE_CHANGED, + EVENT_PLAY_WHEN_READY_CHANGED, + EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED, + EVENT_IS_PLAYING_CHANGED, + EVENT_REPEAT_MODE_CHANGED, + EVENT_SHUFFLE_MODE_ENABLED_CHANGED, + EVENT_PLAYER_ERROR, + EVENT_POSITION_DISCONTINUITY, + EVENT_PLAYBACK_PARAMETERS_CHANGED, + EVENT_LOAD_STARTED, + EVENT_LOAD_COMPLETED, + EVENT_LOAD_CANCELED, + EVENT_LOAD_ERROR, + EVENT_DOWNSTREAM_FORMAT_CHANGED, + EVENT_UPSTREAM_DISCARDED, + EVENT_BANDWIDTH_ESTIMATE, + EVENT_METADATA, + EVENT_AUDIO_ENABLED, + EVENT_AUDIO_DECODER_INITIALIZED, + EVENT_AUDIO_INPUT_FORMAT_CHANGED, + EVENT_AUDIO_POSITION_ADVANCING, + EVENT_AUDIO_UNDERRUN, + EVENT_AUDIO_DECODER_RELEASED, + EVENT_AUDIO_DISABLED, + EVENT_AUDIO_SESSION_ID, + EVENT_AUDIO_ATTRIBUTES_CHANGED, + EVENT_SKIP_SILENCE_ENABLED_CHANGED, + EVENT_AUDIO_SINK_ERROR, + EVENT_VOLUME_CHANGED, + EVENT_VIDEO_ENABLED, + EVENT_VIDEO_DECODER_INITIALIZED, + EVENT_VIDEO_INPUT_FORMAT_CHANGED, + EVENT_DROPPED_VIDEO_FRAMES, + EVENT_VIDEO_DECODER_RELEASED, + EVENT_VIDEO_DISABLED, + EVENT_VIDEO_FRAME_PROCESSING_OFFSET, + EVENT_RENDERED_FIRST_FRAME, + EVENT_VIDEO_SIZE_CHANGED, + EVENT_SURFACE_SIZE_CHANGED, + EVENT_DRM_SESSION_ACQUIRED, + EVENT_DRM_KEYS_LOADED, + EVENT_DRM_SESSION_MANAGER_ERROR, + EVENT_DRM_KEYS_RESTORED, + EVENT_DRM_KEYS_REMOVED, + EVENT_DRM_SESSION_RELEASED, + EVENT_PLAYER_RELEASED, + }) + @interface EventFlags {} + /** {@link Player#getCurrentTimeline()} changed. */ + int EVENT_TIMELINE_CHANGED = Player.EVENT_TIMELINE_CHANGED; + /** + * {@link Player#getCurrentMediaItem()} changed or the player started repeating the current item. + */ + int EVENT_MEDIA_ITEM_TRANSITION = Player.EVENT_MEDIA_ITEM_TRANSITION; + /** + * {@link Player#getCurrentTrackGroups()} or {@link Player#getCurrentTrackSelections()} changed. + */ + int EVENT_TRACKS_CHANGED = Player.EVENT_TRACKS_CHANGED; + /** {@link Player#getCurrentStaticMetadata()} changed. */ + int EVENT_STATIC_METADATA_CHANGED = Player.EVENT_STATIC_METADATA_CHANGED; + /** {@link Player#isLoading()} ()} changed. */ + int EVENT_IS_LOADING_CHANGED = Player.EVENT_IS_LOADING_CHANGED; + /** {@link Player#getPlaybackState()} changed. */ + int EVENT_PLAYBACK_STATE_CHANGED = Player.EVENT_PLAYBACK_STATE_CHANGED; + /** {@link Player#getPlayWhenReady()} changed. */ + int EVENT_PLAY_WHEN_READY_CHANGED = Player.EVENT_PLAY_WHEN_READY_CHANGED; + /** {@link Player#getPlaybackSuppressionReason()} changed. */ + int EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED = Player.EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED; + /** {@link Player#isPlaying()} changed. */ + int EVENT_IS_PLAYING_CHANGED = Player.EVENT_IS_PLAYING_CHANGED; + /** {@link Player#getRepeatMode()} changed. */ + int EVENT_REPEAT_MODE_CHANGED = Player.EVENT_REPEAT_MODE_CHANGED; + /** {@link Player#getShuffleModeEnabled()} changed. */ + int EVENT_SHUFFLE_MODE_ENABLED_CHANGED = Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED; + /** {@link Player#getPlayerError()} changed. */ + int EVENT_PLAYER_ERROR = Player.EVENT_PLAYER_ERROR; + /** + * A position discontinuity occurred. See {@link + * Player.EventListener#onPositionDiscontinuity(int)}. + */ + int EVENT_POSITION_DISCONTINUITY = Player.EVENT_POSITION_DISCONTINUITY; + /** {@link Player#getPlaybackParameters()} changed. */ + int EVENT_PLAYBACK_PARAMETERS_CHANGED = Player.EVENT_PLAYBACK_PARAMETERS_CHANGED; + /** A source started loading data. */ + int EVENT_LOAD_STARTED = 1000; // Intentional gap to leave space for new Player events + /** A source started completed loading data. */ + int EVENT_LOAD_COMPLETED = 1001; + /** A source canceled loading data. */ + int EVENT_LOAD_CANCELED = 1002; + /** A source had a non-fatal error loading data. */ + int EVENT_LOAD_ERROR = 1003; + /** The downstream format sent to renderers changed. */ + int EVENT_DOWNSTREAM_FORMAT_CHANGED = 1004; + /** Data was removed from the end of the media buffer. */ + int EVENT_UPSTREAM_DISCARDED = 1005; + /** The bandwidth estimate has been updated. */ + int EVENT_BANDWIDTH_ESTIMATE = 1006; + /** Metadata associated with the current playback time was reported. */ + int EVENT_METADATA = 1007; + /** An audio renderer was enabled. */ + int EVENT_AUDIO_ENABLED = 1008; + /** An audio renderer created a decoder. */ + int EVENT_AUDIO_DECODER_INITIALIZED = 1009; + /** The format consumed by an audio renderer changed. */ + int EVENT_AUDIO_INPUT_FORMAT_CHANGED = 1010; + /** The audio position has increased for the first time since the last pause or position reset. */ + int EVENT_AUDIO_POSITION_ADVANCING = 1011; + /** An audio underrun occurred. */ + int EVENT_AUDIO_UNDERRUN = 1012; + /** An audio renderer released a decoder. */ + int EVENT_AUDIO_DECODER_RELEASED = 1013; + /** An audio renderer was disabled. */ + int EVENT_AUDIO_DISABLED = 1014; + /** An audio session id was set. */ + int EVENT_AUDIO_SESSION_ID = 1015; + /** Audio attributes changed. */ + int EVENT_AUDIO_ATTRIBUTES_CHANGED = 1016; + /** Skipping silences was enabled or disabled in the audio stream. */ + int EVENT_SKIP_SILENCE_ENABLED_CHANGED = 1017; + /** The audio sink encountered a non-fatal error. */ + int EVENT_AUDIO_SINK_ERROR = 1018; + /** The volume changed. */ + int EVENT_VOLUME_CHANGED = 1019; + /** A video renderer was enabled. */ + int EVENT_VIDEO_ENABLED = 1020; + /** A video renderer created a decoder. */ + int EVENT_VIDEO_DECODER_INITIALIZED = 1021; + /** The format consumed by a video renderer changed. */ + int EVENT_VIDEO_INPUT_FORMAT_CHANGED = 1022; + /** Video frames have been dropped. */ + int EVENT_DROPPED_VIDEO_FRAMES = 1023; + /** A video renderer released a decoder. */ + int EVENT_VIDEO_DECODER_RELEASED = 1024; + /** A video renderer was disabled. */ + int EVENT_VIDEO_DISABLED = 1025; + /** Video frame processing offset data has been reported. */ + int EVENT_VIDEO_FRAME_PROCESSING_OFFSET = 1026; + /** + * The first frame has been rendered since setting the surface, since the renderer was reset or + * since the stream changed. + */ + int EVENT_RENDERED_FIRST_FRAME = 1027; + /** The video size changed. */ + int EVENT_VIDEO_SIZE_CHANGED = 1028; + /** The surface size changed. */ + int EVENT_SURFACE_SIZE_CHANGED = 1029; + /** A DRM session has been acquired. */ + int EVENT_DRM_SESSION_ACQUIRED = 1030; + /** DRM keys were loaded. */ + int EVENT_DRM_KEYS_LOADED = 1031; + /** A non-fatal DRM session manager error occurred. */ + int EVENT_DRM_SESSION_MANAGER_ERROR = 1032; + /** DRM keys were restored. */ + int EVENT_DRM_KEYS_RESTORED = 1033; + /** DRM keys were removed. */ + int EVENT_DRM_KEYS_REMOVED = 1034; + /** A DRM session has been released. */ + int EVENT_DRM_SESSION_RELEASED = 1035; + /** The player was released. */ + int EVENT_PLAYER_RELEASED = 1036; + /** Time information of an event. */ final class EventTime { @@ -336,6 +601,23 @@ default void onPlayerError(EventTime eventTime, ExoPlaybackException error) {} default void onTracksChanged( EventTime eventTime, TrackGroupArray trackGroups, TrackSelectionArray trackSelections) {} + /** + * Called when the static metadata changes. + * + *

    The provided {@code metadataList} is an immutable list of {@link Metadata} instances, where + * the elements correspond to the current track selections (as returned by {@link + * #onTracksChanged(EventTime, TrackGroupArray, TrackSelectionArray)}, or an empty list if there + * are no track selections or the selected tracks contain no static metadata. + * + *

    The metadata is considered static in the sense that it comes from the tracks' declared + * Formats, rather than being timed (or dynamic) metadata, which is represented within a metadata + * track. + * + * @param eventTime The event time. + * @param metadataList The static metadata. + */ + default void onStaticMetadataChanged(EventTime eventTime, List metadataList) {} + /** * Called when a media source started loading data. * @@ -433,8 +715,8 @@ default void onDecoderInitialized( EventTime eventTime, int trackType, String decoderName, long initializationDurationMs) {} /** - * @deprecated Use {@link #onAudioInputFormatChanged} and {@link #onVideoInputFormatChanged} - * instead. + * @deprecated Use {@link #onAudioInputFormatChanged(EventTime, Format, DecoderReuseEvaluation)} + * and {@link #onVideoInputFormatChanged(EventTime, Format, DecoderReuseEvaluation)}. instead. */ @Deprecated default void onDecoderInputFormatChanged(EventTime eventTime, int trackType, Format format) {} @@ -463,13 +745,26 @@ default void onAudioEnabled(EventTime eventTime, DecoderCounters counters) {} default void onAudioDecoderInitialized( EventTime eventTime, String decoderName, long initializationDurationMs) {} + /** + * @deprecated Use {@link #onAudioInputFormatChanged(EventTime, Format, DecoderReuseEvaluation)}. + */ + @Deprecated + default void onAudioInputFormatChanged(EventTime eventTime, Format format) {} + /** * Called when the format of the media being consumed by an audio renderer changes. * * @param eventTime The event time. * @param format The new format. + * @param decoderReuseEvaluation The result of the evaluation to determine whether an existing + * decoder instance can be reused for the new format, or {@code null} if the renderer did not + * have a decoder. */ - default void onAudioInputFormatChanged(EventTime eventTime, Format format) {} + @SuppressWarnings("deprecation") + default void onAudioInputFormatChanged( + EventTime eventTime, Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { + onAudioInputFormatChanged(eventTime, format); + } /** * Called when the audio position has increased for the first time since the last pause or @@ -493,6 +788,14 @@ default void onAudioPositionAdvancing(EventTime eventTime, long playoutStartSyst default void onAudioUnderrun( EventTime eventTime, int bufferSize, long bufferSizeMs, long elapsedSinceLastFeedMs) {} + /** + * Called when an audio renderer releases a decoder. + * + * @param eventTime The event time. + * @param decoderName The decoder that was released. + */ + default void onAudioDecoderReleased(EventTime eventTime, String decoderName) {} + /** * Called when an audio renderer is disabled. * @@ -502,12 +805,12 @@ default void onAudioUnderrun( default void onAudioDisabled(EventTime eventTime, DecoderCounters counters) {} /** - * Called when the audio session id is set. + * Called when the audio session ID changes. * * @param eventTime The event time. - * @param audioSessionId The audio session id. + * @param audioSessionId The audio session ID. */ - default void onAudioSessionId(EventTime eventTime, int audioSessionId) {} + default void onAudioSessionIdChanged(EventTime eventTime, int audioSessionId) {} /** * Called when the audio attributes change. @@ -525,6 +828,16 @@ default void onAudioAttributesChanged(EventTime eventTime, AudioAttributes audio */ default void onSkipSilenceEnabledChanged(EventTime eventTime, boolean skipSilenceEnabled) {} + /** + * Called when {@link AudioSink} has encountered an error. These errors are just for informational + * purposes and the player may recover. + * + * @param eventTime The event time. + * @param audioSinkError Either a {@link AudioSink.InitializationException} or a {@link + * AudioSink.WriteException} describing the error. + */ + default void onAudioSinkError(EventTime eventTime, Exception audioSinkError) {} + /** * Called when the volume changes. * @@ -552,13 +865,26 @@ default void onVideoEnabled(EventTime eventTime, DecoderCounters counters) {} default void onVideoDecoderInitialized( EventTime eventTime, String decoderName, long initializationDurationMs) {} + /** + * @deprecated Use {@link #onVideoInputFormatChanged(EventTime, Format, DecoderReuseEvaluation)}. + */ + @Deprecated + default void onVideoInputFormatChanged(EventTime eventTime, Format format) {} + /** * Called when the format of the media being consumed by a video renderer changes. * * @param eventTime The event time. * @param format The new format. + * @param decoderReuseEvaluation The result of the evaluation to determine whether an existing + * decoder instance can be reused for the new format, or {@code null} if the renderer did not + * have a decoder. */ - default void onVideoInputFormatChanged(EventTime eventTime, Format format) {} + @SuppressWarnings("deprecation") + default void onVideoInputFormatChanged( + EventTime eventTime, Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { + onVideoInputFormatChanged(eventTime, format); + } /** * Called after video frames have been dropped. @@ -571,6 +897,14 @@ default void onVideoInputFormatChanged(EventTime eventTime, Format format) {} */ default void onDroppedVideoFrames(EventTime eventTime, int droppedFrames, long elapsedMs) {} + /** + * Called when a video renderer releases a decoder. + * + * @param eventTime The event time. + * @param decoderName The decoder that was released. + */ + default void onVideoDecoderReleased(EventTime eventTime, String decoderName) {} + /** * Called when a video renderer is disabled. * @@ -681,4 +1015,40 @@ default void onDrmKeysRemoved(EventTime eventTime) {} * @param eventTime The event time. */ default void onDrmSessionReleased(EventTime eventTime) {} + + /** + * Called when the {@link Player} is released. + * + * @param eventTime The event time. + */ + default void onPlayerReleased(EventTime eventTime) {} + + /** + * Called after one or more events occurred. + * + *

    State changes and events that happen within one {@link Looper} message queue iteration are + * reported together and only after all individual callbacks were triggered. + * + *

    Listeners should prefer this method over individual callbacks in the following cases: + * + *

      + *
    • They intend to trigger the same logic for multiple events (e.g. when updating a UI for + * both {@link #onPlaybackStateChanged(EventTime, int)} and {@link + * #onPlayWhenReadyChanged(EventTime, boolean, int)}). + *
    • They need access to the {@link Player} object to trigger further events (e.g. to call + * {@link Player#seekTo(long)} after a {@link + * AnalyticsListener#onMediaItemTransition(EventTime, MediaItem, int)}). + *
    • They intend to use multiple state values together or in combination with {@link Player} + * getter methods. For example using {@link Player#getCurrentWindowIndex()} with the {@code + * timeline} provided in {@link #onTimelineChanged(EventTime, int)} is only safe from within + * this method. + *
    • They are interested in events that logically happened together (e.g {@link + * #onPlaybackStateChanged(EventTime, int)} to {@link Player#STATE_BUFFERING} because of + * {@link #onMediaItemTransition(EventTime, MediaItem, int)}). + *
    + * + * @param player The {@link Player}. + * @param events The {@link Events} that occurred in this iteration. + */ + default void onEvents(Player player, Events events) {} } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/analytics/PlaybackStatsListener.java b/library/core/src/main/java/com/google/android/exoplayer2/analytics/PlaybackStatsListener.java index ab137f98e19..d8f4074c68b 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/analytics/PlaybackStatsListener.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/analytics/PlaybackStatsListener.java @@ -15,14 +15,15 @@ */ package com.google.android.exoplayer2.analytics; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; import static java.lang.Math.max; import android.os.SystemClock; +import android.util.Pair; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.Format; -import com.google.android.exoplayer2.PlaybackParameters; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.Timeline.Period; @@ -33,9 +34,7 @@ import com.google.android.exoplayer2.source.LoadEventInfo; import com.google.android.exoplayer2.source.MediaLoadData; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; -import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.trackselection.TrackSelection; -import com.google.android.exoplayer2.trackselection.TrackSelectionArray; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.Util; @@ -82,11 +81,17 @@ public interface Callback { private PlaybackStats finishedPlaybackStats; @Nullable private String activeContentPlayback; @Nullable private String activeAdPlayback; - private boolean playWhenReady; - @Player.State private int playbackState; - private boolean isSuppressed; - private float playbackSpeed; - private boolean onSeekStartedCalled; + + @Nullable private EventTime onSeekStartedEventTime; + @Player.DiscontinuityReason int discontinuityReason; + int droppedFrames; + @Nullable Exception nonFatalException; + long bandwidthTimeMs; + long bandwidthBytes; + @Nullable Format videoFormat; + @Nullable Format audioFormat; + int videoHeight; + int videoWidth; /** * Creates listener for playback stats. @@ -102,9 +107,6 @@ public PlaybackStatsListener(boolean keepHistory, @Nullable Callback callback) { playbackStatsTrackers = new HashMap<>(); sessionStartEventTimes = new HashMap<>(); finishedPlaybackStats = PlaybackStats.EMPTY; - playWhenReady = false; - playbackState = Player.STATE_IDLE; - playbackSpeed = 1f; period = new Period(); sessionManager.setListener(this); } @@ -145,47 +147,18 @@ public PlaybackStats getPlaybackStats() { return activeStatsTracker == null ? null : activeStatsTracker.build(/* isFinal= */ false); } - /** - * Finishes all pending playback sessions. Should be called when the listener is removed from the - * player or when the player is released. - */ - public void finishAllSessions() { - // TODO: Add AnalyticsListener.onAttachedToPlayer and onDetachedFromPlayer to auto-release with - // an actual EventTime. Should also simplify other cases where the listener needs to be released - // separately from the player. - sessionManager.finishAllSessions( - new EventTime( - SystemClock.elapsedRealtime(), - Timeline.EMPTY, - /* windowIndex= */ 0, - /* mediaPeriodId= */ null, - /* eventPlaybackPositionMs= */ 0, - Timeline.EMPTY, - /* currentWindowIndex= */ 0, - /* currentMediaPeriodId= */ null, - /* currentPlaybackPositionMs= */ 0, - /* totalBufferedDurationMs= */ 0)); - } - // PlaybackSessionManager.Listener implementation. @Override public void onSessionCreated(EventTime eventTime, String session) { PlaybackStatsTracker tracker = new PlaybackStatsTracker(keepHistory, eventTime); - if (onSeekStartedCalled) { - tracker.onSeekStarted(eventTime, /* belongsToPlayback= */ true); - } - tracker.onPlaybackStateChanged(eventTime, playbackState, /* belongsToPlayback= */ true); - tracker.onPlayWhenReadyChanged(eventTime, playWhenReady, /* belongsToPlayback= */ true); - tracker.onIsSuppressedChanged(eventTime, isSuppressed, /* belongsToPlayback= */ true); - tracker.onPlaybackSpeedChanged(eventTime, playbackSpeed); playbackStatsTrackers.put(session, tracker); sessionStartEventTimes.put(session, eventTime); } @Override public void onSessionActive(EventTime eventTime, String session) { - Assertions.checkNotNull(playbackStatsTrackers.get(session)).onForeground(eventTime); + checkNotNull(playbackStatsTrackers.get(session)).onForeground(); if (eventTime.mediaPeriodId != null && eventTime.mediaPeriodId.isAd()) { activeAdPlayback = session; } else { @@ -195,33 +168,7 @@ public void onSessionActive(EventTime eventTime, String session) { @Override public void onAdPlaybackStarted(EventTime eventTime, String contentSession, String adSession) { - Assertions.checkState(Assertions.checkNotNull(eventTime.mediaPeriodId).isAd()); - long contentPeriodPositionUs = - eventTime - .timeline - .getPeriodByUid(eventTime.mediaPeriodId.periodUid, period) - .getAdGroupTimeUs(eventTime.mediaPeriodId.adGroupIndex); - long contentWindowPositionUs = - contentPeriodPositionUs == C.TIME_END_OF_SOURCE - ? C.TIME_END_OF_SOURCE - : contentPeriodPositionUs + period.getPositionInWindowUs(); - EventTime contentEventTime = - new EventTime( - eventTime.realtimeMs, - eventTime.timeline, - eventTime.windowIndex, - new MediaPeriodId( - eventTime.mediaPeriodId.periodUid, - eventTime.mediaPeriodId.windowSequenceNumber, - eventTime.mediaPeriodId.adGroupIndex), - /* eventPlaybackPositionMs= */ C.usToMs(contentWindowPositionUs), - eventTime.timeline, - eventTime.currentWindowIndex, - eventTime.currentMediaPeriodId, - eventTime.currentPlaybackPositionMs, - eventTime.totalBufferedDurationMs); - Assertions.checkNotNull(playbackStatsTrackers.get(contentSession)) - .onInterruptedByAd(contentEventTime); + checkNotNull(playbackStatsTrackers.get(contentSession)).onInterruptedByAd(); } @Override @@ -231,13 +178,9 @@ public void onSessionFinished(EventTime eventTime, String session, boolean autom } else if (session.equals(activeContentPlayback)) { activeContentPlayback = null; } - PlaybackStatsTracker tracker = Assertions.checkNotNull(playbackStatsTrackers.remove(session)); - EventTime startEventTime = Assertions.checkNotNull(sessionStartEventTimes.remove(session)); - if (automaticTransition) { - // Simulate ENDED state to record natural ending of playback. - tracker.onPlaybackStateChanged(eventTime, Player.STATE_ENDED, /* belongsToPlayback= */ false); - } - tracker.onFinished(eventTime); + PlaybackStatsTracker tracker = checkNotNull(playbackStatsTrackers.remove(session)); + EventTime startEventTime = checkNotNull(sessionStartEventTimes.remove(session)); + tracker.onFinished(eventTime, automaticTransition); PlaybackStats playbackStats = tracker.build(/* isFinal= */ true); finishedPlaybackStats = PlaybackStats.merge(finishedPlaybackStats, playbackStats); if (callback != null) { @@ -247,212 +190,180 @@ public void onSessionFinished(EventTime eventTime, String session, boolean autom // AnalyticsListener implementation. - @Override - public void onPlaybackStateChanged(EventTime eventTime, @Player.State int state) { - playbackState = state; - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - boolean belongsToPlayback = sessionManager.belongsToSession(eventTime, session); - playbackStatsTrackers - .get(session) - .onPlaybackStateChanged(eventTime, playbackState, belongsToPlayback); - } - } - - @Override - public void onPlayWhenReadyChanged( - EventTime eventTime, boolean playWhenReady, @Player.PlayWhenReadyChangeReason int reason) { - this.playWhenReady = playWhenReady; - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - boolean belongsToPlayback = sessionManager.belongsToSession(eventTime, session); - playbackStatsTrackers - .get(session) - .onPlayWhenReadyChanged(eventTime, playWhenReady, belongsToPlayback); - } - } - - @Override - public void onPlaybackSuppressionReasonChanged( - EventTime eventTime, @Player.PlaybackSuppressionReason int playbackSuppressionReason) { - isSuppressed = playbackSuppressionReason != Player.PLAYBACK_SUPPRESSION_REASON_NONE; - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - boolean belongsToPlayback = sessionManager.belongsToSession(eventTime, session); - playbackStatsTrackers - .get(session) - .onIsSuppressedChanged(eventTime, isSuppressed, belongsToPlayback); - } - } - - @Override - public void onTimelineChanged(EventTime eventTime, @Player.TimelineChangeReason int reason) { - sessionManager.updateSessionsWithTimelineChange(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onPositionDiscontinuity(eventTime, /* isSeek= */ false); - } - } - } - @Override public void onPositionDiscontinuity(EventTime eventTime, @Player.DiscontinuityReason int reason) { - boolean isCompletelyIdle = eventTime.timeline.isEmpty() && playbackState == Player.STATE_IDLE; - if (!isCompletelyIdle) { - sessionManager.updateSessionsWithDiscontinuity(eventTime, reason); - } - if (reason == Player.DISCONTINUITY_REASON_SEEK) { - onSeekStartedCalled = false; - } - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers - .get(session) - .onPositionDiscontinuity( - eventTime, /* isSeek= */ reason == Player.DISCONTINUITY_REASON_SEEK); - } - } + discontinuityReason = reason; } @Override public void onSeekStarted(EventTime eventTime) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - boolean belongsToPlayback = sessionManager.belongsToSession(eventTime, session); - playbackStatsTrackers.get(session).onSeekStarted(eventTime, belongsToPlayback); - } - onSeekStartedCalled = true; + onSeekStartedEventTime = eventTime; } @Override - public void onPlayerError(EventTime eventTime, ExoPlaybackException error) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onFatalError(eventTime, error); - } - } + public void onDroppedVideoFrames(EventTime eventTime, int droppedFrames, long elapsedMs) { + this.droppedFrames = droppedFrames; } @Override - public void onPlaybackParametersChanged( - EventTime eventTime, PlaybackParameters playbackParameters) { - playbackSpeed = playbackParameters.speed; - maybeAddSession(eventTime); - for (PlaybackStatsTracker tracker : playbackStatsTrackers.values()) { - tracker.onPlaybackSpeedChanged(eventTime, playbackSpeed); - } + public void onLoadError( + EventTime eventTime, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + nonFatalException = error; } @Override - public void onTracksChanged( - EventTime eventTime, TrackGroupArray trackGroups, TrackSelectionArray trackSelections) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onTracksChanged(eventTime, trackSelections); - } - } + public void onDrmSessionManagerError(EventTime eventTime, Exception error) { + nonFatalException = error; } @Override - public void onLoadStarted( - EventTime eventTime, LoadEventInfo loadEventInfo, MediaLoadData mediaLoadData) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onLoadStarted(eventTime); - } - } + public void onBandwidthEstimate( + EventTime eventTime, int totalLoadTimeMs, long totalBytesLoaded, long bitrateEstimate) { + bandwidthTimeMs = totalLoadTimeMs; + bandwidthBytes = totalBytesLoaded; } @Override public void onDownstreamFormatChanged(EventTime eventTime, MediaLoadData mediaLoadData) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onDownstreamFormatChanged(eventTime, mediaLoadData); - } + if (mediaLoadData.trackType == C.TRACK_TYPE_VIDEO + || mediaLoadData.trackType == C.TRACK_TYPE_DEFAULT) { + videoFormat = mediaLoadData.trackFormat; + } else if (mediaLoadData.trackType == C.TRACK_TYPE_AUDIO) { + audioFormat = mediaLoadData.trackFormat; } } @Override public void onVideoSizeChanged( - EventTime eventTime, - int width, - int height, - int unappliedRotationDegrees, - float pixelWidthHeightRatio) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onVideoSizeChanged(eventTime, width, height); - } - } + EventTime eventTime, int width, int height, int rotationDegrees, float pixelRatio) { + videoWidth = width; + videoHeight = height; } @Override - public void onBandwidthEstimate( - EventTime eventTime, int totalLoadTimeMs, long totalBytesLoaded, long bitrateEstimate) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onBandwidthData(totalLoadTimeMs, totalBytesLoaded); - } + public void onEvents(Player player, Events events) { + if (events.size() == 0) { + return; } - } - - @Override - public void onAudioUnderrun( - EventTime eventTime, int bufferSize, long bufferSizeMs, long elapsedSinceLastFeedMs) { - maybeAddSession(eventTime); + maybeAddSessions(player, events); for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onAudioUnderrun(); - } + Pair eventTimeAndBelongsToPlayback = findBestEventTime(events, session); + PlaybackStatsTracker tracker = playbackStatsTrackers.get(session); + boolean hasPositionDiscontinuity = + hasEvent(events, session, EVENT_POSITION_DISCONTINUITY) + || hasEvent(events, session, EVENT_TIMELINE_CHANGED); + boolean hasDroppedFrames = hasEvent(events, session, EVENT_DROPPED_VIDEO_FRAMES); + boolean hasAudioUnderrun = hasEvent(events, session, EVENT_AUDIO_UNDERRUN); + boolean startedLoading = hasEvent(events, session, EVENT_LOAD_STARTED); + boolean hasFatalError = hasEvent(events, session, EVENT_PLAYER_ERROR); + boolean hasNonFatalException = + hasEvent(events, session, EVENT_LOAD_ERROR) + || hasEvent(events, session, EVENT_DRM_SESSION_MANAGER_ERROR); + boolean hasBandwidthData = hasEvent(events, session, EVENT_BANDWIDTH_ESTIMATE); + boolean hasFormatData = hasEvent(events, session, EVENT_DOWNSTREAM_FORMAT_CHANGED); + boolean hasVideoSize = hasEvent(events, session, EVENT_VIDEO_SIZE_CHANGED); + tracker.onEvents( + player, + /* eventTime= */ eventTimeAndBelongsToPlayback.first, + /* belongsToPlayback= */ eventTimeAndBelongsToPlayback.second, + /* seeked= */ onSeekStartedEventTime != null, + hasPositionDiscontinuity, + hasDroppedFrames ? droppedFrames : 0, + hasAudioUnderrun, + startedLoading, + hasFatalError ? player.getPlayerError() : null, + hasNonFatalException ? nonFatalException : null, + hasBandwidthData ? bandwidthTimeMs : 0, + hasBandwidthData ? bandwidthBytes : 0, + hasFormatData ? videoFormat : null, + hasFormatData ? audioFormat : null, + hasVideoSize ? videoHeight : Format.NO_VALUE, + hasVideoSize ? videoWidth : Format.NO_VALUE); + } + onSeekStartedEventTime = null; + videoFormat = null; + audioFormat = null; + if (events.contains(AnalyticsListener.EVENT_PLAYER_RELEASED)) { + sessionManager.finishAllSessions(events.getEventTime(EVENT_PLAYER_RELEASED)); } } - @Override - public void onDroppedVideoFrames(EventTime eventTime, int droppedFrames, long elapsedMs) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onDroppedVideoFrames(droppedFrames); - } + private void maybeAddSessions(Player player, Events events) { + if (player.getCurrentTimeline().isEmpty() && player.getPlaybackState() == Player.STATE_IDLE) { + // Player is completely idle. Don't add new sessions. + return; } - } - - @Override - public void onLoadError( - EventTime eventTime, - LoadEventInfo loadEventInfo, - MediaLoadData mediaLoadData, - IOException error, - boolean wasCanceled) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onNonFatalError(eventTime, error); + for (int i = 0; i < events.size(); i++) { + @EventFlags int event = events.get(i); + EventTime eventTime = events.getEventTime(event); + if (event == EVENT_TIMELINE_CHANGED) { + sessionManager.updateSessionsWithTimelineChange(eventTime); + } else if (event == EVENT_POSITION_DISCONTINUITY) { + sessionManager.updateSessionsWithDiscontinuity(eventTime, discontinuityReason); + } else { + sessionManager.updateSessions(eventTime); } } } - @Override - public void onDrmSessionManagerError(EventTime eventTime, Exception error) { - maybeAddSession(eventTime); - for (String session : playbackStatsTrackers.keySet()) { - if (sessionManager.belongsToSession(eventTime, session)) { - playbackStatsTrackers.get(session).onNonFatalError(eventTime, error); - } - } + private Pair findBestEventTime(Events events, String session) { + // Check all event times of the events as well as the event time when a seek started. + @Nullable EventTime eventTime = onSeekStartedEventTime; + boolean belongsToPlayback = + onSeekStartedEventTime != null + && sessionManager.belongsToSession(onSeekStartedEventTime, session); + for (int i = 0; i < events.size(); i++) { + @EventFlags int event = events.get(i); + EventTime newEventTime = events.getEventTime(event); + boolean newBelongsToPlayback = sessionManager.belongsToSession(newEventTime, session); + if (eventTime == null + || (newBelongsToPlayback && !belongsToPlayback) + || (newBelongsToPlayback == belongsToPlayback + && newEventTime.realtimeMs > eventTime.realtimeMs)) { + // Prefer event times for the current playback and prefer later timestamps. + eventTime = newEventTime; + belongsToPlayback = newBelongsToPlayback; + } + } + checkNotNull(eventTime); + if (!belongsToPlayback && eventTime.mediaPeriodId != null && eventTime.mediaPeriodId.isAd()) { + // Replace ad event time with content event time unless it's for the ad playback itself. + long contentPeriodPositionUs = + eventTime + .timeline + .getPeriodByUid(eventTime.mediaPeriodId.periodUid, period) + .getAdGroupTimeUs(eventTime.mediaPeriodId.adGroupIndex); + if (contentPeriodPositionUs == C.TIME_END_OF_SOURCE) { + contentPeriodPositionUs = period.durationUs; + } + long contentWindowPositionUs = contentPeriodPositionUs + period.getPositionInWindowUs(); + eventTime = + new EventTime( + eventTime.realtimeMs, + eventTime.timeline, + eventTime.windowIndex, + new MediaPeriodId( + eventTime.mediaPeriodId.periodUid, + eventTime.mediaPeriodId.windowSequenceNumber, + eventTime.mediaPeriodId.adGroupIndex), + /* eventPlaybackPositionMs= */ C.usToMs(contentWindowPositionUs), + eventTime.timeline, + eventTime.currentWindowIndex, + eventTime.currentMediaPeriodId, + eventTime.currentPlaybackPositionMs, + eventTime.totalBufferedDurationMs); + belongsToPlayback = sessionManager.belongsToSession(eventTime, session); + } + return Pair.create(eventTime, belongsToPlayback); } - private void maybeAddSession(EventTime eventTime) { - boolean isCompletelyIdle = eventTime.timeline.isEmpty() && playbackState == Player.STATE_IDLE; - if (!isCompletelyIdle) { - sessionManager.updateSessions(eventTime); - } + private boolean hasEvent(Events events, String session, @EventFlags int event) { + return events.contains(event) + && sessionManager.belongsToSession(events.getEventTime(event), session); } /** Tracker for playback stats of a single playback. */ @@ -500,10 +411,6 @@ private static final class PlaybackStatsTracker { private boolean isSeeking; private boolean isForeground; private boolean isInterruptedByAd; - private boolean isFinished; - private boolean playWhenReady; - @Player.State private int playerPlaybackState; - private boolean isSuppressed; private boolean hasFatalError; private boolean startedLoading; private long lastRebufferStartTimeMs; @@ -530,7 +437,6 @@ public PlaybackStatsTracker(boolean keepHistory, EventTime startTime) { nonFatalErrorHistory = keepHistory ? new ArrayList<>() : Collections.emptyList(); currentPlaybackState = PlaybackStats.PLAYBACK_STATE_NOT_STARTED; currentPlaybackStateStartTimeMs = startTime.realtimeMs; - playerPlaybackState = Player.STATE_IDLE; firstReportedTimeMs = C.TIME_UNSET; maxRebufferTimeMs = C.TIME_UNSET; isAd = startTime.mediaPeriodId != null && startTime.mediaPeriodId.isAd(); @@ -540,247 +446,156 @@ public PlaybackStatsTracker(boolean keepHistory, EventTime startTime) { currentPlaybackSpeed = 1f; } - /** - * Notifies the tracker of a playback state change event, including all playback state changes - * while the playback is not in the foreground. - * - * @param eventTime The {@link EventTime}. - * @param state The current {@link Player.State}. - * @param belongsToPlayback Whether the {@code eventTime} belongs to the current playback. - */ - public void onPlaybackStateChanged( - EventTime eventTime, @Player.State int state, boolean belongsToPlayback) { - playerPlaybackState = state; - if (state != Player.STATE_IDLE) { - hasFatalError = false; - } - if (state != Player.STATE_BUFFERING) { - isSeeking = false; - } - if (state == Player.STATE_IDLE || state == Player.STATE_ENDED) { - isInterruptedByAd = false; - } - maybeUpdatePlaybackState(eventTime, belongsToPlayback); - } - - /** - * Notifies the tracker of a play when ready change event, including all play when ready changes - * while the playback is not in the foreground. - * - * @param eventTime The {@link EventTime}. - * @param playWhenReady Whether the playback will proceed when ready. - * @param belongsToPlayback Whether the {@code eventTime} belongs to the current playback. - */ - public void onPlayWhenReadyChanged( - EventTime eventTime, boolean playWhenReady, boolean belongsToPlayback) { - this.playWhenReady = playWhenReady; - maybeUpdatePlaybackState(eventTime, belongsToPlayback); - } - - /** - * Notifies the tracker of a change to the playback suppression (e.g. due to audio focus loss), - * including all updates while the playback is not in the foreground. - * - * @param eventTime The {@link EventTime}. - * @param isSuppressed Whether playback is suppressed. - * @param belongsToPlayback Whether the {@code eventTime} belongs to the current playback. - */ - public void onIsSuppressedChanged( - EventTime eventTime, boolean isSuppressed, boolean belongsToPlayback) { - this.isSuppressed = isSuppressed; - maybeUpdatePlaybackState(eventTime, belongsToPlayback); - } - - /** - * Notifies the tracker of a position discontinuity or timeline update for the current playback. - * - * @param eventTime The {@link EventTime}. - * @param isSeek Whether the position discontinuity is for a seek. - */ - public void onPositionDiscontinuity(EventTime eventTime, boolean isSeek) { - if (isSeek && playerPlaybackState == Player.STATE_IDLE) { - isSeeking = false; - } - isInterruptedByAd = false; - maybeUpdatePlaybackState(eventTime, /* belongsToPlayback= */ true); - } - - /** - * Notifies the tracker of the start of a seek, including all seeks while the playback is not in - * the foreground. - * - * @param eventTime The {@link EventTime}. - * @param belongsToPlayback Whether the {@code eventTime} belongs to the current playback. - */ - public void onSeekStarted(EventTime eventTime, boolean belongsToPlayback) { - isSeeking = true; - maybeUpdatePlaybackState(eventTime, belongsToPlayback); - } - - /** - * Notifies the tracker of fatal player error in the current playback. - * - * @param eventTime The {@link EventTime}. - */ - public void onFatalError(EventTime eventTime, Exception error) { - fatalErrorCount++; - if (keepHistory) { - fatalErrorHistory.add(new EventTimeAndException(eventTime, error)); - } - hasFatalError = true; - isInterruptedByAd = false; - isSeeking = false; - maybeUpdatePlaybackState(eventTime, /* belongsToPlayback= */ true); - } - - /** - * Notifies the tracker that a load for the current playback has started. - * - * @param eventTime The {@link EventTime}. - */ - public void onLoadStarted(EventTime eventTime) { - startedLoading = true; - maybeUpdatePlaybackState(eventTime, /* belongsToPlayback= */ true); - } - - /** - * Notifies the tracker that the current playback became the active foreground playback. - * - * @param eventTime The {@link EventTime}. - */ - public void onForeground(EventTime eventTime) { + /** Notifies the tracker that the current playback became the active foreground playback. */ + public void onForeground() { isForeground = true; - maybeUpdatePlaybackState(eventTime, /* belongsToPlayback= */ true); } - /** - * Notifies the tracker that the current playback has been interrupted for ad playback. - * - * @param eventTime The {@link EventTime}. - */ - public void onInterruptedByAd(EventTime eventTime) { + /** Notifies the tracker that the current playback is interrupted by an ad. */ + public void onInterruptedByAd() { isInterruptedByAd = true; isSeeking = false; - maybeUpdatePlaybackState(eventTime, /* belongsToPlayback= */ true); } /** * Notifies the tracker that the current playback has finished. * - * @param eventTime The {@link EventTime}. Not guaranteed to belong to the current playback. + * @param eventTime The {@link EventTime}. Does not belong to this playback. + * @param automaticTransition Whether the playback finished because of an automatic transition + * to the next playback item. */ - public void onFinished(EventTime eventTime) { - isFinished = true; - maybeUpdatePlaybackState(eventTime, /* belongsToPlayback= */ false); + public void onFinished(EventTime eventTime, boolean automaticTransition) { + // Simulate state change to ENDED to record natural ending of playback. + @PlaybackState + int finalPlaybackState = + currentPlaybackState == PlaybackStats.PLAYBACK_STATE_ENDED || automaticTransition + ? PlaybackStats.PLAYBACK_STATE_ENDED + : PlaybackStats.PLAYBACK_STATE_ABANDONED; + maybeUpdateMediaTimeHistory(eventTime.realtimeMs, /* mediaTimeMs= */ C.TIME_UNSET); + maybeRecordVideoFormatTime(eventTime.realtimeMs); + maybeRecordAudioFormatTime(eventTime.realtimeMs); + updatePlaybackState(finalPlaybackState, eventTime); } /** - * Notifies the tracker that the track selection for the current playback changed. + * Notifies the tracker of new events. * - * @param eventTime The {@link EventTime}. - * @param trackSelections The new {@link TrackSelectionArray}. + * @param player The {@link Player}. + * @param eventTime The {@link EventTime} of the events. + * @param belongsToPlayback Whether the {@code eventTime} belongs to this playback. + * @param seeked Whether a seek occurred. + * @param positionDiscontinuity Whether a position discontinuity occurred for this playback. + * @param droppedFrameCount The number of newly dropped frames for this playback. + * @param hasAudioUnderun Whether a new audio underrun occurred for this playback. + * @param startedLoading Whether this playback started loading. + * @param fatalError A fatal error for this playback, or null. + * @param nonFatalException A non-fatal exception for this playback, or null. + * @param bandwidthTimeMs The time in milliseconds spent loading for this playback. + * @param bandwidthBytes The number of bytes loaded for this playback. + * @param videoFormat A reported downstream video format for this playback, or null. + * @param audioFormat A reported downstream audio format for this playback, or null. + * @param videoHeight The reported video height for this playback, or {@link Format#NO_VALUE}. + * @param videoWidth The reported video width for this playback, or {@link Format#NO_VALUE}. */ - public void onTracksChanged(EventTime eventTime, TrackSelectionArray trackSelections) { - boolean videoEnabled = false; - boolean audioEnabled = false; - for (TrackSelection trackSelection : trackSelections.getAll()) { - if (trackSelection != null && trackSelection.length() > 0) { - int trackType = MimeTypes.getTrackType(trackSelection.getFormat(0).sampleMimeType); - if (trackType == C.TRACK_TYPE_VIDEO) { - videoEnabled = true; - } else if (trackType == C.TRACK_TYPE_AUDIO) { - audioEnabled = true; + public void onEvents( + Player player, + EventTime eventTime, + boolean belongsToPlayback, + boolean seeked, + boolean positionDiscontinuity, + int droppedFrameCount, + boolean hasAudioUnderun, + boolean startedLoading, + @Nullable ExoPlaybackException fatalError, + @Nullable Exception nonFatalException, + long bandwidthTimeMs, + long bandwidthBytes, + @Nullable Format videoFormat, + @Nullable Format audioFormat, + int videoHeight, + int videoWidth) { + if (seeked) { + isSeeking = true; + } + if (player.getPlaybackState() != Player.STATE_BUFFERING) { + isSeeking = false; + } + int playerPlaybackState = player.getPlaybackState(); + if (playerPlaybackState == Player.STATE_IDLE + || playerPlaybackState == Player.STATE_ENDED + || positionDiscontinuity) { + isInterruptedByAd = false; + } + if (fatalError != null) { + hasFatalError = true; + fatalErrorCount++; + if (keepHistory) { + fatalErrorHistory.add(new EventTimeAndException(eventTime, fatalError)); + } + } else if (player.getPlayerError() == null) { + hasFatalError = false; + } + if (isForeground && !isInterruptedByAd) { + boolean videoEnabled = false; + boolean audioEnabled = false; + for (TrackSelection trackSelection : player.getCurrentTrackSelections().getAll()) { + if (trackSelection != null && trackSelection.length() > 0) { + int trackType = MimeTypes.getTrackType(trackSelection.getFormat(0).sampleMimeType); + if (trackType == C.TRACK_TYPE_VIDEO) { + videoEnabled = true; + } else if (trackType == C.TRACK_TYPE_AUDIO) { + audioEnabled = true; + } } } + if (!videoEnabled) { + maybeUpdateVideoFormat(eventTime, /* newFormat= */ null); + } + if (!audioEnabled) { + maybeUpdateAudioFormat(eventTime, /* newFormat= */ null); + } } - if (!videoEnabled) { - maybeUpdateVideoFormat(eventTime, /* newFormat= */ null); + if (videoFormat != null) { + maybeUpdateVideoFormat(eventTime, videoFormat); } - if (!audioEnabled) { - maybeUpdateAudioFormat(eventTime, /* newFormat= */ null); + if (audioFormat != null) { + maybeUpdateAudioFormat(eventTime, audioFormat); } - } - - /** - * Notifies the tracker that a format being read by the renderers for the current playback - * changed. - * - * @param eventTime The {@link EventTime}. - * @param mediaLoadData The {@link MediaLoadData} describing the format change. - */ - public void onDownstreamFormatChanged(EventTime eventTime, MediaLoadData mediaLoadData) { - if (mediaLoadData.trackType == C.TRACK_TYPE_VIDEO - || mediaLoadData.trackType == C.TRACK_TYPE_DEFAULT) { - maybeUpdateVideoFormat(eventTime, mediaLoadData.trackFormat); - } else if (mediaLoadData.trackType == C.TRACK_TYPE_AUDIO) { - maybeUpdateAudioFormat(eventTime, mediaLoadData.trackFormat); + if (currentVideoFormat != null + && currentVideoFormat.height == Format.NO_VALUE + && videoHeight != Format.NO_VALUE) { + Format formatWithHeightAndWidth = + currentVideoFormat.buildUpon().setWidth(videoWidth).setHeight(videoHeight).build(); + maybeUpdateVideoFormat(eventTime, formatWithHeightAndWidth); } - } - - /** - * Notifies the tracker that the video size for the current playback changed. - * - * @param eventTime The {@link EventTime}. - * @param width The video width in pixels. - * @param height The video height in pixels. - */ - public void onVideoSizeChanged(EventTime eventTime, int width, int height) { - if (currentVideoFormat != null && currentVideoFormat.height == Format.NO_VALUE) { - Format formatWithHeight = - currentVideoFormat.buildUpon().setWidth(width).setHeight(height).build(); - maybeUpdateVideoFormat(eventTime, formatWithHeight); + if (startedLoading) { + this.startedLoading = true; + } + if (hasAudioUnderun) { + audioUnderruns++; + } + this.droppedFrames += droppedFrameCount; + this.bandwidthTimeMs += bandwidthTimeMs; + this.bandwidthBytes += bandwidthBytes; + if (nonFatalException != null) { + nonFatalErrorCount++; + if (keepHistory) { + nonFatalErrorHistory.add(new EventTimeAndException(eventTime, nonFatalException)); + } } - } - - /** - * Notifies the tracker of a playback speed change, including all playback speed changes while - * the playback is not in the foreground. - * - * @param eventTime The {@link EventTime}. - * @param playbackSpeed The new playback speed. - */ - public void onPlaybackSpeedChanged(EventTime eventTime, float playbackSpeed) { - maybeUpdateMediaTimeHistory(eventTime.realtimeMs, eventTime.eventPlaybackPositionMs); - maybeRecordVideoFormatTime(eventTime.realtimeMs); - maybeRecordAudioFormatTime(eventTime.realtimeMs); - currentPlaybackSpeed = playbackSpeed; - } - - /** Notifies the builder of an audio underrun for the current playback. */ - public void onAudioUnderrun() { - audioUnderruns++; - } - - /** - * Notifies the tracker of dropped video frames for the current playback. - * - * @param droppedFrames The number of dropped video frames. - */ - public void onDroppedVideoFrames(int droppedFrames) { - this.droppedFrames += droppedFrames; - } - - /** - * Notifies the tracker of bandwidth measurement data for the current playback. - * - * @param timeMs The time for which bandwidth measurement data is available, in milliseconds. - * @param bytes The bytes transferred during {@code timeMs}. - */ - public void onBandwidthData(long timeMs, long bytes) { - bandwidthTimeMs += timeMs; - bandwidthBytes += bytes; - } - /** - * Notifies the tracker of a non-fatal error in the current playback. - * - * @param eventTime The {@link EventTime}. - * @param error The error. - */ - public void onNonFatalError(EventTime eventTime, Exception error) { - nonFatalErrorCount++; - if (keepHistory) { - nonFatalErrorHistory.add(new EventTimeAndException(eventTime, error)); + @PlaybackState int newPlaybackState = resolveNewPlaybackState(player); + float newPlaybackSpeed = player.getPlaybackParameters().speed; + if (currentPlaybackState != newPlaybackState || currentPlaybackSpeed != newPlaybackSpeed) { + maybeUpdateMediaTimeHistory( + eventTime.realtimeMs, + belongsToPlayback ? eventTime.eventPlaybackPositionMs : C.TIME_UNSET); + maybeRecordVideoFormatTime(eventTime.realtimeMs); + maybeRecordAudioFormatTime(eventTime.realtimeMs); + } + currentPlaybackSpeed = newPlaybackSpeed; + if (currentPlaybackState != newPlaybackState) { + updatePlaybackState(newPlaybackState, eventTime); } } @@ -860,13 +675,8 @@ public PlaybackStats build(boolean isFinal) { nonFatalErrorHistory); } - private void maybeUpdatePlaybackState(EventTime eventTime, boolean belongsToPlayback) { - @PlaybackState int newPlaybackState = resolveNewPlaybackState(); - if (newPlaybackState == currentPlaybackState) { - return; - } + private void updatePlaybackState(@PlaybackState int newPlaybackState, EventTime eventTime) { Assertions.checkArgument(eventTime.realtimeMs >= currentPlaybackStateStartTimeMs); - long stateDurationMs = eventTime.realtimeMs - currentPlaybackStateStartTimeMs; playbackStateDurationsMs[currentPlaybackState] += stateDurationMs; if (firstReportedTimeMs == C.TIME_UNSET) { @@ -890,13 +700,7 @@ private void maybeUpdatePlaybackState(EventTime eventTime, boolean belongsToPlay && newPlaybackState == PlaybackStats.PLAYBACK_STATE_PAUSED_BUFFERING) { pauseBufferCount++; } - - maybeUpdateMediaTimeHistory( - eventTime.realtimeMs, - /* mediaTimeMs= */ belongsToPlayback ? eventTime.eventPlaybackPositionMs : C.TIME_UNSET); maybeUpdateMaxRebufferTimeMs(eventTime.realtimeMs); - maybeRecordVideoFormatTime(eventTime.realtimeMs); - maybeRecordAudioFormatTime(eventTime.realtimeMs); currentPlaybackState = newPlaybackState; currentPlaybackStateStartTimeMs = eventTime.realtimeMs; @@ -905,13 +709,9 @@ private void maybeUpdatePlaybackState(EventTime eventTime, boolean belongsToPlay } } - private @PlaybackState int resolveNewPlaybackState() { - if (isFinished) { - // Keep VIDEO_STATE_ENDED if playback naturally ended (or progressed to next item). - return currentPlaybackState == PlaybackStats.PLAYBACK_STATE_ENDED - ? PlaybackStats.PLAYBACK_STATE_ENDED - : PlaybackStats.PLAYBACK_STATE_ABANDONED; - } else if (isSeeking && isForeground) { + private @PlaybackState int resolveNewPlaybackState(Player player) { + @Player.State int playerPlaybackState = player.getPlaybackState(); + if (isSeeking && isForeground) { // Seeking takes precedence over errors such that we report a seek while in error state. return PlaybackStats.PLAYBACK_STATE_SEEKING; } else if (hasFatalError) { @@ -932,17 +732,17 @@ private void maybeUpdatePlaybackState(EventTime eventTime, boolean belongsToPlay || currentPlaybackState == PlaybackStats.PLAYBACK_STATE_INTERRUPTED_BY_AD) { return PlaybackStats.PLAYBACK_STATE_JOINING_FOREGROUND; } - if (!playWhenReady) { + if (!player.getPlayWhenReady()) { return PlaybackStats.PLAYBACK_STATE_PAUSED_BUFFERING; } - return isSuppressed + return player.getPlaybackSuppressionReason() != Player.PLAYBACK_SUPPRESSION_REASON_NONE ? PlaybackStats.PLAYBACK_STATE_SUPPRESSED_BUFFERING : PlaybackStats.PLAYBACK_STATE_BUFFERING; } else if (playerPlaybackState == Player.STATE_READY) { - if (!playWhenReady) { + if (!player.getPlayWhenReady()) { return PlaybackStats.PLAYBACK_STATE_PAUSED; } - return isSuppressed + return player.getPlaybackSuppressionReason() != Player.PLAYBACK_SUPPRESSION_REASON_NONE ? PlaybackStats.PLAYBACK_STATE_SUPPRESSED : PlaybackStats.PLAYBACK_STATE_PLAYING; } else if (playerPlaybackState == Player.STATE_IDLE diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioRendererEventListener.java b/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioRendererEventListener.java index f921141f24e..82ca6f4b9a0 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioRendererEventListener.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioRendererEventListener.java @@ -17,13 +17,17 @@ import static com.google.android.exoplayer2.util.Util.castNonNull; +import android.media.AudioTrack; import android.os.Handler; import android.os.SystemClock; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Renderer; import com.google.android.exoplayer2.decoder.DecoderCounters; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.util.Assertions; /** @@ -40,13 +44,6 @@ public interface AudioRendererEventListener { */ default void onAudioEnabled(DecoderCounters counters) {} - /** - * Called when the audio session is set. - * - * @param audioSessionId The audio session id. - */ - default void onAudioSessionId(int audioSessionId) {} - /** * Called when a decoder is created. * @@ -58,12 +55,23 @@ default void onAudioSessionId(int audioSessionId) {} default void onAudioDecoderInitialized( String decoderName, long initializedTimestampMs, long initializationDurationMs) {} + /** @deprecated Use {@link #onAudioInputFormatChanged(Format, DecoderReuseEvaluation)}. */ + @Deprecated + default void onAudioInputFormatChanged(Format format) {} + /** * Called when the format of the media being consumed by the renderer changes. * * @param format The new format. + * @param decoderReuseEvaluation The result of the evaluation to determine whether an existing + * decoder instance can be reused for the new format, or {@code null} if the renderer did not + * have a decoder. */ - default void onAudioInputFormatChanged(Format format) {} + @SuppressWarnings("deprecation") + default void onAudioInputFormatChanged( + Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { + onAudioInputFormatChanged(format); + } /** * Called when the audio position has increased for the first time since the last pause or @@ -84,6 +92,13 @@ default void onAudioPositionAdvancing(long playoutStartSystemTimeMs) {} */ default void onAudioUnderrun(int bufferSize, long bufferSizeMs, long elapsedSinceLastFeedMs) {} + /** + * Called when a decoder is released. + * + * @param decoderName The decoder that was released. + */ + default void onAudioDecoderReleased(String decoderName) {} + /** * Called when the renderer is disabled. * @@ -98,6 +113,28 @@ default void onAudioDisabled(DecoderCounters counters) {} */ default void onSkipSilenceEnabledChanged(boolean skipSilenceEnabled) {} + /** + * Called when {@link AudioSink} has encountered an error. + * + *

    If the sink writes to a platform {@link AudioTrack}, this will called for all {@link + * AudioTrack} errors. + * + *

    This method being called does not indicate that playback has failed, or that it will fail. + * The player may be able to recover from the error (for example by recreating the AudioTrack, + * possibly with different settings) and continue. Hence applications should not + * implement this method to display a user visible error or initiate an application level retry + * ({@link Player.EventListener#onPlayerError} is the appropriate place to implement such + * behavior). This method is called to provide the application with an opportunity to log the + * error if it wishes to do so. + * + *

    Fatal errors that cannot be recovered will be reported wrapped in a {@link + * ExoPlaybackException} by {@link Player.EventListener#onPlayerError(ExoPlaybackException)}. + * + * @param audioSinkError Either an {@link AudioSink.InitializationException} or a {@link + * AudioSink.WriteException} describing the error. + */ + default void onAudioSinkError(Exception audioSinkError) {} + /** Dispatches events to an {@link AudioRendererEventListener}. */ final class EventDispatcher { @@ -135,9 +172,11 @@ public void decoderInitialized( } /** Invokes {@link AudioRendererEventListener#onAudioInputFormatChanged(Format)}. */ - public void inputFormatChanged(Format format) { + public void inputFormatChanged( + Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { if (handler != null) { - handler.post(() -> castNonNull(listener).onAudioInputFormatChanged(format)); + handler.post( + () -> castNonNull(listener).onAudioInputFormatChanged(format, decoderReuseEvaluation)); } } @@ -159,6 +198,13 @@ public void underrun(int bufferSize, long bufferSizeMs, long elapsedSinceLastFee } } + /** Invokes {@link AudioRendererEventListener#onAudioDecoderReleased(String)}. */ + public void decoderReleased(String decoderName) { + if (handler != null) { + handler.post(() -> castNonNull(listener).onAudioDecoderReleased(decoderName)); + } + } + /** Invokes {@link AudioRendererEventListener#onAudioDisabled(DecoderCounters)}. */ public void disabled(DecoderCounters counters) { counters.ensureUpdated(); @@ -171,17 +217,17 @@ public void disabled(DecoderCounters counters) { } } - /** Invokes {@link AudioRendererEventListener#onAudioSessionId(int)}. */ - public void audioSessionId(int audioSessionId) { + /** Invokes {@link AudioRendererEventListener#onSkipSilenceEnabledChanged(boolean)}. */ + public void skipSilenceEnabledChanged(boolean skipSilenceEnabled) { if (handler != null) { - handler.post(() -> castNonNull(listener).onAudioSessionId(audioSessionId)); + handler.post(() -> castNonNull(listener).onSkipSilenceEnabledChanged(skipSilenceEnabled)); } } - /** Invokes {@link AudioRendererEventListener#onSkipSilenceEnabledChanged(boolean)}. */ - public void skipSilenceEnabledChanged(boolean skipSilenceEnabled) { + /** Invokes {@link AudioRendererEventListener#onAudioSinkError(Exception)}. */ + public void audioSinkError(Exception audioSinkError) { if (handler != null) { - handler.post(() -> castNonNull(listener).onSkipSilenceEnabledChanged(skipSilenceEnabled)); + handler.post(() -> castNonNull(listener).onAudioSinkError(audioSinkError)); } } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioSink.java b/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioSink.java index b7d375fd9dc..463461916f6 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioSink.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/audio/AudioSink.java @@ -19,8 +19,10 @@ import androidx.annotation.IntDef; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.PlaybackParameters; +import com.google.android.exoplayer2.Player; import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; @@ -47,9 +49,9 @@ * *

    The implementation may be backed by a platform {@link AudioTrack}. In this case, {@link * #setAudioSessionId(int)}, {@link #setAudioAttributes(AudioAttributes)}, {@link - * #enableTunnelingV21(int)} and/or {@link #disableTunneling()} may be called before writing data to - * the sink. These methods may also be called after writing data to the sink, in which case it will - * be reinitialized as required. For implementations that are not based on platform {@link + * #enableTunnelingV21()} and {@link #disableTunneling()} may be called before writing data to the + * sink. These methods may also be called after writing data to the sink, in which case it will be + * reinitialized as required. For implementations that are not based on platform {@link * AudioTrack}s, calling methods relating to audio sessions, audio attributes, and tunneling may * have no effect. */ @@ -60,13 +62,6 @@ public interface AudioSink { */ interface Listener { - /** - * Called if the audio sink has started rendering audio to a new platform audio session. - * - * @param audioSessionId The newly generated audio session's identifier. - */ - void onAudioSessionId(int audioSessionId); - /** * Called when the audio sink handles a buffer whose timestamp is discontinuous with the last * buffer handled since it was reset. @@ -113,6 +108,28 @@ default void onOffloadBufferEmptying() {} * #onOffloadBufferEmptying()} will be called. */ default void onOffloadBufferFull(long bufferEmptyingDeadlineMs) {} + + /** + * Called when {@link AudioSink} has encountered an error. + * + *

    If the sink writes to a platform {@link AudioTrack}, this will called for all {@link + * AudioTrack} errors. + * + *

    This method being called does not indicate that playback has failed, or that it will fail. + * The player may be able to recover from the error (for example by recreating the AudioTrack, + * possibly with different settings) and continue. Hence applications should not + * implement this method to display a user visible error or initiate an application level retry + * ({@link Player.EventListener#onPlayerError} is the appropriate place to implement such + * behavior). This method is called to provide the application with an opportunity to log the + * error if it wishes to do so. + * + *

    Fatal errors that cannot be recovered will be reported wrapped in a {@link + * ExoPlaybackException} by {@link Player.EventListener#onPlayerError(ExoPlaybackException)}. + * + * @param audioSinkError Either an {@link AudioSink.InitializationException} or a {@link + * AudioSink.WriteException} describing the error. + */ + default void onAudioSinkError(Exception audioSinkError) {} } /** @@ -120,50 +137,65 @@ default void onOffloadBufferFull(long bufferEmptyingDeadlineMs) {} */ final class ConfigurationException extends Exception { - /** - * Creates a new configuration exception with the specified {@code cause} and no message. - */ - public ConfigurationException(Throwable cause) { + /** Input {@link Format} of the sink when the configuration failure occurs. */ + public final Format format; + + /** Creates a new configuration exception with the specified {@code cause} and no message. */ + public ConfigurationException(Throwable cause, Format format) { super(cause); + this.format = format; } - /** - * Creates a new configuration exception with the specified {@code message} and no cause. - */ - public ConfigurationException(String message) { + /** Creates a new configuration exception with the specified {@code message} and no cause. */ + public ConfigurationException(String message, Format format) { super(message); + this.format = format; } - } - /** - * Thrown when a failure occurs initializing the sink. - */ + /** Thrown when a failure occurs initializing the sink. */ final class InitializationException extends Exception { - /** - * The underlying {@link AudioTrack}'s state, if applicable. - */ + /** The underlying {@link AudioTrack}'s state. */ public final int audioTrackState; + /** If the exception can be recovered by recreating the sink. */ + public final boolean isRecoverable; + /** The input {@link Format} of the sink when the error occurs. */ + public final Format format; /** - * @param audioTrackState The underlying {@link AudioTrack}'s state, if applicable. + * Creates a new instance. + * + * @param audioTrackState The underlying {@link AudioTrack}'s state. * @param sampleRate The requested sample rate in Hz. * @param channelConfig The requested channel configuration. * @param bufferSize The requested buffer size in bytes. + * @param format The input format of the sink when the error occurs. + * @param isRecoverable Whether the exception can be recovered by recreating the sink. + * @param audioTrackException Exception thrown during the creation of the {@link AudioTrack}. */ - public InitializationException(int audioTrackState, int sampleRate, int channelConfig, - int bufferSize) { - super("AudioTrack init failed: " + audioTrackState + ", Config(" + sampleRate + ", " - + channelConfig + ", " + bufferSize + ")"); + public InitializationException( + int audioTrackState, + int sampleRate, + int channelConfig, + int bufferSize, + Format format, + boolean isRecoverable, + @Nullable Exception audioTrackException) { + super( + "AudioTrack init failed " + + audioTrackState + + " " + + ("Config(" + sampleRate + ", " + channelConfig + ", " + bufferSize + ")") + + (isRecoverable ? " (recoverable)" : ""), + audioTrackException); this.audioTrackState = audioTrackState; + this.isRecoverable = isRecoverable; + this.format = format; } - } - /** - * Thrown when a failure occurs writing to the sink. - */ + /** Thrown when a failure occurs writing to the sink. */ final class WriteException extends Exception { /** @@ -173,13 +205,23 @@ final class WriteException extends Exception { * Otherwise, the meaning of the error code depends on the sink implementation. */ public final int errorCode; + /** If the exception can be recovered by recreating the sink. */ + public final boolean isRecoverable; + /** The input {@link Format} of the sink when the error occurs. */ + public final Format format; /** + * Creates an instance. + * * @param errorCode The error value returned from the sink implementation. + * @param format The input format of the sink when the error occurs. + * @param isRecoverable Whether the exception can be recovered by recreating the sink. */ - public WriteException(int errorCode) { + public WriteException(int errorCode, Format format, boolean isRecoverable) { super("AudioTrack write failed: " + errorCode); + this.isRecoverable = isRecoverable; this.errorCode = errorCode; + this.format = format; } } @@ -343,14 +385,13 @@ boolean handleBuffer(ByteBuffer buffer, long presentationTimeUs, int encodedAcce void setAuxEffectInfo(AuxEffectInfo auxEffectInfo); /** - * Enables tunneling, if possible. The sink is reset if tunneling was previously disabled or if - * the audio session id has changed. Enabling tunneling is only possible if the sink is based on a - * platform {@link AudioTrack}, and requires platform API version 21 onwards. + * Enables tunneling, if possible. The sink is reset if tunneling was previously disabled. + * Enabling tunneling is only possible if the sink is based on a platform {@link AudioTrack}, and + * requires platform API version 21 onwards. * - * @param tunnelingAudioSessionId The audio session id to use. * @throws IllegalStateException Thrown if enabling tunneling on platform API version < 21. */ - void enableTunnelingV21(int tunnelingAudioSessionId); + void enableTunnelingV21(); /** * Disables tunneling. If tunneling was previously enabled then the sink is reset and any audio diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/DecoderAudioRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/audio/DecoderAudioRenderer.java index 1c1e593e223..12962a786a6 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/audio/DecoderAudioRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/audio/DecoderAudioRenderer.java @@ -15,9 +15,11 @@ */ package com.google.android.exoplayer2.audio; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_DRM_SESSION_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_REUSE_NOT_IMPLEMENTED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_NO; import static java.lang.Math.max; -import android.media.audiofx.Virtualizer; import android.os.Handler; import android.os.SystemClock; import androidx.annotation.CallSuper; @@ -38,6 +40,7 @@ import com.google.android.exoplayer2.decoder.DecoderCounters; import com.google.android.exoplayer2.decoder.DecoderException; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.decoder.SimpleOutputBuffer; import com.google.android.exoplayer2.drm.DrmSession; import com.google.android.exoplayer2.drm.DrmSession.DrmSessionException; @@ -211,10 +214,10 @@ public MediaClock getMediaClock() { @Capabilities public final int supportsFormat(Format format) { if (!MimeTypes.isAudio(format.sampleMimeType)) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } - @FormatSupport int formatSupport = supportsFormatInternal(format); - if (formatSupport <= FORMAT_UNSUPPORTED_DRM) { + @C.FormatSupport int formatSupport = supportsFormatInternal(format); + if (formatSupport <= C.FORMAT_UNSUPPORTED_DRM) { return RendererCapabilities.create(formatSupport); } @TunnelingSupport @@ -223,12 +226,12 @@ public final int supportsFormat(Format format) { } /** - * Returns the {@link FormatSupport} for the given {@link Format}. + * Returns the {@link C.FormatSupport} for the given {@link Format}. * * @param format The format, which has an audio {@link Format#sampleMimeType}. - * @return The {@link FormatSupport} for this {@link Format}. + * @return The {@link C.FormatSupport} for this {@link Format}. */ - @FormatSupport + @C.FormatSupport protected abstract int supportsFormatInternal(Format format); /** @@ -257,7 +260,7 @@ public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackEx try { audioSink.playToEndOfStream(); } catch (AudioSink.WriteException e) { - throw createRendererException(e, inputFormat); + throw createRendererException(e, e.format, e.isRecoverable); } return; } @@ -296,28 +299,19 @@ public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackEx while (drainOutputBuffer()) {} while (feedInputBuffer()) {} TraceUtil.endSection(); - } catch (DecoderException - | AudioSink.ConfigurationException - | AudioSink.InitializationException - | AudioSink.WriteException e) { + } catch (DecoderException e) { throw createRendererException(e, inputFormat); + } catch (AudioSink.ConfigurationException e) { + throw createRendererException(e, e.format); + } catch (AudioSink.InitializationException e) { + throw createRendererException(e, e.format, e.isRecoverable); + } catch (AudioSink.WriteException e) { + throw createRendererException(e, e.format, e.isRecoverable); } decoderCounters.ensureUpdated(); } } - /** - * Called when the audio session id becomes known. The default implementation is a no-op. One - * reason for overriding this method would be to instantiate and enable a {@link Virtualizer} in - * order to spatialize the audio channels. For this use case, any {@link Virtualizer} instances - * should be released in {@link #onDisabled()} (if not before). - * - *

    See {@link AudioSink.Listener#onAudioSessionId(int)}. - */ - protected void onAudioSessionId(int audioSessionId) { - // Do nothing. - } - /** See {@link AudioSink.Listener#onPositionDiscontinuity()}. */ @CallSuper protected void onPositionDiscontinuity() { @@ -346,14 +340,19 @@ protected abstract T createDecoder(Format format, @Nullable ExoMediaCrypto media protected abstract Format getOutputFormat(T decoder); /** - * Returns whether the existing decoder can be kept for a new format. + * Evaluates whether the existing decoder can be reused for a new {@link Format}. + * + *

    The default implementation does not allow decoder reuse. * + * @param decoderName The name of the decoder. * @param oldFormat The previous format. * @param newFormat The new format. - * @return Whether the existing decoder can be kept. + * @return The result of the evaluation. */ - protected boolean canKeepCodec(Format oldFormat, Format newFormat) { - return false; + protected DecoderReuseEvaluation canReuseDecoder( + String decoderName, Format oldFormat, Format newFormat) { + return new DecoderReuseEvaluation( + decoderName, oldFormat, newFormat, REUSE_RESULT_NO, DISCARD_REASON_REUSE_NOT_IMPLEMENTED); } private boolean drainOutputBuffer() @@ -383,7 +382,7 @@ private boolean drainOutputBuffer() try { processEndOfStream(); } catch (AudioSink.WriteException e) { - throw createRendererException(e, getOutputFormat(decoder)); + throw createRendererException(e, e.format, e.isRecoverable); } } return false; @@ -513,9 +512,8 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) throws ExoPlaybackException { decoderCounters = new DecoderCounters(); eventDispatcher.enabled(decoderCounters); - int tunnelingAudioSessionId = getConfiguration().tunnelingAudioSessionId; - if (tunnelingAudioSessionId != C.AUDIO_SESSION_ID_UNSET) { - audioSink.enableTunnelingV21(tunnelingAudioSessionId); + if (getConfiguration().tunneling) { + audioSink.enableTunnelingV21(); } else { audioSink.disableTunneling(); } @@ -602,8 +600,8 @@ private void maybeInitDecoder() throws ExoPlaybackException { if (mediaCrypto == null) { DrmSessionException drmError = decoderDrmSession.getError(); if (drmError != null) { - // Continue for now. We may be able to avoid failure if the session recovers, or if a new - // input format causes the session to be replaced before it's used. + // Continue for now. We may be able to avoid failure if a new input format causes the + // session to be replaced without it having been used. } else { // The drm session isn't open yet. return; @@ -620,7 +618,7 @@ private void maybeInitDecoder() throws ExoPlaybackException { eventDispatcher.decoderInitialized(decoder.getName(), codecInitializedTimestamp, codecInitializedTimestamp - codecInitializingTimestamp); decoderCounters.decoderInitCount++; - } catch (DecoderException e) { + } catch (DecoderException | OutOfMemoryError e) { throw createRendererException(e, inputFormat); } } @@ -631,9 +629,10 @@ private void releaseDecoder() { decoderReinitializationState = REINITIALIZATION_STATE_NONE; decoderReceivedBuffers = false; if (decoder != null) { + decoderCounters.decoderReleaseCount++; decoder.release(); + eventDispatcher.decoderReleased(decoder.getName()); decoder = null; - decoderCounters.decoderReleaseCount++; } setDecoderDrmSession(null); } @@ -653,10 +652,29 @@ private void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybackE setSourceDrmSession(formatHolder.drmSession); Format oldFormat = inputFormat; inputFormat = newFormat; + encoderDelay = newFormat.encoderDelay; + encoderPadding = newFormat.encoderPadding; if (decoder == null) { maybeInitDecoder(); - } else if (sourceDrmSession != decoderDrmSession || !canKeepCodec(oldFormat, inputFormat)) { + eventDispatcher.inputFormatChanged(inputFormat, /* decoderReuseEvaluation= */ null); + return; + } + + DecoderReuseEvaluation evaluation; + if (sourceDrmSession != decoderDrmSession) { + evaluation = + new DecoderReuseEvaluation( + decoder.getName(), + oldFormat, + newFormat, + REUSE_RESULT_NO, + DISCARD_REASON_DRM_SESSION_CHANGED); + } else { + evaluation = canReuseDecoder(decoder.getName(), oldFormat, newFormat); + } + + if (evaluation.result == REUSE_RESULT_NO) { if (decoderReceivedBuffers) { // Signal end of stream and wait for any final output buffers before re-initialization. decoderReinitializationState = REINITIALIZATION_STATE_SIGNAL_END_OF_STREAM; @@ -667,10 +685,7 @@ private void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybackE audioTrackNeedsConfigure = true; } } - - encoderDelay = inputFormat.encoderDelay; - encoderPadding = inputFormat.encoderPadding; - eventDispatcher.inputFormatChanged(inputFormat); + eventDispatcher.inputFormatChanged(inputFormat, evaluation); } private void onQueueInputBuffer(DecoderInputBuffer buffer) { @@ -698,12 +713,6 @@ private void updateCurrentPosition() { private final class AudioSinkListener implements AudioSink.Listener { - @Override - public void onAudioSessionId(int audioSessionId) { - eventDispatcher.audioSessionId(audioSessionId); - DecoderAudioRenderer.this.onAudioSessionId(audioSessionId); - } - @Override public void onPositionDiscontinuity() { DecoderAudioRenderer.this.onPositionDiscontinuity(); @@ -723,5 +732,10 @@ public void onUnderrun(int bufferSize, long bufferSizeMs, long elapsedSinceLastF public void onSkipSilenceEnabledChanged(boolean skipSilenceEnabled) { eventDispatcher.skipSilenceEnabledChanged(skipSilenceEnabled); } + + @Override + public void onAudioSinkError(Exception audioSinkError) { + eventDispatcher.audioSinkError(audioSinkError); + } } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/DefaultAudioSink.java b/library/core/src/main/java/com/google/android/exoplayer2/audio/DefaultAudioSink.java index 691580143a7..8f16df115a4 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/audio/DefaultAudioSink.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/audio/DefaultAudioSink.java @@ -18,7 +18,6 @@ import static java.lang.Math.max; import static java.lang.Math.min; -import android.annotation.SuppressLint; import android.media.AudioFormat; import android.media.AudioManager; import android.media.AudioTrack; @@ -243,37 +242,24 @@ public long getSkippedOutputFrameCount() { private static final int AC3_BUFFER_MULTIPLICATION_FACTOR = 2; /** - * @see AudioTrack#ERROR_BAD_VALUE - */ - private static final int ERROR_BAD_VALUE = AudioTrack.ERROR_BAD_VALUE; - /** - * @see AudioTrack#MODE_STATIC - */ - private static final int MODE_STATIC = AudioTrack.MODE_STATIC; - /** - * @see AudioTrack#MODE_STREAM - */ - private static final int MODE_STREAM = AudioTrack.MODE_STREAM; - /** - * @see AudioTrack#STATE_INITIALIZED - */ - private static final int STATE_INITIALIZED = AudioTrack.STATE_INITIALIZED; - /** - * @see AudioTrack#WRITE_NON_BLOCKING + * Native error code equivalent of {@link AudioTrack#ERROR_DEAD_OBJECT} to workaround missing + * error code translation on some devices. + * + *

    On some devices, AudioTrack native error codes are not always converted to their SDK + * equivalent. + * + *

    For example: {@link AudioTrack#write(byte[], int, int)} can return -32 instead of {@link + * AudioTrack#ERROR_DEAD_OBJECT}. */ - @SuppressLint("InlinedApi") - private static final int WRITE_NON_BLOCKING = AudioTrack.WRITE_NON_BLOCKING; - - private static final String TAG = "AudioTrack"; + private static final int ERROR_NATIVE_DEAD_OBJECT = -32; /** - * Whether to enable a workaround for an issue where an audio effect does not keep its session - * active across releasing/initializing a new audio track, on platform builds where - * {@link Util#SDK_INT} < 21. - *

    - * The flag must be set before creating a player. + * The duration for which failed attempts to initialize or write to the audio track may be retried + * before throwing an exception, in milliseconds. */ - public static boolean enablePreV21AudioSessionWorkaround = false; + private static final int AUDIO_TRACK_RETRY_DURATION_MS = 100; + + private static final String TAG = "DefaultAudioSink"; /** * Whether to throw an {@link InvalidAudioTrackTimestampException} when a spurious timestamp is @@ -297,13 +283,11 @@ public long getSkippedOutputFrameCount() { private final boolean enableAudioTrackPlaybackParams; private final boolean enableOffload; @MonotonicNonNull private StreamEventCallbackV29 offloadStreamEventCallbackV29; + private final PendingExceptionHolder + initializationExceptionPendingExceptionHolder; + private final PendingExceptionHolder writeExceptionPendingExceptionHolder; @Nullable private Listener listener; - /** - * Used to keep the audio session active on pre-V21 builds (see {@link #initializeAudioTrack()}). - */ - @Nullable private AudioTrack keepSessionIdAudioTrack; - @Nullable private Configuration pendingConfiguration; @MonotonicNonNull private Configuration configuration; @Nullable private AudioTrack audioTrack; @@ -338,6 +322,7 @@ public long getSkippedOutputFrameCount() { private boolean stoppedAudioTrack; private boolean playing; + private boolean externalAudioSessionIdProvided; private int audioSessionId; private AuxEffectInfo auxEffectInfo; private boolean tunneling; @@ -443,6 +428,10 @@ public DefaultAudioSink( activeAudioProcessors = new AudioProcessor[0]; outputBuffers = new ByteBuffer[0]; mediaPositionParametersCheckpoints = new ArrayDeque<>(); + initializationExceptionPendingExceptionHolder = + new PendingExceptionHolder<>(AUDIO_TRACK_RETRY_DURATION_MS); + writeExceptionPendingExceptionHolder = + new PendingExceptionHolder<>(AUDIO_TRACK_RETRY_DURATION_MS); } // AudioSink implementation. @@ -540,7 +529,7 @@ public void configure(Format inputFormat, int specifiedBufferSize, @Nullable int outputFormat = nextFormat; } } catch (UnhandledAudioFormatException e) { - throw new ConfigurationException(e); + throw new ConfigurationException(e, inputFormat); } } @@ -567,7 +556,8 @@ public void configure(Format inputFormat, int specifiedBufferSize, @Nullable int Pair encodingAndChannelConfig = getEncodingAndChannelConfigForPassthrough(inputFormat, audioCapabilities); if (encodingAndChannelConfig == null) { - throw new ConfigurationException("Unable to configure passthrough for: " + inputFormat); + throw new ConfigurationException( + "Unable to configure passthrough for: " + inputFormat, inputFormat); } outputEncoding = encodingAndChannelConfig.first; outputChannelConfig = encodingAndChannelConfig.second; @@ -576,11 +566,12 @@ public void configure(Format inputFormat, int specifiedBufferSize, @Nullable int if (outputEncoding == C.ENCODING_INVALID) { throw new ConfigurationException( - "Invalid output encoding (mode=" + outputMode + ") for: " + inputFormat); + "Invalid output encoding (mode=" + outputMode + ") for: " + inputFormat, inputFormat); } if (outputChannelConfig == AudioFormat.CHANNEL_INVALID) { throw new ConfigurationException( - "Invalid output channel config (mode=" + outputMode + ") for: " + inputFormat); + "Invalid output channel config (mode=" + outputMode + ") for: " + inputFormat, + inputFormat); } offloadDisabledUntilNextConfiguration = false; @@ -642,27 +633,7 @@ private void initializeAudioTrack() throws InitializationException { audioTrack.setOffloadDelayPadding( configuration.inputFormat.encoderDelay, configuration.inputFormat.encoderPadding); } - int audioSessionId = audioTrack.getAudioSessionId(); - if (enablePreV21AudioSessionWorkaround) { - if (Util.SDK_INT < 21) { - // The workaround creates an audio track with a two byte buffer on the same session, and - // does not release it until this object is released, which keeps the session active. - if (keepSessionIdAudioTrack != null - && audioSessionId != keepSessionIdAudioTrack.getAudioSessionId()) { - releaseKeepSessionIdAudioTrack(); - } - if (keepSessionIdAudioTrack == null) { - keepSessionIdAudioTrack = initializeKeepSessionIdAudioTrack(audioSessionId); - } - } - } - if (this.audioSessionId != audioSessionId) { - this.audioSessionId = audioSessionId; - if (listener != null) { - listener.onAudioSessionId(audioSessionId); - } - } - + audioSessionId = audioTrack.getAudioSessionId(); audioTrackPositionTracker.setAudioTrack( audioTrack, /* isPassthrough= */ configuration.outputMode == OUTPUT_MODE_PASSTHROUGH, @@ -728,8 +699,17 @@ public boolean handleBuffer( } if (!isAudioTrackInitialized()) { - initializeAudioTrack(); + try { + initializeAudioTrack(); + } catch (InitializationException e) { + if (e.isRecoverable) { + throw e; // Do not delay the exception if it can be recovered at higher level. + } + initializationExceptionPendingExceptionHolder.throwExceptionIfDeadlineIsReached(e); + return false; + } } + initializationExceptionPendingExceptionHolder.clear(); if (startMediaTimeUsNeedsInit) { startMediaTimeUs = max(0, presentationTimeUs); @@ -845,6 +825,9 @@ private AudioTrack buildAudioTrack() throws InitializationException { .buildAudioTrack(tunneling, audioAttributes, audioSessionId); } catch (InitializationException e) { maybeDisableOffload(); + if (listener != null) { + listener.onAudioSinkError(e); + } throw e; } } @@ -912,36 +895,49 @@ private void writeBuffer(ByteBuffer buffer, long avSyncPresentationTimeUs) throw } } int bytesRemaining = buffer.remaining(); - int bytesWritten = 0; + int bytesWrittenOrError = 0; // Error if negative if (Util.SDK_INT < 21) { // outputMode == OUTPUT_MODE_PCM. // Work out how many bytes we can write without the risk of blocking. int bytesToWrite = audioTrackPositionTracker.getAvailableBufferSize(writtenPcmBytes); if (bytesToWrite > 0) { bytesToWrite = min(bytesRemaining, bytesToWrite); - bytesWritten = audioTrack.write(preV21OutputBuffer, preV21OutputBufferOffset, bytesToWrite); - if (bytesWritten > 0) { - preV21OutputBufferOffset += bytesWritten; - buffer.position(buffer.position() + bytesWritten); + bytesWrittenOrError = + audioTrack.write(preV21OutputBuffer, preV21OutputBufferOffset, bytesToWrite); + if (bytesWrittenOrError > 0) { // No error + preV21OutputBufferOffset += bytesWrittenOrError; + buffer.position(buffer.position() + bytesWrittenOrError); } } } else if (tunneling) { Assertions.checkState(avSyncPresentationTimeUs != C.TIME_UNSET); - bytesWritten = + bytesWrittenOrError = writeNonBlockingWithAvSyncV21( audioTrack, buffer, bytesRemaining, avSyncPresentationTimeUs); } else { - bytesWritten = writeNonBlockingV21(audioTrack, buffer, bytesRemaining); + bytesWrittenOrError = writeNonBlockingV21(audioTrack, buffer, bytesRemaining); } lastFeedElapsedRealtimeMs = SystemClock.elapsedRealtime(); - if (bytesWritten < 0) { - boolean isRecoverable = isAudioTrackDeadObject(bytesWritten); + if (bytesWrittenOrError < 0) { + int error = bytesWrittenOrError; + boolean isRecoverable = isAudioTrackDeadObject(error); if (isRecoverable) { maybeDisableOffload(); } - throw new WriteException(bytesWritten); + WriteException e = new WriteException(error, configuration.inputFormat, isRecoverable); + if (listener != null) { + listener.onAudioSinkError(e); + } + if (e.isRecoverable) { + throw e; // Do not delay the exception if it can be recovered at higher level. + } + writeExceptionPendingExceptionHolder.throwExceptionIfDeadlineIsReached(e); + return; } + writeExceptionPendingExceptionHolder.clear(); + + int bytesWritten = bytesWrittenOrError; if (isOffloadedPlayback(audioTrack)) { // After calling AudioTrack.setOffloadEndOfStream, the AudioTrack internally stops and @@ -998,7 +994,8 @@ private void maybeDisableOffload() { } private static boolean isAudioTrackDeadObject(int status) { - return Util.SDK_INT >= 24 && status == AudioTrack.ERROR_DEAD_OBJECT; + return (Util.SDK_INT >= 24 && status == AudioTrack.ERROR_DEAD_OBJECT) + || status == ERROR_NATIVE_DEAD_OBJECT; } private boolean drainToEndOfStream() throws WriteException { @@ -1085,13 +1082,13 @@ public void setAudioAttributes(AudioAttributes audioAttributes) { return; } flush(); - audioSessionId = C.AUDIO_SESSION_ID_UNSET; } @Override public void setAudioSessionId(int audioSessionId) { if (this.audioSessionId != audioSessionId) { this.audioSessionId = audioSessionId; + externalAudioSessionIdProvided = audioSessionId != C.AUDIO_SESSION_ID_UNSET; flush(); } } @@ -1115,11 +1112,11 @@ public void setAuxEffectInfo(AuxEffectInfo auxEffectInfo) { } @Override - public void enableTunnelingV21(int tunnelingAudioSessionId) { + public void enableTunnelingV21() { Assertions.checkState(Util.SDK_INT >= 21); - if (!tunneling || audioSessionId != tunnelingAudioSessionId) { + Assertions.checkState(externalAudioSessionIdProvided); + if (!tunneling) { tunneling = true; - audioSessionId = tunnelingAudioSessionId; flush(); } } @@ -1128,7 +1125,6 @@ public void enableTunnelingV21(int tunnelingAudioSessionId) { public void disableTunneling() { if (tunneling) { tunneling = false; - audioSessionId = C.AUDIO_SESSION_ID_UNSET; flush(); } } @@ -1173,6 +1169,14 @@ public void flush() { // AudioTrack.release can take some time, so we call it on a background thread. final AudioTrack toRelease = audioTrack; audioTrack = null; + if (Util.SDK_INT < 21 && !externalAudioSessionIdProvided) { + // Prior to API level 21, audio sessions are not kept alive once there are no components + // associated with them. If we generated the session ID internally, the only component + // associated with the session is the audio track that's being released, and therefore + // the session will not be kept alive. As a result, we need to generate a new session when + // we next create an audio track. + audioSessionId = C.AUDIO_SESSION_ID_UNSET; + } if (pendingConfiguration != null) { configuration = pendingConfiguration; pendingConfiguration = null; @@ -1191,6 +1195,8 @@ public void run() { } }.start(); } + writeExceptionPendingExceptionHolder.clear(); + initializationExceptionPendingExceptionHolder.clear(); } @Override @@ -1202,6 +1208,9 @@ public void experimentalFlushWithoutAudioTrackRelease() { return; } + writeExceptionPendingExceptionHolder.clear(); + initializationExceptionPendingExceptionHolder.clear(); + if (!isAudioTrackInitialized()) { return; } @@ -1226,14 +1235,12 @@ public void experimentalFlushWithoutAudioTrackRelease() { @Override public void reset() { flush(); - releaseKeepSessionIdAudioTrack(); for (AudioProcessor audioProcessor : toIntPcmAvailableAudioProcessors) { audioProcessor.reset(); } for (AudioProcessor audioProcessor : toFloatPcmAvailableAudioProcessors) { audioProcessor.reset(); } - audioSessionId = C.AUDIO_SESSION_ID_UNSET; playing = false; offloadDisabledUntilNextConfiguration = false; } @@ -1268,23 +1275,6 @@ private void resetSinkStateForFlush() { flushAudioProcessors(); } - /** Releases {@link #keepSessionIdAudioTrack} asynchronously, if it is non-{@code null}. */ - private void releaseKeepSessionIdAudioTrack() { - if (keepSessionIdAudioTrack == null) { - return; - } - - // AudioTrack.release can take some time, so we call it on a background thread. - final AudioTrack toRelease = keepSessionIdAudioTrack; - keepSessionIdAudioTrack = null; - new Thread() { - @Override - public void run() { - toRelease.release(); - } - }.start(); - } - @RequiresApi(23) private void setAudioTrackPlaybackParametersV23(PlaybackParameters audioTrackPlaybackParameters) { if (isAudioTrackInitialized()) { @@ -1467,28 +1457,69 @@ private static Pair getEncodingAndChannelConfigForPassthrough( if (!supportedEncoding) { return null; } - - // E-AC3 JOC is object based, so any channel count specified in the format is arbitrary. Use 6, - // since the E-AC3 compatible part of the stream is 5.1. - int channelCount = encoding == C.ENCODING_E_AC3_JOC ? 6 : format.channelCount; - if (channelCount > audioCapabilities.getMaxChannelCount()) { + if (encoding == C.ENCODING_E_AC3_JOC + && !audioCapabilities.supportsEncoding(C.ENCODING_E_AC3_JOC)) { + // E-AC3 receivers support E-AC3 JOC streams (but decode only the base layer). + encoding = C.ENCODING_E_AC3; + } + if (!audioCapabilities.supportsEncoding(encoding)) { return null; } + int channelCount; + if (encoding == C.ENCODING_E_AC3_JOC) { + // E-AC3 JOC is object based so the format channel count is arbitrary. From API 29 we can get + // the channel count for this encoding, but before then there is no way to query it so we + // assume 6 channel audio is supported. + if (Util.SDK_INT >= 29) { + channelCount = + getMaxSupportedChannelCountForPassthroughV29(C.ENCODING_E_AC3_JOC, format.sampleRate); + if (channelCount == 0) { + Log.w(TAG, "E-AC3 JOC encoding supported but no channel count supported"); + return null; + } + } else { + channelCount = 6; + } + } else { + channelCount = format.channelCount; + if (channelCount > audioCapabilities.getMaxChannelCount()) { + return null; + } + } int channelConfig = getChannelConfigForPassthrough(channelCount); if (channelConfig == AudioFormat.CHANNEL_INVALID) { return null; } - if (audioCapabilities.supportsEncoding(encoding)) { - return Pair.create(encoding, channelConfig); - } else if (encoding == C.ENCODING_E_AC3_JOC - && audioCapabilities.supportsEncoding(C.ENCODING_E_AC3)) { - // E-AC3 receivers support E-AC3 JOC streams (but decode in 2-D rather than 3-D). - return Pair.create(C.ENCODING_E_AC3, channelConfig); - } + return Pair.create(encoding, channelConfig); + } - return null; + /** + * Returns the maximum number of channels supported for passthrough playback of audio in the given + * format, or 0 if the format is unsupported. + */ + @RequiresApi(29) + private static int getMaxSupportedChannelCountForPassthroughV29( + @C.Encoding int encoding, int sampleRate) { + android.media.AudioAttributes audioAttributes = + new android.media.AudioAttributes.Builder() + .setUsage(android.media.AudioAttributes.USAGE_MEDIA) + .setContentType(android.media.AudioAttributes.CONTENT_TYPE_MOVIE) + .build(); + // TODO(internal b/25994457): Query supported channel masks directly once it's supported. + for (int channelCount = 8; channelCount > 0; channelCount--) { + AudioFormat audioFormat = + new AudioFormat.Builder() + .setEncoding(encoding) + .setSampleRate(sampleRate) + .setChannelMask(Util.getAudioTrackChannelConfig(channelCount)) + .build(); + if (AudioTrack.isDirectPlaybackSupported(audioFormat, audioAttributes)) { + return channelCount; + } + } + return 0; } private static int getChannelConfigForPassthrough(int channelCount) { @@ -1552,21 +1583,6 @@ private static boolean isOffloadedGaplessPlaybackSupported() { return Util.SDK_INT >= 30 && Util.MODEL.startsWith("Pixel"); } - private static AudioTrack initializeKeepSessionIdAudioTrack(int audioSessionId) { - int sampleRate = 4000; // Equal to private AudioTrack.MIN_SAMPLE_RATE. - int channelConfig = AudioFormat.CHANNEL_OUT_MONO; - @C.PcmEncoding int encoding = C.ENCODING_PCM_16BIT; - int bufferSize = 2; // Use a two byte buffer, as it is not actually used for playback. - return new AudioTrack( - C.STREAM_TYPE_DEFAULT, - sampleRate, - channelConfig, - encoding, - bufferSize, - MODE_STATIC, - audioSessionId); - } - private static int getMaximumEncodedRateBytesPerSecond(@C.Encoding int encoding) { switch (encoding) { case C.ENCODING_MP3: @@ -1657,7 +1673,7 @@ private static int getFramesPerEncodedSample(@C.Encoding int encoding, ByteBuffe @RequiresApi(21) private static int writeNonBlockingV21(AudioTrack audioTrack, ByteBuffer buffer, int size) { - return audioTrack.write(buffer, size, WRITE_NON_BLOCKING); + return audioTrack.write(buffer, size, AudioTrack.WRITE_NON_BLOCKING); } @RequiresApi(21) @@ -1665,7 +1681,8 @@ private int writeNonBlockingWithAvSyncV21( AudioTrack audioTrack, ByteBuffer buffer, int size, long presentationTimeUs) { if (Util.SDK_INT >= 26) { // The underlying platform AudioTrack writes AV sync headers directly. - return audioTrack.write(buffer, size, WRITE_NON_BLOCKING, presentationTimeUs * 1000); + return audioTrack.write( + buffer, size, AudioTrack.WRITE_NON_BLOCKING, presentationTimeUs * 1000); } if (avSyncHeader == null) { avSyncHeader = ByteBuffer.allocate(16); @@ -1680,7 +1697,8 @@ private int writeNonBlockingWithAvSyncV21( } int avSyncHeaderBytesRemaining = avSyncHeader.remaining(); if (avSyncHeaderBytesRemaining > 0) { - int result = audioTrack.write(avSyncHeader, avSyncHeaderBytesRemaining, WRITE_NON_BLOCKING); + int result = + audioTrack.write(avSyncHeader, avSyncHeaderBytesRemaining, AudioTrack.WRITE_NON_BLOCKING); if (result < 0) { bytesUntilNextAvSync = 0; return result; @@ -1730,18 +1748,22 @@ public StreamEventCallbackV29() { new AudioTrack.StreamEventCallback() { @Override public void onDataRequest(AudioTrack track, int size) { - Assertions.checkState(track == DefaultAudioSink.this.audioTrack); - if (listener != null) { + Assertions.checkState(track == audioTrack); + if (listener != null && playing) { + // Do not signal that the buffer is emptying if not playing as it is a transient + // state. listener.onOffloadBufferEmptying(); } } @Override public void onTearDown(@NonNull AudioTrack track) { + Assertions.checkState(track == audioTrack); if (listener != null && playing) { - // A new Audio Track needs to be created and it's buffer filled, which will be done - // on the next handleBuffer call. Request this call explicitly in case ExoPlayer is - // sleeping waiting for a data request. + // The audio track was destroyed while in use. Thus a new AudioTrack needs to be + // created and its buffer filled, which will be done on the next handleBuffer call. + // Request this call explicitly in case ExoPlayer is sleeping waiting for a data + // request. listener.onOffloadBufferEmptying(); } } @@ -1931,20 +1953,33 @@ public AudioTrack buildAudioTrack( AudioTrack audioTrack; try { audioTrack = createAudioTrack(tunneling, audioAttributes, audioSessionId); - } catch (UnsupportedOperationException e) { + } catch (UnsupportedOperationException | IllegalArgumentException e) { throw new InitializationException( - AudioTrack.STATE_UNINITIALIZED, outputSampleRate, outputChannelConfig, bufferSize); + AudioTrack.STATE_UNINITIALIZED, + outputSampleRate, + outputChannelConfig, + bufferSize, + inputFormat, + /* isRecoverable= */ outputModeIsOffload(), + e); } int state = audioTrack.getState(); - if (state != STATE_INITIALIZED) { + if (state != AudioTrack.STATE_INITIALIZED) { try { audioTrack.release(); } catch (Exception e) { // The track has already failed to initialize, so it wouldn't be that surprising if // release were to fail too. Swallow the exception. } - throw new InitializationException(state, outputSampleRate, outputChannelConfig, bufferSize); + throw new InitializationException( + state, + outputSampleRate, + outputChannelConfig, + bufferSize, + inputFormat, + /* isRecoverable= */ outputModeIsOffload(), + /* audioTrackException= */ null); } return audioTrack; } @@ -1984,7 +2019,7 @@ private AudioTrack createAudioTrackV21( getAudioTrackAttributesV21(audioAttributes, tunneling), getAudioFormat(outputSampleRate, outputChannelConfig, outputEncoding), bufferSize, - MODE_STREAM, + AudioTrack.MODE_STREAM, audioSessionId); } @@ -1997,7 +2032,7 @@ private AudioTrack createAudioTrackV9(AudioAttributes audioAttributes, int audio outputChannelConfig, outputEncoding, bufferSize, - MODE_STREAM); + AudioTrack.MODE_STREAM); } else { // Re-attach to the same audio session. return new AudioTrack( @@ -2006,7 +2041,7 @@ private AudioTrack createAudioTrackV9(AudioAttributes audioAttributes, int audio outputChannelConfig, outputEncoding, bufferSize, - MODE_STREAM, + AudioTrack.MODE_STREAM, audioSessionId); } } @@ -2040,7 +2075,7 @@ private int getEncodedDefaultBufferSize(long bufferDurationUs) { private int getPcmDefaultBufferSize(float maxAudioTrackPlaybackSpeed) { int minBufferSize = AudioTrack.getMinBufferSize(outputSampleRate, outputChannelConfig, outputEncoding); - Assertions.checkState(minBufferSize != ERROR_BAD_VALUE); + Assertions.checkState(minBufferSize != AudioTrack.ERROR_BAD_VALUE); int multipliedBufferSize = minBufferSize * BUFFER_MULTIPLICATION_FACTOR; int minAppBufferSize = (int) durationUsToFrames(MIN_BUFFER_DURATION_US) * outputPcmFrameSize; int maxAppBufferSize = @@ -2077,4 +2112,37 @@ public boolean outputModeIsOffload() { return outputMode == OUTPUT_MODE_OFFLOAD; } } + + private static final class PendingExceptionHolder { + + private final long throwDelayMs; + + @Nullable private T pendingException; + private long throwDeadlineMs; + + public PendingExceptionHolder(long throwDelayMs) { + this.throwDelayMs = throwDelayMs; + } + + public void throwExceptionIfDeadlineIsReached(T exception) throws T { + long nowMs = SystemClock.elapsedRealtime(); + if (pendingException == null) { + pendingException = exception; + throwDeadlineMs = nowMs + throwDelayMs; + } + if (nowMs >= throwDeadlineMs) { + if (pendingException != exception) { + // All retry exception are probably the same, thus only save the last one to save memory. + pendingException.addSuppressed(exception); + } + T pendingException = this.pendingException; + clear(); + throw pendingException; + } + } + + public void clear() { + pendingException = null; + } + } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/ForwardingAudioSink.java b/library/core/src/main/java/com/google/android/exoplayer2/audio/ForwardingAudioSink.java index 7460d124576..1e60dc3ed1d 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/audio/ForwardingAudioSink.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/audio/ForwardingAudioSink.java @@ -124,8 +124,8 @@ public void setAuxEffectInfo(AuxEffectInfo auxEffectInfo) { } @Override - public void enableTunnelingV21(int tunnelingAudioSessionId) { - sink.enableTunnelingV21(tunnelingAudioSessionId); + public void enableTunnelingV21() { + sink.enableTunnelingV21(); } @Override diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/MediaCodecAudioRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/audio/MediaCodecAudioRenderer.java index 07c3541552b..5786af503db 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/audio/MediaCodecAudioRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/audio/MediaCodecAudioRenderer.java @@ -15,6 +15,8 @@ */ package com.google.android.exoplayer2.audio; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_MAX_INPUT_SIZE_EXCEEDED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_NO; import static com.google.android.exoplayer2.util.Assertions.checkNotNull; import static java.lang.Math.max; @@ -24,7 +26,6 @@ import android.media.MediaCodec; import android.media.MediaCrypto; import android.media.MediaFormat; -import android.media.audiofx.Virtualizer; import android.os.Handler; import androidx.annotation.CallSuper; import androidx.annotation.Nullable; @@ -37,7 +38,11 @@ import com.google.android.exoplayer2.PlayerMessage.Target; import com.google.android.exoplayer2.RendererCapabilities; import com.google.android.exoplayer2.audio.AudioRendererEventListener.EventDispatcher; +import com.google.android.exoplayer2.audio.AudioSink.InitializationException; +import com.google.android.exoplayer2.audio.AudioSink.WriteException; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DecoderDiscardReasons; import com.google.android.exoplayer2.mediacodec.MediaCodecAdapter; import com.google.android.exoplayer2.mediacodec.MediaCodecInfo; import com.google.android.exoplayer2.mediacodec.MediaCodecRenderer; @@ -90,7 +95,6 @@ public class MediaCodecAudioRenderer extends MediaCodecRenderer implements Media private int codecMaxInputSize; private boolean codecNeedsDiscardChannelsWorkaround; - private boolean codecNeedsEosBufferTimestampWorkaround; /** Codec used for DRM decryption only in passthrough and offload. */ @Nullable private Format decryptOnlyCodecFormat; @@ -168,6 +172,7 @@ public MediaCodecAudioRenderer( AudioSink audioSink) { this( context, + MediaCodecAdapter.Factory.DEFAULT, mediaCodecSelector, /* enableDecoderFallback= */ false, eventHandler, @@ -193,8 +198,42 @@ public MediaCodecAudioRenderer( @Nullable Handler eventHandler, @Nullable AudioRendererEventListener eventListener, AudioSink audioSink) { + this( + context, + MediaCodecAdapter.Factory.DEFAULT, + mediaCodecSelector, + enableDecoderFallback, + eventHandler, + eventListener, + audioSink); + } + + /** + * Creates a new instance. + * + * @param context A context. + * @param codecAdapterFactory The {@link MediaCodecAdapter.Factory} used to create {@link + * MediaCodecAdapter} instances. + * @param mediaCodecSelector A decoder selector. + * @param enableDecoderFallback Whether to enable fallback to lower-priority decoders if decoder + * initialization fails. This may result in using a decoder that is slower/less efficient than + * the primary decoder. + * @param eventHandler A handler to use when delivering events to {@code eventListener}. May be + * null if delivery of events is not required. + * @param eventListener A listener of events. May be null if delivery of events is not required. + * @param audioSink The sink to which audio will be output. + */ + public MediaCodecAudioRenderer( + Context context, + MediaCodecAdapter.Factory codecAdapterFactory, + MediaCodecSelector mediaCodecSelector, + boolean enableDecoderFallback, + @Nullable Handler eventHandler, + @Nullable AudioRendererEventListener eventListener, + AudioSink audioSink) { super( C.TRACK_TYPE_AUDIO, + codecAdapterFactory, mediaCodecSelector, enableDecoderFallback, /* assumedMinimumCodecOperatingRate= */ 44100); @@ -227,7 +266,7 @@ public void experimentalSetEnableKeepAudioTrackOnSeek(boolean enableKeepAudioTra protected int supportsFormat(MediaCodecSelector mediaCodecSelector, Format format) throws DecoderQueryException { if (!MimeTypes.isAudio(format.sampleMimeType)) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } @TunnelingSupport int tunnelingSupport = Util.SDK_INT >= 21 ? TUNNELING_SUPPORTED : TUNNELING_NOT_SUPPORTED; @@ -238,25 +277,25 @@ protected int supportsFormat(MediaCodecSelector mediaCodecSelector, Format forma if (supportsFormatDrm && audioSink.supportsFormat(format) && (!formatHasDrm || MediaCodecUtil.getDecryptOnlyDecoderInfo() != null)) { - return RendererCapabilities.create(FORMAT_HANDLED, ADAPTIVE_NOT_SEAMLESS, tunnelingSupport); + return RendererCapabilities.create(C.FORMAT_HANDLED, ADAPTIVE_NOT_SEAMLESS, tunnelingSupport); } // If the input is PCM then it will be passed directly to the sink. Hence the sink must support // the input format directly. if (MimeTypes.AUDIO_RAW.equals(format.sampleMimeType) && !audioSink.supportsFormat(format)) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_SUBTYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_SUBTYPE); } // For all other input formats, we expect the decoder to output 16-bit PCM. if (!audioSink.supportsFormat( Util.getPcmFormat(C.ENCODING_PCM_16BIT, format.channelCount, format.sampleRate))) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_SUBTYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_SUBTYPE); } List decoderInfos = getDecoderInfos(mediaCodecSelector, format, /* requiresSecureDecoder= */ false); if (decoderInfos.isEmpty()) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_SUBTYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_SUBTYPE); } if (!supportsFormatDrm) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_DRM); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_DRM); } // Check capabilities for the first decoder in the list, which takes priority. MediaCodecInfo decoderInfo = decoderInfos.get(0); @@ -266,8 +305,8 @@ protected int supportsFormat(MediaCodecSelector mediaCodecSelector, Format forma isFormatSupported && decoderInfo.isSeamlessAdaptationSupported(format) ? ADAPTIVE_SEAMLESS : ADAPTIVE_NOT_SEAMLESS; - @FormatSupport - int formatSupport = isFormatSupported ? FORMAT_HANDLED : FORMAT_EXCEEDS_CAPABILITIES; + @C.FormatSupport + int formatSupport = isFormatSupported ? C.FORMAT_HANDLED : C.FORMAT_EXCEEDS_CAPABILITIES; return RendererCapabilities.create(formatSupport, adaptiveSupport, tunnelingSupport); } @@ -309,16 +348,15 @@ protected boolean shouldUseBypass(Format format) { @Override protected void configureCodec( MediaCodecInfo codecInfo, - MediaCodecAdapter codecAdapter, + MediaCodecAdapter codec, Format format, @Nullable MediaCrypto crypto, float codecOperatingRate) { codecMaxInputSize = getCodecMaxInputSize(codecInfo, format, getStreamFormats()); codecNeedsDiscardChannelsWorkaround = codecNeedsDiscardChannelsWorkaround(codecInfo.name); - codecNeedsEosBufferTimestampWorkaround = codecNeedsEosBufferTimestampWorkaround(codecInfo.name); MediaFormat mediaFormat = getMediaFormat(format, codecInfo.codecMimeType, codecMaxInputSize, codecOperatingRate); - codecAdapter.configure(mediaFormat, /* surface= */ null, crypto, /* flags= */ 0); + codec.configure(mediaFormat, /* surface= */ null, crypto, /* flags= */ 0); // Store the input MIME type if we're only using the codec for decryption. boolean decryptOnlyCodecEnabled = MimeTypes.AUDIO_RAW.equals(codecInfo.mimeType) @@ -327,40 +365,21 @@ protected void configureCodec( } @Override - protected @KeepCodecResult int canKeepCodec( - MediaCodec codec, MediaCodecInfo codecInfo, Format oldFormat, Format newFormat) { + protected DecoderReuseEvaluation canReuseCodec( + MediaCodecInfo codecInfo, Format oldFormat, Format newFormat) { + DecoderReuseEvaluation evaluation = codecInfo.canReuseCodec(oldFormat, newFormat); + + @DecoderDiscardReasons int discardReasons = evaluation.discardReasons; if (getCodecMaxInputSize(codecInfo, newFormat) > codecMaxInputSize) { - return KEEP_CODEC_RESULT_NO; - } else if (codecInfo.isSeamlessAdaptationSupported( - oldFormat, newFormat, /* isNewFormatComplete= */ true)) { - return KEEP_CODEC_RESULT_YES_WITHOUT_RECONFIGURATION; - } else if (canKeepCodecWithFlush(oldFormat, newFormat)) { - return KEEP_CODEC_RESULT_YES_WITH_FLUSH; - } else { - return KEEP_CODEC_RESULT_NO; + discardReasons |= DISCARD_REASON_MAX_INPUT_SIZE_EXCEEDED; } - } - /** - * Returns whether the codec can be flushed and reused when switching to a new format. Reuse is - * generally possible when the codec would be configured in an identical way after the format - * change (excluding {@link MediaFormat#KEY_MAX_INPUT_SIZE} and configuration that does not come - * from the {@link Format}). - * - * @param oldFormat The first format. - * @param newFormat The second format. - * @return Whether the codec can be flushed and reused when switching to a new format. - */ - protected boolean canKeepCodecWithFlush(Format oldFormat, Format newFormat) { - // Flush and reuse the codec if the audio format and initialization data matches. For Opus, we - // don't flush and reuse the codec because the decoder may discard samples after flushing, which - // would result in audio being dropped just after a stream change (see [Internal: b/143450854]). - return Util.areEqual(oldFormat.sampleMimeType, newFormat.sampleMimeType) - && oldFormat.channelCount == newFormat.channelCount - && oldFormat.sampleRate == newFormat.sampleRate - && oldFormat.pcmEncoding == newFormat.pcmEncoding - && oldFormat.initializationDataEquals(newFormat) - && !MimeTypes.AUDIO_OPUS.equals(oldFormat.sampleMimeType); + return new DecoderReuseEvaluation( + codecInfo.name, + oldFormat, + newFormat, + discardReasons != 0 ? REUSE_RESULT_NO : evaluation.result, + discardReasons); } @Override @@ -371,7 +390,7 @@ public MediaClock getMediaClock() { @Override protected float getCodecOperatingRateV23( - float operatingRate, Format format, Format[] streamFormats) { + float targetPlaybackSpeed, Format format, Format[] streamFormats) { // Use the highest known stream sample-rate up front, to avoid having to reconfigure the codec // should an adaptive switch to that stream occur. int maxSampleRate = -1; @@ -381,7 +400,7 @@ protected float getCodecOperatingRateV23( maxSampleRate = max(maxSampleRate, streamSampleRate); } } - return maxSampleRate == -1 ? CODEC_OPERATING_RATE_UNSET : (maxSampleRate * operatingRate); + return maxSampleRate == -1 ? CODEC_OPERATING_RATE_UNSET : (maxSampleRate * targetPlaybackSpeed); } @Override @@ -391,9 +410,17 @@ protected void onCodecInitialized( } @Override - protected void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybackException { - super.onInputFormatChanged(formatHolder); - eventDispatcher.inputFormatChanged(formatHolder.format); + protected void onCodecReleased(String name) { + eventDispatcher.decoderReleased(name); + } + + @Override + @Nullable + protected DecoderReuseEvaluation onInputFormatChanged(FormatHolder formatHolder) + throws ExoPlaybackException { + @Nullable DecoderReuseEvaluation evaluation = super.onInputFormatChanged(formatHolder); + eventDispatcher.inputFormatChanged(formatHolder.format, evaluation); + return evaluation; } @Override @@ -443,22 +470,10 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF try { audioSink.configure(audioSinkInputFormat, /* specifiedBufferSize= */ 0, channelMap); } catch (AudioSink.ConfigurationException e) { - throw createRendererException(e, format); + throw createRendererException(e, e.format); } } - /** - * Called when the audio session id becomes known. The default implementation is a no-op. One - * reason for overriding this method would be to instantiate and enable a {@link Virtualizer} in - * order to spatialize the audio channels. For this use case, any {@link Virtualizer} instances - * should be released in {@link #onDisabled()} (if not before). - * - *

    See {@link AudioSink.Listener#onAudioSessionId(int)}. - */ - protected void onAudioSessionId(int audioSessionId) { - // Do nothing. - } - /** See {@link AudioSink.Listener#onPositionDiscontinuity()}. */ @CallSuper protected void onPositionDiscontinuity() { @@ -471,9 +486,8 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) throws ExoPlaybackException { super.onEnabled(joining, mayRenderStartOfStream); eventDispatcher.enabled(decoderCounters); - int tunnelingAudioSessionId = getConfiguration().tunnelingAudioSessionId; - if (tunnelingAudioSessionId != C.AUDIO_SESSION_ID_UNSET) { - audioSink.enableTunnelingV21(tunnelingAudioSessionId); + if (getConfiguration().tunneling) { + audioSink.enableTunnelingV21(); } else { audioSink.disableTunneling(); } @@ -583,7 +597,7 @@ protected void onProcessedStreamChange() { protected boolean processOutputBuffer( long positionUs, long elapsedRealtimeUs, - @Nullable MediaCodec codec, + @Nullable MediaCodecAdapter codec, @Nullable ByteBuffer buffer, int bufferIndex, int bufferFlags, @@ -594,13 +608,6 @@ protected boolean processOutputBuffer( Format format) throws ExoPlaybackException { checkNotNull(buffer); - if (codec != null - && codecNeedsEosBufferTimestampWorkaround - && bufferPresentationTimeUs == 0 - && (bufferFlags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0 - && getLargestQueuedPresentationTimeUs() != C.TIME_UNSET) { - bufferPresentationTimeUs = getLargestQueuedPresentationTimeUs(); - } if (decryptOnlyCodecFormat != null && (bufferFlags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) { @@ -621,8 +628,10 @@ && getLargestQueuedPresentationTimeUs() != C.TIME_UNSET) { boolean fullyConsumed; try { fullyConsumed = audioSink.handleBuffer(buffer, bufferPresentationTimeUs, sampleCount); - } catch (AudioSink.InitializationException | AudioSink.WriteException e) { - throw createRendererException(e, format); + } catch (InitializationException e) { + throw createRendererException(e, e.format, e.isRecoverable); + } catch (WriteException e) { + throw createRendererException(e, format, e.isRecoverable); } if (fullyConsumed) { @@ -641,8 +650,7 @@ protected void renderToEndOfStream() throws ExoPlaybackException { try { audioSink.playToEndOfStream(); } catch (AudioSink.WriteException e) { - @Nullable Format outputFormat = getOutputFormat(); - throw createRendererException(e, outputFormat != null ? outputFormat : getInputFormat()); + throw createRendererException(e, e.format, e.isRecoverable); } } @@ -693,8 +701,7 @@ protected int getCodecMaxInputSize( return maxInputSize; } for (Format streamFormat : streamFormats) { - if (codecInfo.isSeamlessAdaptationSupported( - format, streamFormat, /* isNewFormatComplete= */ false)) { + if (codecInfo.canReuseCodec(format, streamFormat).result != REUSE_RESULT_NO) { maxInputSize = max(maxInputSize, getCodecMaxInputSize(codecInfo, streamFormat)); } } @@ -803,32 +810,8 @@ private static boolean codecNeedsDiscardChannelsWorkaround(String codecName) { || Util.DEVICE.startsWith("heroqlte")); } - /** - * Returns whether the decoder may output a non-empty buffer with timestamp 0 as the end of stream - * buffer. - * - *

    See GitHub issue #5045. - */ - private static boolean codecNeedsEosBufferTimestampWorkaround(String codecName) { - return Util.SDK_INT < 21 - && "OMX.SEC.mp3.dec".equals(codecName) - && "samsung".equals(Util.MANUFACTURER) - && (Util.DEVICE.startsWith("baffin") - || Util.DEVICE.startsWith("grand") - || Util.DEVICE.startsWith("fortuna") - || Util.DEVICE.startsWith("gprimelte") - || Util.DEVICE.startsWith("j2y18lte") - || Util.DEVICE.startsWith("ms01")); - } - private final class AudioSinkListener implements AudioSink.Listener { - @Override - public void onAudioSessionId(int audioSessionId) { - eventDispatcher.audioSessionId(audioSessionId); - MediaCodecAudioRenderer.this.onAudioSessionId(audioSessionId); - } - @Override public void onPositionDiscontinuity() { MediaCodecAudioRenderer.this.onPositionDiscontinuity(); @@ -862,5 +845,10 @@ public void onOffloadBufferFull(long bufferEmptyingDeadlineMs) { wakeupListener.onSleep(bufferEmptyingDeadlineMs); } } + + @Override + public void onAudioSinkError(Exception audioSinkError) { + eventDispatcher.audioSinkError(audioSinkError); + } } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/audio/SonicAudioProcessor.java b/library/core/src/main/java/com/google/android/exoplayer2/audio/SonicAudioProcessor.java index f8597619547..bbfd77469d4 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/audio/SonicAudioProcessor.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/audio/SonicAudioProcessor.java @@ -34,7 +34,7 @@ public final class SonicAudioProcessor implements AudioProcessor { public static final int SAMPLE_RATE_NO_CHANGE = -1; /** The threshold below which the difference between two pitch/speed factors is negligible. */ - private static final float CLOSE_THRESHOLD = 0.01f; + private static final float CLOSE_THRESHOLD = 0.0001f; /** * The minimum number of output bytes required for duration scaling to be calculated using the @@ -79,7 +79,7 @@ public SonicAudioProcessor() { * processor. The value returned by {@link #isActive()} may change, and the processor must be * {@link #flush() flushed} before queueing more data. * - * @param speed The target playback speed. + * @param speed The target factor by which playback should be sped up. */ public void setSpeed(float speed) { if (this.speed != speed) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/decoder/DecoderReuseEvaluation.java b/library/core/src/main/java/com/google/android/exoplayer2/decoder/DecoderReuseEvaluation.java new file mode 100644 index 00000000000..a23c92ec65a --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/decoder/DecoderReuseEvaluation.java @@ -0,0 +1,178 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.decoder; + +import static com.google.android.exoplayer2.util.Assertions.checkArgument; +import static com.google.android.exoplayer2.util.Assertions.checkNotEmpty; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; + +import androidx.annotation.IntDef; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.video.ColorInfo; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; + +/** + * The result of an evaluation to determine whether a decoder can be reused for a new input format. + */ +public final class DecoderReuseEvaluation { + + /** Possible outcomes of the evaluation. */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + REUSE_RESULT_NO, + REUSE_RESULT_YES_WITH_FLUSH, + REUSE_RESULT_YES_WITH_RECONFIGURATION, + REUSE_RESULT_YES_WITHOUT_RECONFIGURATION + }) + public @interface DecoderReuseResult {} + /** The decoder cannot be reused. */ + public static final int REUSE_RESULT_NO = 0; + /** The decoder can be reused, but must be flushed. */ + public static final int REUSE_RESULT_YES_WITH_FLUSH = 1; + /** + * The decoder can be reused. It does not need to be flushed, but must be reconfigured by + * prefixing the next input buffer with the new format's configuration data. + */ + public static final int REUSE_RESULT_YES_WITH_RECONFIGURATION = 2; + /** The decoder can be kept. It does not need to be flushed and no reconfiguration is required. */ + public static final int REUSE_RESULT_YES_WITHOUT_RECONFIGURATION = 3; + + /** Possible reasons why reuse is not possible. */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef( + flag = true, + value = { + DISCARD_REASON_REUSE_NOT_IMPLEMENTED, + DISCARD_REASON_WORKAROUND, + DISCARD_REASON_APP_OVERRIDE, + DISCARD_REASON_MIME_TYPE_CHANGED, + DISCARD_REASON_OPERATING_RATE_CHANGED, + DISCARD_REASON_INITIALIZATION_DATA_CHANGED, + DISCARD_REASON_DRM_SESSION_CHANGED, + DISCARD_REASON_MAX_INPUT_SIZE_EXCEEDED, + DISCARD_REASON_VIDEO_MAX_RESOLUTION_EXCEEDED, + DISCARD_REASON_VIDEO_RESOLUTION_CHANGED, + DISCARD_REASON_VIDEO_ROTATION_CHANGED, + DISCARD_REASON_VIDEO_COLOR_INFO_CHANGED, + DISCARD_REASON_AUDIO_CHANNEL_COUNT_CHANGED, + DISCARD_REASON_AUDIO_SAMPLE_RATE_CHANGED, + DISCARD_REASON_AUDIO_ENCODING_CHANGED + }) + public @interface DecoderDiscardReasons {} + + /** Decoder reuse is not implemented. */ + public static final int DISCARD_REASON_REUSE_NOT_IMPLEMENTED = 1 << 0; + /** Decoder reuse is disabled by a workaround. */ + public static final int DISCARD_REASON_WORKAROUND = 1 << 1; + /** Decoder reuse is disabled by overriding behavior in application code. */ + public static final int DISCARD_REASON_APP_OVERRIDE = 1 << 2; + /** The sample MIME type is changing. */ + public static final int DISCARD_REASON_MIME_TYPE_CHANGED = 1 << 3; + /** The codec's operating rate is changing. */ + public static final int DISCARD_REASON_OPERATING_RATE_CHANGED = 1 << 4; + /** The format initialization data is changing. */ + public static final int DISCARD_REASON_INITIALIZATION_DATA_CHANGED = 1 << 5; + /** The new format may exceed the decoder's configured maximum sample size, in bytes. */ + public static final int DISCARD_REASON_MAX_INPUT_SIZE_EXCEEDED = 1 << 6; + /** The DRM session is changing. */ + public static final int DISCARD_REASON_DRM_SESSION_CHANGED = 1 << 7; + /** The new format may exceed the decoder's configured maximum resolution. */ + public static final int DISCARD_REASON_VIDEO_MAX_RESOLUTION_EXCEEDED = 1 << 8; + /** The video resolution is changing. */ + public static final int DISCARD_REASON_VIDEO_RESOLUTION_CHANGED = 1 << 9; + /** The video rotation is changing. */ + public static final int DISCARD_REASON_VIDEO_ROTATION_CHANGED = 1 << 10; + /** The video {@link ColorInfo} is changing. */ + public static final int DISCARD_REASON_VIDEO_COLOR_INFO_CHANGED = 1 << 11; + /** The audio channel count is changing. */ + public static final int DISCARD_REASON_AUDIO_CHANNEL_COUNT_CHANGED = 1 << 12; + /** The audio sample rate is changing. */ + public static final int DISCARD_REASON_AUDIO_SAMPLE_RATE_CHANGED = 1 << 13; + /** The audio encoding is changing. */ + public static final int DISCARD_REASON_AUDIO_ENCODING_CHANGED = 1 << 14; + + /** The name of the decoder. */ + public final String decoderName; + + /** The {@link Format} for which the decoder was previously configured. */ + public final Format oldFormat; + + /** The new {@link Format} being evaluated. */ + public final Format newFormat; + + /** The {@link DecoderReuseResult result} of the evaluation. */ + @DecoderReuseResult public final int result; + + /** + * {@link DecoderDiscardReasons Reasons} why the decoder cannot be reused. Always {@code 0} if + * reuse is possible. May also be {code 0} if reuse is not possible for an unspecified reason. + */ + @DecoderDiscardReasons public final int discardReasons; + + /** + * @param decoderName The name of the decoder. + * @param oldFormat The {@link Format} for which the decoder was previously configured. + * @param newFormat The new {@link Format} being evaluated. + * @param result The {@link DecoderReuseResult result} of the evaluation. + * @param discardReasons One or more {@link DecoderDiscardReasons reasons} why the decoder cannot + * be reused, or {@code 0} if reuse is possible. + */ + public DecoderReuseEvaluation( + String decoderName, + Format oldFormat, + Format newFormat, + @DecoderReuseResult int result, + @DecoderDiscardReasons int discardReasons) { + checkArgument(result == REUSE_RESULT_NO || discardReasons == 0); + this.decoderName = checkNotEmpty(decoderName); + this.oldFormat = checkNotNull(oldFormat); + this.newFormat = checkNotNull(newFormat); + this.result = result; + this.discardReasons = discardReasons; + } + + @Override + public boolean equals(@Nullable Object obj) { + if (this == obj) { + return true; + } + if (obj == null || getClass() != obj.getClass()) { + return false; + } + DecoderReuseEvaluation other = (DecoderReuseEvaluation) obj; + return result == other.result + && discardReasons == other.discardReasons + && decoderName.equals(other.decoderName) + && oldFormat.equals(other.oldFormat) + && newFormat.equals(other.newFormat); + } + + @Override + public int hashCode() { + int hashCode = 17; + hashCode = 31 * hashCode + result; + hashCode = 31 * hashCode + discardReasons; + hashCode = 31 * hashCode + decoderName.hashCode(); + hashCode = 31 * hashCode + oldFormat.hashCode(); + hashCode = 31 * hashCode + newFormat.hashCode(); + return hashCode; + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSession.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSession.java index bb3ad910f02..f7d7a097a04 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSession.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSession.java @@ -26,6 +26,7 @@ import android.os.Message; import android.os.SystemClock; import android.util.Pair; +import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import com.google.android.exoplayer2.C; @@ -256,6 +257,11 @@ public boolean playClearSamplesWithoutKeys() { return state == STATE_ERROR ? lastException : null; } + @Override + public final UUID getSchemeUuid() { + return uuid; + } + @Override public final @Nullable ExoMediaCrypto getMediaCrypto() { return mediaCrypto; @@ -303,7 +309,7 @@ public void release(@Nullable DrmSessionEventListener.EventDispatcher eventDispa // Assigning null to various non-null variables for clean-up. state = STATE_RELEASED; Util.castNonNull(responseHandler).removeCallbacksAndMessages(null); - Util.castNonNull(requestHandler).removeCallbacksAndMessages(null); + Util.castNonNull(requestHandler).release(); requestHandler = null; Util.castNonNull(requestHandlerThread).quit(); requestHandlerThread = null; @@ -565,6 +571,9 @@ public void handleMessage(Message msg) { @SuppressLint("HandlerLeak") private class RequestHandler extends Handler { + @GuardedBy("this") + private boolean isReleased; + public RequestHandler(Looper backgroundLooper) { super(backgroundLooper); } @@ -605,9 +614,13 @@ public void handleMessage(Message msg) { response = e; } loadErrorHandlingPolicy.onLoadTaskConcluded(requestTask.taskId); - responseHandler - .obtainMessage(msg.what, Pair.create(requestTask.request, response)) - .sendToTarget(); + synchronized (this) { + if (!isReleased) { + responseHandler + .obtainMessage(msg.what, Pair.create(requestTask.request, response)) + .sendToTarget(); + } + } } private boolean maybeRetryRequest(Message originalMsg, MediaDrmCallbackException exception) { @@ -642,8 +655,18 @@ private boolean maybeRetryRequest(Message originalMsg, MediaDrmCallbackException // The error is fatal. return false; } - sendMessageDelayed(Message.obtain(originalMsg), retryDelayMs); - return true; + synchronized (this) { + if (!isReleased) { + sendMessageDelayed(Message.obtain(originalMsg), retryDelayMs); + return true; + } + } + return false; + } + + public synchronized void release() { + removeCallbacksAndMessages(/* token= */ null); + isReleased = true; } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManager.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManager.java index be02faeba87..67cb095b8d9 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManager.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManager.java @@ -36,6 +36,7 @@ import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.Util; import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; import com.google.common.collect.Sets; import java.lang.annotation.Documented; import java.lang.annotation.Retention; @@ -632,12 +633,12 @@ private DefaultDrmSession createAndAcquireSessionWithRetry( // ResourceBusyException is only available at API 19, so on earlier versions we always // eagerly release regardless of the underlying error. if (!keepaliveSessions.isEmpty()) { - // Make a local copy, because sessions are removed from this.timingOutSessions during + // Make a local copy, because sessions are removed from this.keepaliveSessions during // release (via callback). - ImmutableList timingOutSessions = - ImmutableList.copyOf(this.keepaliveSessions); - for (DrmSession timingOutSession : timingOutSessions) { - timingOutSession.release(/* eventDispatcher= */ null); + ImmutableSet keepaliveSessions = + ImmutableSet.copyOf(this.keepaliveSessions); + for (DrmSession keepaliveSession : keepaliveSessions) { + keepaliveSession.release(/* eventDispatcher= */ null); } // Undo the acquisitions from createAndAcquireSession(). session.release(eventDispatcher); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSourceDrmHelper.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManagerProvider.java similarity index 81% rename from library/core/src/main/java/com/google/android/exoplayer2/source/MediaSourceDrmHelper.java rename to library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManagerProvider.java index f4a7b89fc7c..10bd2953d5c 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSourceDrmHelper.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManagerProvider.java @@ -13,17 +13,13 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.google.android.exoplayer2.source; +package com.google.android.exoplayer2.drm; -import static com.google.android.exoplayer2.ExoPlayerLibraryInfo.DEFAULT_USER_AGENT; import static com.google.android.exoplayer2.drm.DefaultDrmSessionManager.MODE_PLAYBACK; import androidx.annotation.Nullable; import com.google.android.exoplayer2.MediaItem; -import com.google.android.exoplayer2.drm.DefaultDrmSessionManager; -import com.google.android.exoplayer2.drm.DrmSessionManager; -import com.google.android.exoplayer2.drm.FrameworkMediaDrm; -import com.google.android.exoplayer2.drm.HttpMediaDrmCallback; +import com.google.android.exoplayer2.upstream.DefaultHttpDataSource; import com.google.android.exoplayer2.upstream.DefaultHttpDataSourceFactory; import com.google.android.exoplayer2.upstream.HttpDataSource; import com.google.android.exoplayer2.util.Assertions; @@ -31,8 +27,8 @@ import com.google.common.primitives.Ints; import java.util.Map; -/** A helper to create a {@link DrmSessionManager} from a {@link MediaItem}. */ -public final class MediaSourceDrmHelper { +/** Default implementation of {@link DrmSessionManagerProvider}. */ +public final class DefaultDrmSessionManagerProvider implements DrmSessionManagerProvider { @Nullable private HttpDataSource.Factory drmHttpDataSourceFactory; @Nullable private String userAgent; @@ -62,18 +58,18 @@ public void setDrmUserAgent(@Nullable String userAgent) { this.userAgent = userAgent; } - /** Creates a {@link DrmSessionManager} for the given media item. */ - public DrmSessionManager create(MediaItem mediaItem) { + @Override + public DrmSessionManager get(MediaItem mediaItem) { Assertions.checkNotNull(mediaItem.playbackProperties); @Nullable MediaItem.DrmConfiguration drmConfiguration = mediaItem.playbackProperties.drmConfiguration; if (drmConfiguration == null || Util.SDK_INT < 18) { - return DrmSessionManager.getDummyDrmSessionManager(); + return DrmSessionManager.DRM_UNSUPPORTED; } HttpDataSource.Factory dataSourceFactory = drmHttpDataSourceFactory != null ? drmHttpDataSourceFactory - : new DefaultHttpDataSourceFactory(userAgent != null ? userAgent : DEFAULT_USER_AGENT); + : new DefaultHttpDataSource.Factory().setUserAgent(userAgent); HttpMediaDrmCallback httpDrmCallback = new HttpMediaDrmCallback( drmConfiguration.licenseUri == null ? null : drmConfiguration.licenseUri.toString(), diff --git a/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSession.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSession.java index 97bb4b3dd16..e72d552a684 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSession.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSession.java @@ -23,6 +23,7 @@ import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.util.Map; +import java.util.UUID; /** A DRM session. */ public interface DrmSession { @@ -66,12 +67,11 @@ public DrmSessionException(Throwable cause) { @Retention(RetentionPolicy.SOURCE) @IntDef({STATE_RELEASED, STATE_ERROR, STATE_OPENING, STATE_OPENED, STATE_OPENED_WITH_KEYS}) @interface State {} - /** - * The session has been released. - */ + /** The session has been released. This is a terminal state. */ int STATE_RELEASED = 0; /** * The session has encountered an error. {@link #getError()} can be used to retrieve the cause. + * This is a terminal state. */ int STATE_ERROR = 1; /** @@ -102,6 +102,9 @@ default boolean playClearSamplesWithoutKeys() { @Nullable DrmSessionException getError(); + /** Returns the DRM scheme UUID for this session. */ + UUID getSchemeUuid(); + /** * Returns an {@link ExoMediaCrypto} for the open session, or null if called before the session * has been opened or after it's been released. diff --git a/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSessionManager.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSessionManager.java index 1168884d768..70dc4fa7f54 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSessionManager.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSessionManager.java @@ -22,13 +22,8 @@ /** Manages a DRM session. */ public interface DrmSessionManager { - /** Returns {@link #DUMMY}. */ - static DrmSessionManager getDummyDrmSessionManager() { - return DUMMY; - } - - /** {@link DrmSessionManager} that supports no DRM schemes. */ - DrmSessionManager DUMMY = + /** An instance that supports no DRM schemes. */ + DrmSessionManager DRM_UNSUPPORTED = new DrmSessionManager() { @Override @@ -54,6 +49,23 @@ public Class getExoMediaCryptoType(Format format) { } }; + /** + * An instance that supports no DRM schemes. + * + * @deprecated Use {@link #DRM_UNSUPPORTED}. + */ + @Deprecated DrmSessionManager DUMMY = DRM_UNSUPPORTED; + + /** + * Returns {@link #DRM_UNSUPPORTED}. + * + * @deprecated Use {@link #DRM_UNSUPPORTED}. + */ + @Deprecated + static DrmSessionManager getDummyDrmSessionManager() { + return DRM_UNSUPPORTED; + } + /** * Acquires any required resources. * diff --git a/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSessionManagerProvider.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSessionManagerProvider.java new file mode 100644 index 00000000000..158350e0fd2 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/DrmSessionManagerProvider.java @@ -0,0 +1,34 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.drm; + +import com.google.android.exoplayer2.MediaItem; + +/** + * A provider to obtain a {@link DrmSessionManager} suitable for playing the content described by a + * {@link MediaItem}. + */ +public interface DrmSessionManagerProvider { + + /** + * Returns a {@link DrmSessionManager} for the given media item. + * + *

    The caller is responsible for {@link DrmSessionManager#prepare() preparing} the {@link + * DrmSessionManager} before use, and subsequently {@link DrmSessionManager#release() releasing} + * it. + */ + DrmSessionManager get(MediaItem mediaItem); +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/drm/ErrorStateDrmSession.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/ErrorStateDrmSession.java index 4253d3011c6..068f1b3782f 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/drm/ErrorStateDrmSession.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/ErrorStateDrmSession.java @@ -16,8 +16,10 @@ package com.google.android.exoplayer2.drm; import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.util.Assertions; import java.util.Map; +import java.util.UUID; /** A {@link DrmSession} that's in a terminal error state. */ public final class ErrorStateDrmSession implements DrmSession { @@ -44,6 +46,11 @@ public DrmSessionException getError() { return error; } + @Override + public final UUID getSchemeUuid() { + return C.UUID_NIL; + } + @Override @Nullable public ExoMediaCrypto getMediaCrypto() { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/drm/ExoMediaDrm.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/ExoMediaDrm.java index 6684064f63e..6a5dffc6b88 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/drm/ExoMediaDrm.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/ExoMediaDrm.java @@ -20,17 +20,22 @@ import android.media.MediaDrm; import android.media.MediaDrmException; import android.media.NotProvisionedException; +import android.media.ResourceBusyException; import android.os.Handler; import android.os.PersistableBundle; +import androidx.annotation.IntDef; import androidx.annotation.Nullable; import com.google.android.exoplayer2.drm.DrmInitData.SchemeData; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.UUID; /** - * Used to obtain keys for decrypting protected media streams. See {@link android.media.MediaDrm}. + * Used to obtain keys for decrypting protected media streams. * *

    Reference counting

    * @@ -41,16 +46,18 @@ *

    Each new instance has an initial reference count of 1. Hence application code that creates a * new instance does not normally need to call {@link #acquire()}, and must call {@link #release()} * when the instance is no longer required. + * + * @see MediaDrm */ public interface ExoMediaDrm { - /** {@link ExoMediaDrm} instances provider. */ + /** Provider for {@link ExoMediaDrm} instances. */ interface Provider { /** * Returns an {@link ExoMediaDrm} instance with an incremented reference count. When the caller - * no longer needs to use the instance, it must call {@link ExoMediaDrm#release()} to decrement - * the reference count. + * no longer needs the instance, it must call {@link ExoMediaDrm#release()} to decrement the + * reference count. */ ExoMediaDrm acquireExoMediaDrm(UUID uuid); } @@ -78,37 +85,37 @@ public ExoMediaDrm acquireExoMediaDrm(UUID uuid) { } } - /** @see MediaDrm#EVENT_KEY_REQUIRED */ + /** Event indicating that keys need to be requested from the license server. */ @SuppressWarnings("InlinedApi") int EVENT_KEY_REQUIRED = MediaDrm.EVENT_KEY_REQUIRED; - /** - * @see MediaDrm#EVENT_KEY_EXPIRED - */ + /** Event indicating that keys have expired, and are no longer usable. */ @SuppressWarnings("InlinedApi") int EVENT_KEY_EXPIRED = MediaDrm.EVENT_KEY_EXPIRED; - /** - * @see MediaDrm#EVENT_PROVISION_REQUIRED - */ + /** Event indicating that a certificate needs to be requested from the provisioning server. */ @SuppressWarnings("InlinedApi") int EVENT_PROVISION_REQUIRED = MediaDrm.EVENT_PROVISION_REQUIRED; /** - * @see MediaDrm#KEY_TYPE_STREAMING + * Key request type for keys that will be used for online use. Streaming keys will not be saved to + * the device for subsequent use when the device is not connected to a network. */ @SuppressWarnings("InlinedApi") int KEY_TYPE_STREAMING = MediaDrm.KEY_TYPE_STREAMING; /** - * @see MediaDrm#KEY_TYPE_OFFLINE + * Key request type for keys that will be used for offline use. They will be saved to the device + * for subsequent use when the device is not connected to a network. */ @SuppressWarnings("InlinedApi") int KEY_TYPE_OFFLINE = MediaDrm.KEY_TYPE_OFFLINE; - /** - * @see MediaDrm#KEY_TYPE_RELEASE - */ + /** Key request type indicating that saved offline keys should be released. */ @SuppressWarnings("InlinedApi") int KEY_TYPE_RELEASE = MediaDrm.KEY_TYPE_RELEASE; - /** @see android.media.MediaDrm.OnEventListener */ + /** + * Called when a DRM event occurs. + * + * @see MediaDrm.OnEventListener + */ interface OnEventListener { /** * Called when an event occurs that requires the app to be notified @@ -127,7 +134,11 @@ void onEvent( @Nullable byte[] data); } - /** @see android.media.MediaDrm.OnKeyStatusChangeListener */ + /** + * Called when the keys in a DRM session change state. + * + * @see MediaDrm.OnKeyStatusChangeListener + */ interface OnKeyStatusChangeListener { /** * Called when the keys in a session change status, such as when the license is renewed or @@ -145,12 +156,16 @@ void onKeyStatusChange( boolean hasNewUsableKey); } - /** @see android.media.MediaDrm.OnExpirationUpdateListener */ + /** + * Called when a session expiration update occurs. + * + * @see MediaDrm.OnExpirationUpdateListener + */ interface OnExpirationUpdateListener { /** * Called when a session expiration update occurs, to inform the app about the change in - * expiration time + * expiration time. * * @param mediaDrm The {@link ExoMediaDrm} object on which the event occurred. * @param sessionId The DRM session ID on which the event occurred @@ -161,67 +176,168 @@ interface OnExpirationUpdateListener { void onExpirationUpdate(ExoMediaDrm mediaDrm, byte[] sessionId, long expirationTimeMs); } - /** @see android.media.MediaDrm.KeyStatus */ + /** + * Defines the status of a key. + * + * @see MediaDrm.KeyStatus + */ final class KeyStatus { private final int statusCode; private final byte[] keyId; + /** + * Creates an instance. + * + * @param statusCode The status code of the key, as defined by {@link + * MediaDrm.KeyStatus#getStatusCode()}. + * @param keyId The ID of the key. + */ public KeyStatus(int statusCode, byte[] keyId) { this.statusCode = statusCode; this.keyId = keyId; } + /** Returns the status of the key, as defined by {@link MediaDrm.KeyStatus#getStatusCode()}. */ public int getStatusCode() { return statusCode; } + /** Returns the ID of the key. */ public byte[] getKeyId() { return keyId; } - } - /** @see android.media.MediaDrm.KeyRequest */ + /** + * Contains data used to request keys from a license server. + * + * @see MediaDrm.KeyRequest + */ final class KeyRequest { + /** + * Key request types. One of {@link #REQUEST_TYPE_UNKNOWN}, {@link #REQUEST_TYPE_INITIAL}, + * {@link #REQUEST_TYPE_RENEWAL}, {@link #REQUEST_TYPE_RELEASE}, {@link #REQUEST_TYPE_NONE} or + * {@link #REQUEST_TYPE_UPDATE}. + */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + REQUEST_TYPE_UNKNOWN, + REQUEST_TYPE_INITIAL, + REQUEST_TYPE_RENEWAL, + REQUEST_TYPE_RELEASE, + REQUEST_TYPE_NONE, + REQUEST_TYPE_UPDATE, + }) + public @interface RequestType {} + + /** + * Value returned from {@link #getRequestType()} if the underlying key request does not specify + * a type. + */ + public static final int REQUEST_TYPE_UNKNOWN = Integer.MIN_VALUE; + + /** Key request type for an initial license request. */ + public static final int REQUEST_TYPE_INITIAL = MediaDrm.KeyRequest.REQUEST_TYPE_INITIAL; + /** Key request type for license renewal. */ + public static final int REQUEST_TYPE_RENEWAL = MediaDrm.KeyRequest.REQUEST_TYPE_RENEWAL; + /** Key request type for license release. */ + public static final int REQUEST_TYPE_RELEASE = MediaDrm.KeyRequest.REQUEST_TYPE_RELEASE; + /** + * Key request type if keys are already loaded and available for use. No license request is + * necessary, and no key request data is returned. + */ + public static final int REQUEST_TYPE_NONE = MediaDrm.KeyRequest.REQUEST_TYPE_NONE; + /** + * Key request type if keys have been loaded, but an additional license request is needed to + * update their values. + */ + public static final int REQUEST_TYPE_UPDATE = MediaDrm.KeyRequest.REQUEST_TYPE_UPDATE; + private final byte[] data; private final String licenseServerUrl; + @RequestType private final int requestType; + /** + * Creates an instance with {@link #REQUEST_TYPE_UNKNOWN}. + * + * @param data The opaque key request data. + * @param licenseServerUrl The license server URL to which the request should be made. + */ public KeyRequest(byte[] data, String licenseServerUrl) { + this(data, licenseServerUrl, REQUEST_TYPE_UNKNOWN); + } + + /** + * Creates an instance. + * + * @param data The opaque key request data. + * @param licenseServerUrl The license server URL to which the request should be made. + * @param requestType The type of the request, or {@link #REQUEST_TYPE_UNKNOWN}. + */ + public KeyRequest(byte[] data, String licenseServerUrl, @RequestType int requestType) { this.data = data; this.licenseServerUrl = licenseServerUrl; + this.requestType = requestType; } + /** Returns the opaque key request data. */ public byte[] getData() { return data; } + /** Returns the URL of the license server to which the request should be made. */ public String getLicenseServerUrl() { return licenseServerUrl; } + /** + * Returns the type of the request, or {@link #REQUEST_TYPE_UNKNOWN} if the underlying key + * request does not specify a type. Note that when using a platform {@link MediaDrm} instance, + * key requests only specify a type on API levels 23 and above. + */ + @RequestType + public int getRequestType() { + return requestType; + } } - /** @see android.media.MediaDrm.ProvisionRequest */ + /** + * Contains data to request a certificate from a provisioning server. + * + * @see MediaDrm.ProvisionRequest + */ final class ProvisionRequest { private final byte[] data; private final String defaultUrl; + /** + * Creates an instance. + * + * @param data The opaque provisioning request data. + * @param defaultUrl The default URL of the provisioning server to which the request can be + * made, or the empty string if not known. + */ public ProvisionRequest(byte[] data, String defaultUrl) { this.data = data; this.defaultUrl = defaultUrl; } + /** Returns the opaque provisioning request data. */ public byte[] getData() { return data; } + /** + * Returns the default URL of the provisioning server to which the request can be made, or the + * empty string if not known. + */ public String getDefaultUrl() { return defaultUrl; } - } /** @@ -260,11 +376,20 @@ public String getDefaultUrl() { */ void setOnExpirationUpdateListener(@Nullable OnExpirationUpdateListener listener); - /** @see MediaDrm#openSession() */ + /** + * Opens a new DRM session. A session ID is returned. + * + * @return The session ID. + * @throws NotProvisionedException If provisioning is needed. + * @throws ResourceBusyException If required resources are in use. + * @throws MediaDrmException If the session could not be opened. + */ byte[] openSession() throws MediaDrmException; /** - * @see MediaDrm#closeSession(byte[]) + * Closes a DRM session. + * + * @param sessionId The ID of the session to close. */ void closeSession(byte[] sessionId); @@ -272,8 +397,8 @@ public String getDefaultUrl() { * Generates a key request. * * @param scope If {@code keyType} is {@link #KEY_TYPE_STREAMING} or {@link #KEY_TYPE_OFFLINE}, - * the session id that the keys will be provided to. If {@code keyType} is {@link - * #KEY_TYPE_RELEASE}, the keySetId of the keys to release. + * the ID of the session that the keys will be provided to. If {@code keyType} is {@link + * #KEY_TYPE_RELEASE}, the {@code keySetId} of the keys to release. * @param schemeDatas If key type is {@link #KEY_TYPE_STREAMING} or {@link #KEY_TYPE_OFFLINE}, a * list of {@link SchemeData} instances extracted from the media. Null otherwise. * @param keyType The type of the request. Either {@link #KEY_TYPE_STREAMING} to acquire keys for @@ -293,23 +418,45 @@ KeyRequest getKeyRequest( @Nullable HashMap optionalParameters) throws NotProvisionedException; - /** @see MediaDrm#provideKeyResponse(byte[], byte[]) */ + /** + * Provides a key response for the last request to be generated using {@link #getKeyRequest}. + * + * @param scope If the request had type {@link #KEY_TYPE_STREAMING} or {@link #KEY_TYPE_OFFLINE}, + * the ID of the session to provide the keys to. If {@code keyType} is {@link + * #KEY_TYPE_RELEASE}, the {@code keySetId} of the keys being released. + * @param response The response data from the server. + * @return If the request had type {@link #KEY_TYPE_OFFLINE}, the {@code keySetId} for the offline + * keys. An empty byte array or {@code null} may be returned for other cases. + * @throws NotProvisionedException If the response indicates that provisioning is needed. + * @throws DeniedByServerException If the response indicates that the server rejected the request. + */ @Nullable byte[] provideKeyResponse(byte[] scope, byte[] response) throws NotProvisionedException, DeniedByServerException; /** - * @see MediaDrm#getProvisionRequest() + * Generates a provisioning request. + * + * @return The generated provisioning request. */ ProvisionRequest getProvisionRequest(); /** - * @see MediaDrm#provideProvisionResponse(byte[]) + * Provides a provisioning response for the last request to be generated using {@link + * #getProvisionRequest()}. + * + * @param response The response data from the server. + * @throws DeniedByServerException If the response indicates that the server rejected the request. */ void provideProvisionResponse(byte[] response) throws DeniedByServerException; /** - * @see MediaDrm#queryKeyStatus(byte[]) + * Returns the key status for a given session, as {name, value} pairs. Since DRM license policies + * vary by vendor, the returned entries depend on the DRM plugin being used. Refer to your DRM + * provider's documentation for more information. + * + * @param sessionId The ID of the session being queried. + * @return The key status for the session. */ Map queryKeyStatus(byte[] sessionId); @@ -329,43 +476,64 @@ byte[] provideKeyResponse(byte[] scope, byte[] response) void release(); /** - * @see MediaDrm#restoreKeys(byte[], byte[]) + * Restores persisted offline keys into a session. + * + * @param sessionId The ID of the session into which the keys will be restored. + * @param keySetId The {@code keySetId} of the keys to restore, as provided by the call to {@link + * #provideKeyResponse} that persisted them. */ void restoreKeys(byte[] sessionId, byte[] keySetId); /** - * Returns drm metrics. May be null if unavailable. - * - * @see MediaDrm#getMetrics() + * Returns metrics data for this ExoMediaDrm instance, or {@code null} if metrics are unavailable. */ @Nullable PersistableBundle getMetrics(); /** - * @see MediaDrm#getPropertyString(String) + * Returns the value of a string property. For standard property names, see {@link + * MediaDrm#getPropertyString}. + * + * @param propertyName The property name. + * @return The property value. + * @throws IllegalArgumentException If the underlying DRM plugin does not support the property. */ String getPropertyString(String propertyName); /** - * @see MediaDrm#getPropertyByteArray(String) + * Returns the value of a byte array property. For standard property names, see {@link + * MediaDrm#getPropertyByteArray}. + * + * @param propertyName The property name. + * @return The property value. + * @throws IllegalArgumentException If the underlying DRM plugin does not support the property. */ byte[] getPropertyByteArray(String propertyName); /** - * @see MediaDrm#setPropertyString(String, String) + * Sets the value of a string property. + * + * @param propertyName The property name. + * @param value The value. + * @throws IllegalArgumentException If the underlying DRM plugin does not support the property. */ void setPropertyString(String propertyName, String value); /** - * @see MediaDrm#setPropertyByteArray(String, byte[]) + * Sets the value of a byte array property. + * + * @param propertyName The property name. + * @param value The value. + * @throws IllegalArgumentException If the underlying DRM plugin does not support the property. */ void setPropertyByteArray(String propertyName, byte[] value); /** - * @see android.media.MediaCrypto#MediaCrypto(UUID, byte[]) - * @param sessionId The DRM session ID. - * @return An object extends {@link ExoMediaCrypto}, using opaque crypto scheme specific data. - * @throws MediaCryptoException If the instance can't be created. + * Creates an {@link ExoMediaCrypto} for a given session. + * + * @param sessionId The ID of the session. + * @return An {@link ExoMediaCrypto} for the given session. + * @throws MediaCryptoException If an {@link ExoMediaCrypto} could not be created. */ ExoMediaCrypto createMediaCrypto(byte[] sessionId) throws MediaCryptoException; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/drm/FrameworkMediaDrm.java b/library/core/src/main/java/com/google/android/exoplayer2/drm/FrameworkMediaDrm.java index 26fe66e7923..a54975392b3 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/drm/FrameworkMediaDrm.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/drm/FrameworkMediaDrm.java @@ -210,7 +210,11 @@ public KeyRequest getKeyRequest( licenseServerUrl = schemeData.licenseServerUrl; } - return new KeyRequest(requestData, licenseServerUrl); + @KeyRequest.RequestType + int requestType = + Util.SDK_INT >= 23 ? request.getRequestType() : KeyRequest.REQUEST_TYPE_UNKNOWN; + + return new KeyRequest(requestData, licenseServerUrl, requestType); } @Override diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecAdapter.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecAdapter.java index 84ff985495b..e5c50c1f8ed 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecAdapter.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecAdapter.java @@ -19,32 +19,92 @@ import android.media.MediaCodec; import android.media.MediaCrypto; import android.media.MediaFormat; +import android.os.Bundle; import android.os.Handler; import android.os.HandlerThread; import android.view.Surface; -import androidx.annotation.GuardedBy; import androidx.annotation.IntDef; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.decoder.CryptoInfo; -import com.google.android.exoplayer2.util.Util; +import com.google.common.base.Supplier; import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; -import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import java.nio.ByteBuffer; /** - * A {@link MediaCodecAdapter} that operates the underlying {@link MediaCodec} in asynchronous mode - * and routes {@link MediaCodec.Callback} callbacks on a dedicated thread that is managed - * internally. - * - *

    This adapter supports queueing input buffers asynchronously. + * A {@link MediaCodecAdapter} that operates the underlying {@link MediaCodec} in asynchronous mode, + * routes {@link MediaCodec.Callback} callbacks on a dedicated thread that is managed internally, + * and queues input buffers asynchronously. */ @RequiresApi(23) -/* package */ final class AsynchronousMediaCodecAdapter extends MediaCodec.Callback - implements MediaCodecAdapter { +/* package */ final class AsynchronousMediaCodecAdapter implements MediaCodecAdapter { + + /** A factory for {@link AsynchronousMediaCodecAdapter} instances. */ + public static final class Factory implements MediaCodecAdapter.Factory { + private final Supplier callbackThreadSupplier; + private final Supplier queueingThreadSupplier; + private final boolean forceQueueingSynchronizationWorkaround; + private final boolean synchronizeCodecInteractionsWithQueueing; + + /** Creates a factory for the specified {@code trackType}. */ + public Factory(int trackType) { + this( + trackType, + /* forceQueueingSynchronizationWorkaround= */ false, + /* synchronizeCodecInteractionsWithQueueing= */ false); + } + + /** + * Creates an factory for {@link AsynchronousMediaCodecAdapter} instances. + * + * @param trackType One of {@link C#TRACK_TYPE_AUDIO} or {@link C#TRACK_TYPE_VIDEO}. Used for + * labelling the internal thread accordingly. + * @param forceQueueingSynchronizationWorkaround Whether the queueing synchronization workaround + * will be enabled by default or only for the predefined devices. + * @param synchronizeCodecInteractionsWithQueueing Whether the adapter should synchronize {@link + * MediaCodec} interactions with asynchronous buffer queueing. When {@code true}, codec + * interactions will wait until all input buffers pending queueing wil be submitted to the + * {@link MediaCodec}. + */ + public Factory( + int trackType, + boolean forceQueueingSynchronizationWorkaround, + boolean synchronizeCodecInteractionsWithQueueing) { + this( + /* callbackThreadSupplier= */ () -> + new HandlerThread(createCallbackThreadLabel(trackType)), + /* queueingThreadSupplier= */ () -> + new HandlerThread(createQueueingThreadLabel(trackType)), + forceQueueingSynchronizationWorkaround, + synchronizeCodecInteractionsWithQueueing); + } + + @VisibleForTesting + /* package */ Factory( + Supplier callbackThreadSupplier, + Supplier queueingThreadSupplier, + boolean forceQueueingSynchronizationWorkaround, + boolean synchronizeCodecInteractionsWithQueueing) { + this.callbackThreadSupplier = callbackThreadSupplier; + this.queueingThreadSupplier = queueingThreadSupplier; + this.forceQueueingSynchronizationWorkaround = forceQueueingSynchronizationWorkaround; + this.synchronizeCodecInteractionsWithQueueing = synchronizeCodecInteractionsWithQueueing; + } + + @Override + public AsynchronousMediaCodecAdapter createAdapter(MediaCodec codec) { + return new AsynchronousMediaCodecAdapter( + codec, + callbackThreadSupplier.get(), + queueingThreadSupplier.get(), + forceQueueingSynchronizationWorkaround, + synchronizeCodecInteractionsWithQueueing); + } + } @Documented @Retention(RetentionPolicy.SOURCE) @@ -56,72 +116,25 @@ private static final int STATE_STARTED = 2; private static final int STATE_SHUT_DOWN = 3; - private final Object lock; - - @GuardedBy("lock") - private final MediaCodecAsyncCallback mediaCodecAsyncCallback; - private final MediaCodec codec; - private final HandlerThread handlerThread; - private @MonotonicNonNull Handler handler; - - @GuardedBy("lock") - private long pendingFlushCount; - - private @State int state; - private final MediaCodecInputBufferEnqueuer bufferEnqueuer; - - @GuardedBy("lock") - @Nullable - private IllegalStateException internalException; - - /** - * Creates an instance that wraps the specified {@link MediaCodec}. Instances created with this - * constructor will queue input buffers to the {@link MediaCodec} synchronously. - * - * @param codec The {@link MediaCodec} to wrap. - * @param trackType One of {@link C#TRACK_TYPE_AUDIO} or {@link C#TRACK_TYPE_VIDEO}. Used for - * labelling the internal thread accordingly. - */ - /* package */ AsynchronousMediaCodecAdapter(MediaCodec codec, int trackType) { - this( - codec, - /* enableAsynchronousQueueing= */ false, - trackType, - new HandlerThread(createThreadLabel(trackType))); - } - - /** - * Creates an instance that wraps the specified {@link MediaCodec}. - * - * @param codec The {@link MediaCodec} to wrap. - * @param enableAsynchronousQueueing Whether input buffers will be queued asynchronously. - * @param trackType One of {@link C#TRACK_TYPE_AUDIO} or {@link C#TRACK_TYPE_VIDEO}. Used for - * labelling the internal thread accordingly. - */ - /* package */ AsynchronousMediaCodecAdapter( - MediaCodec codec, boolean enableAsynchronousQueueing, int trackType) { - this( - codec, - enableAsynchronousQueueing, - trackType, - new HandlerThread(createThreadLabel(trackType))); - } + private final AsynchronousMediaCodecCallback asynchronousMediaCodecCallback; + private final AsynchronousMediaCodecBufferEnqueuer bufferEnqueuer; + private final boolean synchronizeCodecInteractionsWithQueueing; + private boolean codecReleased; + @State private int state; - @VisibleForTesting - /* package */ AsynchronousMediaCodecAdapter( + private AsynchronousMediaCodecAdapter( MediaCodec codec, - boolean enableAsynchronousQueueing, - int trackType, - HandlerThread handlerThread) { - this.lock = new Object(); - this.mediaCodecAsyncCallback = new MediaCodecAsyncCallback(); + HandlerThread callbackThread, + HandlerThread enqueueingThread, + boolean forceQueueingSynchronizationWorkaround, + boolean synchronizeCodecInteractionsWithQueueing) { this.codec = codec; - this.handlerThread = handlerThread; + this.asynchronousMediaCodecCallback = new AsynchronousMediaCodecCallback(callbackThread); this.bufferEnqueuer = - enableAsynchronousQueueing - ? new AsynchronousMediaCodecBufferEnqueuer(codec, trackType) - : new SynchronousMediaCodecBufferEnqueuer(this.codec); + new AsynchronousMediaCodecBufferEnqueuer( + codec, enqueueingThread, forceQueueingSynchronizationWorkaround); + this.synchronizeCodecInteractionsWithQueueing = synchronizeCodecInteractionsWithQueueing; this.state = STATE_CREATED; } @@ -131,9 +144,7 @@ public void configure( @Nullable Surface surface, @Nullable MediaCrypto crypto, int flags) { - handlerThread.start(); - handler = new Handler(handlerThread.getLooper()); - codec.setCallback(this, handler); + asynchronousMediaCodecCallback.initialize(codec); codec.configure(mediaFormat, surface, crypto, flags); state = STATE_CONFIGURED; } @@ -157,153 +168,134 @@ public void queueSecureInputBuffer( bufferEnqueuer.queueSecureInputBuffer(index, offset, info, presentationTimeUs, flags); } + @Override + public void releaseOutputBuffer(int index, boolean render) { + codec.releaseOutputBuffer(index, render); + } + + @Override + public void releaseOutputBuffer(int index, long renderTimeStampNs) { + codec.releaseOutputBuffer(index, renderTimeStampNs); + } + @Override public int dequeueInputBufferIndex() { - synchronized (lock) { - if (isFlushing()) { - return MediaCodec.INFO_TRY_AGAIN_LATER; - } else { - maybeThrowException(); - return mediaCodecAsyncCallback.dequeueInputBufferIndex(); - } - } + return asynchronousMediaCodecCallback.dequeueInputBufferIndex(); } @Override public int dequeueOutputBufferIndex(MediaCodec.BufferInfo bufferInfo) { - synchronized (lock) { - if (isFlushing()) { - return MediaCodec.INFO_TRY_AGAIN_LATER; - } else { - maybeThrowException(); - return mediaCodecAsyncCallback.dequeueOutputBufferIndex(bufferInfo); - } - } + return asynchronousMediaCodecCallback.dequeueOutputBufferIndex(bufferInfo); } @Override public MediaFormat getOutputFormat() { - synchronized (lock) { - return mediaCodecAsyncCallback.getOutputFormat(); - } + return asynchronousMediaCodecCallback.getOutputFormat(); + } + + @Override + @Nullable + public ByteBuffer getInputBuffer(int index) { + return codec.getInputBuffer(index); + } + + @Override + @Nullable + public ByteBuffer getOutputBuffer(int index) { + return codec.getOutputBuffer(index); } @Override public void flush() { - synchronized (lock) { - bufferEnqueuer.flush(); - codec.flush(); - ++pendingFlushCount; - Util.castNonNull(handler).post(this::onFlushCompleted); - } + // The order of calls is important: + // First, flush the bufferEnqueuer to stop queueing input buffers. + // Second, flush the codec to stop producing available input/output buffers. + // Third, flush the callback after flushing the codec so that in-flight callbacks are discarded. + bufferEnqueuer.flush(); + codec.flush(); + // When flushAsync() is completed, start the codec again. + asynchronousMediaCodecCallback.flushAsync(/* onFlushCompleted= */ codec::start); } @Override - public void shutdown() { - synchronized (lock) { + public void release() { + try { if (state == STATE_STARTED) { bufferEnqueuer.shutdown(); } if (state == STATE_CONFIGURED || state == STATE_STARTED) { - handlerThread.quit(); - mediaCodecAsyncCallback.flush(); - // Leave the adapter in a flushing state so that - // it will not dequeue anything. - ++pendingFlushCount; + asynchronousMediaCodecCallback.shutdown(); } state = STATE_SHUT_DOWN; + } finally { + if (!codecReleased) { + codec.release(); + codecReleased = true; + } } } @Override - public MediaCodec getCodec() { - return codec; + public void setOnFrameRenderedListener(OnFrameRenderedListener listener, Handler handler) { + maybeBlockOnQueueing(); + codec.setOnFrameRenderedListener( + (codec, presentationTimeUs, nanoTime) -> + listener.onFrameRendered( + AsynchronousMediaCodecAdapter.this, presentationTimeUs, nanoTime), + handler); } - // Called from the handler thread. - @Override - public void onInputBufferAvailable(MediaCodec codec, int index) { - synchronized (lock) { - mediaCodecAsyncCallback.onInputBufferAvailable(codec, index); - } + public void setOutputSurface(Surface surface) { + maybeBlockOnQueueing(); + codec.setOutputSurface(surface); } @Override - public void onOutputBufferAvailable(MediaCodec codec, int index, MediaCodec.BufferInfo info) { - synchronized (lock) { - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, index, info); - } + public void setParameters(Bundle params) { + maybeBlockOnQueueing(); + codec.setParameters(params); } @Override - public void onError(MediaCodec codec, MediaCodec.CodecException e) { - synchronized (lock) { - mediaCodecAsyncCallback.onError(codec, e); - } + public void setVideoScalingMode(@C.VideoScalingMode int scalingMode) { + maybeBlockOnQueueing(); + codec.setVideoScalingMode(scalingMode); } - @Override - public void onOutputFormatChanged(MediaCodec codec, MediaFormat format) { - synchronized (lock) { - mediaCodecAsyncCallback.onOutputFormatChanged(codec, format); - } + @VisibleForTesting + /* package */ void onError(MediaCodec.CodecException error) { + asynchronousMediaCodecCallback.onError(codec, error); } - private void onFlushCompleted() { - synchronized (lock) { - onFlushCompletedSynchronized(); - } + @VisibleForTesting + /* package */ void onOutputFormatChanged(MediaFormat format) { + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, format); } - @GuardedBy("lock") - private void onFlushCompletedSynchronized() { - if (state == STATE_SHUT_DOWN) { - return; - } - - --pendingFlushCount; - if (pendingFlushCount > 0) { - // Another flush() has been called. - return; - } else if (pendingFlushCount < 0) { - // This should never happen. - internalException = new IllegalStateException(); - return; - } - - mediaCodecAsyncCallback.flush(); - try { - codec.start(); - } catch (IllegalStateException e) { - internalException = e; - } catch (Exception e) { - internalException = new IllegalStateException(e); + private void maybeBlockOnQueueing() { + if (synchronizeCodecInteractionsWithQueueing) { + try { + bufferEnqueuer.waitUntilQueueingComplete(); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + // The playback thread should not be interrupted. Raising this as an + // IllegalStateException. + throw new IllegalStateException(e); + } } } - @GuardedBy("lock") - private boolean isFlushing() { - return pendingFlushCount > 0; + private static String createCallbackThreadLabel(int trackType) { + return createThreadLabel(trackType, /* prefix= */ "ExoPlayer:MediaCodecAsyncAdapter:"); } - @GuardedBy("lock") - private void maybeThrowException() { - maybeThrowInternalException(); - mediaCodecAsyncCallback.maybeThrowMediaCodecException(); - } - - @GuardedBy("lock") - private void maybeThrowInternalException() { - if (internalException != null) { - IllegalStateException e = internalException; - internalException = null; - throw e; - } + private static String createQueueingThreadLabel(int trackType) { + return createThreadLabel(trackType, /* prefix= */ "ExoPlayer:MediaCodecQueueingThread:"); } - private static String createThreadLabel(int trackType) { - StringBuilder labelBuilder = new StringBuilder("ExoPlayer:MediaCodecAsyncAdapter:"); + private static String createThreadLabel(int trackType, String prefix) { + StringBuilder labelBuilder = new StringBuilder(prefix); if (trackType == C.TRACK_TYPE_AUDIO) { labelBuilder.append("Audio"); } else if (trackType == C.TRACK_TYPE_VIDEO) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecBufferEnqueuer.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecBufferEnqueuer.java index dd9a0864461..21f79a78a28 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecBufferEnqueuer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecBufferEnqueuer.java @@ -17,6 +17,7 @@ package com.google.android.exoplayer2.mediacodec; import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Util.castNonNull; import android.media.MediaCodec; import android.os.Handler; @@ -26,7 +27,6 @@ import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import androidx.annotation.VisibleForTesting; -import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.decoder.CryptoInfo; import com.google.android.exoplayer2.util.ConditionVariable; import com.google.android.exoplayer2.util.Util; @@ -37,17 +37,17 @@ import org.checkerframework.checker.nullness.qual.MonotonicNonNull; /** - * A {@link MediaCodecInputBufferEnqueuer} that defers queueing operations on a background thread. + * Performs {@link MediaCodec} input buffer queueing on a background thread. * *

    The implementation of this class assumes that its public methods will be called from the same * thread. */ @RequiresApi(23) -class AsynchronousMediaCodecBufferEnqueuer implements MediaCodecInputBufferEnqueuer { +class AsynchronousMediaCodecBufferEnqueuer { private static final int MSG_QUEUE_INPUT_BUFFER = 0; private static final int MSG_QUEUE_SECURE_INPUT_BUFFER = 1; - private static final int MSG_FLUSH = 2; + private static final int MSG_OPEN_CV = 2; @GuardedBy("MESSAGE_PARAMS_INSTANCE_POOL") private static final ArrayDeque MESSAGE_PARAMS_INSTANCE_POOL = new ArrayDeque<>(); @@ -66,26 +66,38 @@ class AsynchronousMediaCodecBufferEnqueuer implements MediaCodecInputBufferEnque * Creates a new instance that submits input buffers on the specified {@link MediaCodec}. * * @param codec The {@link MediaCodec} to submit input buffers to. - * @param trackType The type of stream (used for debug logs). + * @param queueingThread The {@link HandlerThread} to use for queueing buffers. */ - public AsynchronousMediaCodecBufferEnqueuer(MediaCodec codec, int trackType) { + public AsynchronousMediaCodecBufferEnqueuer( + MediaCodec codec, + HandlerThread queueingThread, + boolean forceQueueingSynchronizationWorkaround) { this( codec, - new HandlerThread(createThreadLabel(trackType)), + queueingThread, + forceQueueingSynchronizationWorkaround, /* conditionVariable= */ new ConditionVariable()); } @VisibleForTesting /* package */ AsynchronousMediaCodecBufferEnqueuer( - MediaCodec codec, HandlerThread handlerThread, ConditionVariable conditionVariable) { + MediaCodec codec, + HandlerThread handlerThread, + boolean forceQueueingSynchronizationWorkaround, + ConditionVariable conditionVariable) { this.codec = codec; this.handlerThread = handlerThread; this.conditionVariable = conditionVariable; pendingRuntimeException = new AtomicReference<>(); - needsSynchronizationWorkaround = needsSynchronizationWorkaround(); + needsSynchronizationWorkaround = + forceQueueingSynchronizationWorkaround || needsSynchronizationWorkaround(); } - @Override + /** + * Starts this instance. + * + *

    Call this method after creating an instance and before queueing input buffers. + */ public void start() { if (!started) { handlerThread.start(); @@ -100,18 +112,29 @@ public void handleMessage(Message msg) { } } - @Override + /** + * Submits an input buffer for decoding. + * + * @see android.media.MediaCodec#queueInputBuffer + */ public void queueInputBuffer( int index, int offset, int size, long presentationTimeUs, int flags) { maybeThrowException(); MessageParams messageParams = getMessageParams(); messageParams.setQueueParams(index, offset, size, presentationTimeUs, flags); - Message message = - Util.castNonNull(handler).obtainMessage(MSG_QUEUE_INPUT_BUFFER, messageParams); + Message message = castNonNull(handler).obtainMessage(MSG_QUEUE_INPUT_BUFFER, messageParams); message.sendToTarget(); } - @Override + /** + * Submits an input buffer that potentially contains encrypted data for decoding. + * + *

    Note: This method behaves as {@link MediaCodec#queueSecureInputBuffer} with the difference + * that {@code info} is of type {@link CryptoInfo} and not {@link + * android.media.MediaCodec.CryptoInfo}. + * + * @see android.media.MediaCodec#queueSecureInputBuffer + */ public void queueSecureInputBuffer( int index, int offset, CryptoInfo info, long presentationTimeUs, int flags) { maybeThrowException(); @@ -119,11 +142,11 @@ public void queueSecureInputBuffer( messageParams.setQueueParams(index, offset, /* size= */ 0, presentationTimeUs, flags); copy(info, messageParams.cryptoInfo); Message message = - Util.castNonNull(handler).obtainMessage(MSG_QUEUE_SECURE_INPUT_BUFFER, messageParams); + castNonNull(handler).obtainMessage(MSG_QUEUE_SECURE_INPUT_BUFFER, messageParams); message.sendToTarget(); } - @Override + /** Flushes the instance. */ public void flush() { if (started) { try { @@ -137,7 +160,7 @@ public void flush() { } } - @Override + /** Shut down the instance. Make sure to call this method to release its internal resources. */ public void shutdown() { if (started) { flush(); @@ -146,8 +169,45 @@ public void shutdown() { started = false; } + /** Blocks the current thread until all input buffers pending queueing are submitted. */ + public void waitUntilQueueingComplete() throws InterruptedException { + blockUntilHandlerThreadIsIdle(); + } + + private void maybeThrowException() { + @Nullable RuntimeException exception = pendingRuntimeException.getAndSet(null); + if (exception != null) { + throw exception; + } + } + + /** + * Empties all tasks enqueued on the {@link #handlerThread} via the {@link #handler}. This method + * blocks until the {@link #handlerThread} is idle. + */ + private void flushHandlerThread() throws InterruptedException { + Handler handler = castNonNull(this.handler); + handler.removeCallbacksAndMessages(null); + blockUntilHandlerThreadIsIdle(); + // Check if any exceptions happened during the last queueing action. + maybeThrowException(); + } + + private void blockUntilHandlerThreadIsIdle() throws InterruptedException { + conditionVariable.close(); + castNonNull(handler).obtainMessage(MSG_OPEN_CV).sendToTarget(); + conditionVariable.block(); + } + + // Called from the handler thread + + @VisibleForTesting + /* package */ void setPendingRuntimeException(RuntimeException exception) { + pendingRuntimeException.set(exception); + } + private void doHandleMessage(Message msg) { - MessageParams params = null; + @Nullable MessageParams params = null; switch (msg.what) { case MSG_QUEUE_INPUT_BUFFER: params = (MessageParams) msg.obj; @@ -163,7 +223,7 @@ private void doHandleMessage(Message msg) { params.presentationTimeUs, params.flags); break; - case MSG_FLUSH: + case MSG_OPEN_CV: conditionVariable.open(); break; default: @@ -174,34 +234,6 @@ private void doHandleMessage(Message msg) { } } - private void maybeThrowException() { - RuntimeException exception = pendingRuntimeException.getAndSet(null); - if (exception != null) { - throw exception; - } - } - - /** - * Empties all tasks enqueued on the {@link #handlerThread} via the {@link #handler}. This method - * blocks until the {@link #handlerThread} is idle. - */ - private void flushHandlerThread() throws InterruptedException { - Handler handler = Util.castNonNull(this.handler); - handler.removeCallbacksAndMessages(null); - conditionVariable.close(); - handler.obtainMessage(MSG_FLUSH).sendToTarget(); - conditionVariable.block(); - // Check if any exceptions happened during the last queueing action. - maybeThrowException(); - } - - // Called from the handler thread - - @VisibleForTesting - /* package */ void setPendingRuntimeException(RuntimeException exception) { - pendingRuntimeException.set(exception); - } - private void doQueueInputBuffer( int index, int offset, int size, long presentationTimeUs, int flag) { try { @@ -226,13 +258,6 @@ private void doQueueSecureInputBuffer( } } - @VisibleForTesting - /* package */ static int getInstancePoolSize() { - synchronized (MESSAGE_PARAMS_INSTANCE_POOL) { - return MESSAGE_PARAMS_INSTANCE_POOL.size(); - } - } - private static MessageParams getMessageParams() { synchronized (MESSAGE_PARAMS_INSTANCE_POOL) { if (MESSAGE_PARAMS_INSTANCE_POOL.isEmpty()) { @@ -282,18 +307,6 @@ private static boolean needsSynchronizationWorkaround() { return manufacturer.contains("samsung") || manufacturer.contains("motorola"); } - private static String createThreadLabel(int trackType) { - StringBuilder labelBuilder = new StringBuilder("ExoPlayer:MediaCodecBufferEnqueuer:"); - if (trackType == C.TRACK_TYPE_AUDIO) { - labelBuilder.append("Audio"); - } else if (trackType == C.TRACK_TYPE_VIDEO) { - labelBuilder.append("Video"); - } else { - labelBuilder.append("Unknown(").append(trackType).append(")"); - } - return labelBuilder.toString(); - } - /** Performs a deep copy of {@code cryptoInfo} to {@code frameworkCryptoInfo}. */ private static void copy( CryptoInfo cryptoInfo, android.media.MediaCodec.CryptoInfo frameworkCryptoInfo) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecCallback.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecCallback.java new file mode 100644 index 00000000000..872cc77d6ad --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecCallback.java @@ -0,0 +1,332 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.mediacodec; + +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static com.google.android.exoplayer2.util.Assertions.checkStateNotNull; + +import android.media.MediaCodec; +import android.media.MediaFormat; +import android.os.Handler; +import android.os.HandlerThread; +import androidx.annotation.GuardedBy; +import androidx.annotation.NonNull; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.util.IntArrayQueue; +import com.google.android.exoplayer2.util.Util; +import java.util.ArrayDeque; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; + +/** A {@link MediaCodec.Callback} that routes callbacks on a separate thread. */ +@RequiresApi(23) +/* package */ final class AsynchronousMediaCodecCallback extends MediaCodec.Callback { + private final Object lock; + + private final HandlerThread callbackThread; + private @MonotonicNonNull Handler handler; + + @GuardedBy("lock") + private final IntArrayQueue availableInputBuffers; + + @GuardedBy("lock") + private final IntArrayQueue availableOutputBuffers; + + @GuardedBy("lock") + private final ArrayDeque bufferInfos; + + @GuardedBy("lock") + private final ArrayDeque formats; + + @GuardedBy("lock") + @Nullable + private MediaFormat currentFormat; + + @GuardedBy("lock") + @Nullable + private MediaFormat pendingOutputFormat; + + @GuardedBy("lock") + @Nullable + private MediaCodec.CodecException mediaCodecException; + + @GuardedBy("lock") + private long pendingFlushCount; + + @GuardedBy("lock") + private boolean shutDown; + + @GuardedBy("lock") + @Nullable + private IllegalStateException internalException; + + /** + * Creates a new instance. + * + * @param callbackThread The thread that will be used for routing the {@link MediaCodec} + * callbacks. The thread must not be started. + */ + /* package */ AsynchronousMediaCodecCallback(HandlerThread callbackThread) { + this.lock = new Object(); + this.callbackThread = callbackThread; + this.availableInputBuffers = new IntArrayQueue(); + this.availableOutputBuffers = new IntArrayQueue(); + this.bufferInfos = new ArrayDeque<>(); + this.formats = new ArrayDeque<>(); + } + + /** + * Sets the callback on {@code codec} and starts the background callback thread. + * + *

    Make sure to call {@link #shutdown()} to stop the background thread and release its + * resources. + * + * @see MediaCodec#setCallback(MediaCodec.Callback, Handler) + */ + public void initialize(MediaCodec codec) { + checkState(handler == null); + + callbackThread.start(); + Handler handler = new Handler(callbackThread.getLooper()); + codec.setCallback(this, handler); + // Initialize this.handler at the very end ensuring the callback in not considered configured + // if MediaCodec raises an exception. + this.handler = handler; + } + + /** + * Shuts down this instance. + * + *

    This method will stop the callback thread. After calling it, callbacks will no longer be + * handled and dequeue methods will return {@link MediaCodec#INFO_TRY_AGAIN_LATER}. + */ + public void shutdown() { + synchronized (lock) { + shutDown = true; + callbackThread.quit(); + flushInternal(); + } + } + + /** + * Returns the next available input buffer index or {@link MediaCodec#INFO_TRY_AGAIN_LATER} if no + * such buffer exists. + */ + public int dequeueInputBufferIndex() { + synchronized (lock) { + if (isFlushingOrShutdown()) { + return MediaCodec.INFO_TRY_AGAIN_LATER; + } else { + maybeThrowException(); + return availableInputBuffers.isEmpty() + ? MediaCodec.INFO_TRY_AGAIN_LATER + : availableInputBuffers.remove(); + } + } + } + + /** + * Returns the next available output buffer index. If the next available output is a MediaFormat + * change, it will return {@link MediaCodec#INFO_OUTPUT_FORMAT_CHANGED} and you should call {@link + * #getOutputFormat()} to get the format. If there is no available output, this method will return + * {@link MediaCodec#INFO_TRY_AGAIN_LATER}. + */ + public int dequeueOutputBufferIndex(MediaCodec.BufferInfo bufferInfo) { + synchronized (lock) { + if (isFlushingOrShutdown()) { + return MediaCodec.INFO_TRY_AGAIN_LATER; + } else { + maybeThrowException(); + if (availableOutputBuffers.isEmpty()) { + return MediaCodec.INFO_TRY_AGAIN_LATER; + } else { + int bufferIndex = availableOutputBuffers.remove(); + if (bufferIndex >= 0) { + checkStateNotNull(currentFormat); + MediaCodec.BufferInfo nextBufferInfo = bufferInfos.remove(); + bufferInfo.set( + nextBufferInfo.offset, + nextBufferInfo.size, + nextBufferInfo.presentationTimeUs, + nextBufferInfo.flags); + } else if (bufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { + currentFormat = formats.remove(); + } + return bufferIndex; + } + } + } + } + + /** + * Returns the {@link MediaFormat} signalled by the underlying {@link MediaCodec}. + * + *

    Call this after {@link #dequeueOutputBufferIndex} returned {@link + * MediaCodec#INFO_OUTPUT_FORMAT_CHANGED}. + * + * @throws IllegalStateException If called before {@link #dequeueOutputBufferIndex} has returned + * {@link MediaCodec#INFO_OUTPUT_FORMAT_CHANGED}. + */ + public MediaFormat getOutputFormat() { + synchronized (lock) { + if (currentFormat == null) { + throw new IllegalStateException(); + } + return currentFormat; + } + } + + /** + * Initiates a flush asynchronously, which will be completed on the callback thread. When the + * flush is complete, it will trigger {@code onFlushCompleted} from the callback thread. + * + * @param onFlushCompleted A {@link Runnable} that will be called when flush is completed. {@code + * onFlushCompleted} will be called from the scallback thread, therefore it should execute + * synchronized and thread-safe code. + */ + public void flushAsync(Runnable onFlushCompleted) { + synchronized (lock) { + ++pendingFlushCount; + Util.castNonNull(handler).post(() -> this.onFlushCompleted(onFlushCompleted)); + } + } + + // Called from the callback thread. + + @Override + public void onInputBufferAvailable(@NonNull MediaCodec codec, int index) { + synchronized (lock) { + availableInputBuffers.add(index); + } + } + + @Override + public void onOutputBufferAvailable( + @NonNull MediaCodec codec, int index, @NonNull MediaCodec.BufferInfo info) { + synchronized (lock) { + if (pendingOutputFormat != null) { + addOutputFormat(pendingOutputFormat); + pendingOutputFormat = null; + } + availableOutputBuffers.add(index); + bufferInfos.add(info); + } + } + + @Override + public void onError(@NonNull MediaCodec codec, @NonNull MediaCodec.CodecException e) { + synchronized (lock) { + mediaCodecException = e; + } + } + + @Override + public void onOutputFormatChanged(@NonNull MediaCodec codec, @NonNull MediaFormat format) { + synchronized (lock) { + addOutputFormat(format); + pendingOutputFormat = null; + } + } + + private void onFlushCompleted(Runnable onFlushCompleted) { + synchronized (lock) { + onFlushCompletedSynchronized(onFlushCompleted); + } + } + + @GuardedBy("lock") + private void onFlushCompletedSynchronized(Runnable onFlushCompleted) { + if (shutDown) { + return; + } + + --pendingFlushCount; + if (pendingFlushCount > 0) { + // Another flush() has been called. + return; + } else if (pendingFlushCount < 0) { + // This should never happen. + setInternalException(new IllegalStateException()); + return; + } + flushInternal(); + try { + onFlushCompleted.run(); + } catch (IllegalStateException e) { + setInternalException(e); + } catch (Exception e) { + setInternalException(new IllegalStateException(e)); + } + } + + /** Flushes all available input and output buffers and any error that was previously set. */ + @GuardedBy("lock") + private void flushInternal() { + if (!formats.isEmpty()) { + pendingOutputFormat = formats.getLast(); + } else { + // pendingOutputFormat may already be non-null following a previous flush, and remains set in + // this case. + } + availableInputBuffers.clear(); + availableOutputBuffers.clear(); + bufferInfos.clear(); + formats.clear(); + mediaCodecException = null; + } + + @GuardedBy("lock") + private boolean isFlushingOrShutdown() { + return pendingFlushCount > 0 || shutDown; + } + + @GuardedBy("lock") + private void addOutputFormat(MediaFormat mediaFormat) { + availableOutputBuffers.add(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + formats.add(mediaFormat); + } + + @GuardedBy("lock") + private void maybeThrowException() { + maybeThrowInternalException(); + maybeThrowMediaCodecException(); + } + + @GuardedBy("lock") + private void maybeThrowInternalException() { + if (internalException != null) { + IllegalStateException e = internalException; + internalException = null; + throw e; + } + } + + @GuardedBy("lock") + private void maybeThrowMediaCodecException() { + if (mediaCodecException != null) { + MediaCodec.CodecException codecException = mediaCodecException; + mediaCodecException = null; + throw codecException; + } + } + + private void setInternalException(IllegalStateException e) { + synchronized (lock) { + internalException = e; + } + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/BatchBuffer.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/BatchBuffer.java index f770e92a210..a34a3b4f34d 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/BatchBuffer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/BatchBuffer.java @@ -15,172 +15,124 @@ */ package com.google.android.exoplayer2.mediacodec; +import static com.google.android.exoplayer2.util.Assertions.checkArgument; + import androidx.annotation.IntRange; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; -import com.google.android.exoplayer2.util.Assertions; import java.nio.ByteBuffer; -/** Buffer that stores multiple encoded access units to allow batch processing. */ +/** Buffer to which multiple sample buffers can be appended for batch processing */ /* package */ final class BatchBuffer extends DecoderInputBuffer { - /** Arbitrary limit to the number of access unit in a full batch buffer. */ - public static final int DEFAULT_BATCH_SIZE_ACCESS_UNITS = 32; + + /** The default maximum number of samples that can be appended before the buffer is full. */ + public static final int DEFAULT_MAX_SAMPLE_COUNT = 32; /** - * Arbitrary limit to the memory used by a full batch buffer to avoid using too much memory for - * very high bitrate. Equivalent of 75s of mp3 at highest bitrate (320kb/s) and 30s of AAC LC at - * highest bitrate (800kb/s). That limit is ignored for the first access unit to avoid stalling - * stream with huge access units. + * The maximum size of the buffer in bytes. This prevents excessive memory usage for high bitrate + * streams. The limit is equivalent of 75s of mp3 at highest bitrate (320kb/s) and 30s of AAC LC + * at highest bitrate (800kb/s). That limit is ignored for the first sample. */ - private static final int BATCH_SIZE_BYTES = 3 * 1000 * 1024; - - private final DecoderInputBuffer nextAccessUnitBuffer; - private boolean hasPendingAccessUnit; + @VisibleForTesting /* package */ static final int MAX_SIZE_BYTES = 3 * 1000 * 1024; - private long firstAccessUnitTimeUs; - private int accessUnitCount; - private int maxAccessUnitCount; + private long lastSampleTimeUs; + private int sampleCount; + private int maxSampleCount; public BatchBuffer() { super(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); - nextAccessUnitBuffer = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); - clear(); - } - - /** Sets the maximum number of access units the buffer can contain before being full. */ - public void setMaxAccessUnitCount(@IntRange(from = 1) int maxAccessUnitCount) { - Assertions.checkArgument(maxAccessUnitCount > 0); - this.maxAccessUnitCount = maxAccessUnitCount; + maxSampleCount = DEFAULT_MAX_SAMPLE_COUNT; } - /** Gets the maximum number of access units the buffer can contain before being full. */ - public int getMaxAccessUnitCount() { - return maxAccessUnitCount; - } - - /** Resets the state of this object to what it was after construction. */ @Override public void clear() { - flush(); - maxAccessUnitCount = DEFAULT_BATCH_SIZE_ACCESS_UNITS; - } - - /** Clear all access units from the BatchBuffer to empty it. */ - public void flush() { - clearMainBuffer(); - nextAccessUnitBuffer.clear(); - hasPendingAccessUnit = false; + super.clear(); + sampleCount = 0; } - /** Clears the state of the batch buffer to be ready to receive a new sequence of access units. */ - public void batchWasConsumed() { - clearMainBuffer(); - if (hasPendingAccessUnit) { - putAccessUnit(nextAccessUnitBuffer); - hasPendingAccessUnit = false; - } + /** Sets the maximum number of samples that can be appended before the buffer is full. */ + public void setMaxSampleCount(@IntRange(from = 1) int maxSampleCount) { + checkArgument(maxSampleCount > 0); + this.maxSampleCount = maxSampleCount; } /** - * Gets the buffer to fill-out that will then be append to the batch buffer with {@link - * #commitNextAccessUnit()}. + * Returns the timestamp of the first sample in the buffer. The return value is undefined if + * {@link #hasSamples()} is {@code false}. */ - public DecoderInputBuffer getNextAccessUnitBuffer() { - return nextAccessUnitBuffer; - } - - /** Gets the timestamp of the first access unit in the buffer. */ - public long getFirstAccessUnitTimeUs() { - return firstAccessUnitTimeUs; - } - - /** Gets the timestamp of the last access unit in the buffer. */ - public long getLastAccessUnitTimeUs() { + public long getFirstSampleTimeUs() { return timeUs; } - /** Gets the number of access units contained in this batch buffer. */ - public int getAccessUnitCount() { - return accessUnitCount; + /** + * Returns the timestamp of the last sample in the buffer. The return value is undefined if {@link + * #hasSamples()} is {@code false}. + */ + public long getLastSampleTimeUs() { + return lastSampleTimeUs; } - /** If the buffer contains no access units. */ - public boolean isEmpty() { - return accessUnitCount == 0; + /** Returns the number of samples in the buffer. */ + public int getSampleCount() { + return sampleCount; } - /** If more access units should be added to the batch buffer. */ - public boolean isFull() { - return accessUnitCount >= maxAccessUnitCount - || (data != null && data.position() >= BATCH_SIZE_BYTES) - || hasPendingAccessUnit; + /** Returns whether the buffer contains one or more samples. */ + public boolean hasSamples() { + return sampleCount > 0; } /** - * Appends the staged access unit in this batch buffer. + * Attempts to append the provided buffer. * - * @throws IllegalStateException If calling this method on a full or end of stream batch buffer. - * @throws IllegalArgumentException If the {@code accessUnit} is encrypted or has - * supplementalData, as batching of those state has not been implemented. + * @param buffer The buffer to try and append. + * @return Whether the buffer was successfully appended. + * @throws IllegalArgumentException If the {@code buffer} is encrypted, has supplemental data, or + * is an end of stream buffer, none of which are supported. */ - public void commitNextAccessUnit() { - DecoderInputBuffer accessUnit = nextAccessUnitBuffer; - Assertions.checkState(!isFull() && !isEndOfStream()); - Assertions.checkArgument(!accessUnit.isEncrypted() && !accessUnit.hasSupplementalData()); - if (!canBatch(accessUnit)) { - hasPendingAccessUnit = true; // Delay the putAccessUnit until the batch buffer is empty. - return; + public boolean append(DecoderInputBuffer buffer) { + checkArgument(!buffer.isEncrypted()); + checkArgument(!buffer.hasSupplementalData()); + checkArgument(!buffer.isEndOfStream()); + if (!canAppendSampleBuffer(buffer)) { + return false; } - putAccessUnit(accessUnit); - } - - private boolean canBatch(DecoderInputBuffer accessUnit) { - if (isEmpty()) { - return true; // Batching with an empty batch must always succeed or the stream will stall. + if (sampleCount++ == 0) { + timeUs = buffer.timeUs; + if (buffer.isKeyFrame()) { + setFlags(C.BUFFER_FLAG_KEY_FRAME); + } } - if (accessUnit.isDecodeOnly() != isDecodeOnly()) { - return false; // Decode only and non decode only access units can not be batched together. + if (buffer.isDecodeOnly()) { + setFlags(C.BUFFER_FLAG_DECODE_ONLY); } - - @Nullable ByteBuffer accessUnitData = accessUnit.data; - if (accessUnitData != null - && this.data != null - && this.data.position() + accessUnitData.limit() >= BATCH_SIZE_BYTES) { - return false; // The batch buffer does not have the capacity to add this access unit. + @Nullable ByteBuffer bufferData = buffer.data; + if (bufferData != null) { + ensureSpaceForWrite(bufferData.remaining()); + data.put(bufferData); } + lastSampleTimeUs = buffer.timeUs; return true; } - private void putAccessUnit(DecoderInputBuffer accessUnit) { - if (accessUnit.isEndOfStream()) { - setFlags(C.BUFFER_FLAG_END_OF_STREAM); - } else { - timeUs = accessUnit.timeUs; - if (accessUnit.isDecodeOnly()) { - setFlags(C.BUFFER_FLAG_DECODE_ONLY); - } - if (accessUnit.isKeyFrame()) { - setFlags(C.BUFFER_FLAG_KEY_FRAME); - } - @Nullable ByteBuffer accessUnitData = accessUnit.data; - if (accessUnitData != null) { - accessUnit.flip(); - ensureSpaceForWrite(accessUnitData.remaining()); - this.data.put(accessUnitData); - } - accessUnitCount++; - if (accessUnitCount == 1) { - firstAccessUnitTimeUs = timeUs; - } + private boolean canAppendSampleBuffer(DecoderInputBuffer buffer) { + if (!hasSamples()) { + // Always allow appending when the buffer is empty, else no progress can be made. + return true; } - accessUnit.clear(); - } - - private void clearMainBuffer() { - super.clear(); - accessUnitCount = 0; - firstAccessUnitTimeUs = C.TIME_UNSET; - timeUs = C.TIME_UNSET; + if (sampleCount >= maxSampleCount) { + return false; + } + if (buffer.isDecodeOnly() != isDecodeOnly()) { + return false; + } + @Nullable ByteBuffer bufferData = buffer.data; + if (bufferData != null + && data != null + && data.position() + bufferData.remaining() > MAX_SIZE_BYTES) { + return false; + } + return true; } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/C2Mp3TimestampTracker.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/C2Mp3TimestampTracker.java index 0c3fe9facf5..04b453d529e 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/C2Mp3TimestampTracker.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/C2Mp3TimestampTracker.java @@ -30,7 +30,7 @@ /* package */ final class C2Mp3TimestampTracker { // Mirroring the actual codec, as can be found at - // https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/codec2/components/mp3/C2SoftMp3Dec.h;l=55;drc=3665390c9d32a917398b240c5a46ced07a3b65eb + // https://cs.android.com/android/platform/superproject/+/main:frameworks/av/media/codec2/components/mp3/C2SoftMp3Dec.h;l=55;drc=3665390c9d32a917398b240c5a46ced07a3b65eb private static final long DECODER_DELAY_SAMPLES = 529; private static final String TAG = "C2Mp3TimestampTracker"; @@ -76,7 +76,7 @@ public long updateAndGetPresentationTimeUs(Format format, DecoderInputBuffer buf } // These calculations mirror the timestamp calculations in the Codec2 Mp3 Decoder. - // https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/codec2/components/mp3/C2SoftMp3Dec.cpp;l=464;drc=ed134640332fea70ca4b05694289d91a5265bb46 + // https://cs.android.com/android/platform/superproject/+/main:frameworks/av/media/codec2/components/mp3/C2SoftMp3Dec.cpp;l=464;drc=ed134640332fea70ca4b05694289d91a5265bb46 if (processedSamples == 0) { anchorTimestampUs = buffer.timeUs; processedSamples = frameCount - DECODER_DELAY_SAMPLES; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecAdapter.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecAdapter.java index 69875f23674..53b7928b477 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecAdapter.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecAdapter.java @@ -19,19 +19,42 @@ import android.media.MediaCodec; import android.media.MediaCrypto; import android.media.MediaFormat; +import android.os.Bundle; +import android.os.Handler; import android.view.Surface; import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.decoder.CryptoInfo; -import com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.MediaCodecOperationMode; +import java.nio.ByteBuffer; /** * Abstracts {@link MediaCodec} operations. * *

    {@code MediaCodecAdapter} offers a common interface to interact with a {@link MediaCodec} - * regardless of the {@link MediaCodecOperationMode} the {@link MediaCodec} is operating in. + * regardless of the mode the {@link MediaCodec} is operating in. */ public interface MediaCodecAdapter { + /** A factory for {@link MediaCodecAdapter} instances. */ + interface Factory { + + /** Default factory used in most cases. */ + Factory DEFAULT = new SynchronousMediaCodecAdapter.Factory(); + + /** Creates an instance wrapping the provided {@link MediaCodec} instance. */ + MediaCodecAdapter createAdapter(MediaCodec codec); + } + + /** + * Listener to be called when an output frame has rendered on the output surface. + * + * @see MediaCodec.OnFrameRenderedListener + */ + interface OnFrameRenderedListener { + void onFrameRendered(MediaCodecAdapter codec, long presentationTimeUs, long nanoTime); + } + /** * Configures this adapter and the underlying {@link MediaCodec}. Needs to be called before {@link * #start()}. @@ -78,6 +101,22 @@ void configure( */ MediaFormat getOutputFormat(); + /** + * Returns a writable ByteBuffer object for a dequeued input buffer index. + * + * @see MediaCodec#getInputBuffer(int) + */ + @Nullable + ByteBuffer getInputBuffer(int index); + + /** + * Returns a read-only ByteBuffer for a dequeued output buffer index. + * + * @see MediaCodec#getOutputBuffer(int) + */ + @Nullable + ByteBuffer getOutputBuffer(int index); + /** * Submit an input buffer for decoding. * @@ -102,18 +141,61 @@ void configure( void queueSecureInputBuffer( int index, int offset, CryptoInfo info, long presentationTimeUs, int flags); - /** Flushes both the adapter and the underlying {@link MediaCodec}. */ + /** + * Returns the buffer to the {@link MediaCodec}. If the {@link MediaCodec} was configured with an + * output surface, setting {@code render} to {@code true} will first send the buffer to the output + * surface. The surface will release the buffer back to the codec once it is no longer + * used/displayed. + * + * @see MediaCodec#releaseOutputBuffer(int, boolean) + */ + void releaseOutputBuffer(int index, boolean render); + + /** + * Updates the output buffer's surface timestamp and sends it to the {@link MediaCodec} to render + * it on the output surface. If the {@link MediaCodec} is not configured with an output surface, + * this call will simply return the buffer to the {@link MediaCodec}. + * + * @see MediaCodec#releaseOutputBuffer(int, long) + */ + @RequiresApi(21) + void releaseOutputBuffer(int index, long renderTimeStampNs); + + /** Flushes the adapter and the underlying {@link MediaCodec}. */ void flush(); + /** Releases the adapter and the underlying {@link MediaCodec}. */ + void release(); + /** - * Shuts down the adapter. + * Registers a callback to be invoked when an output frame is rendered on the output surface. * - *

    This method does not stop or release the underlying {@link MediaCodec}. It should be called - * before stopping or releasing the {@link MediaCodec} to avoid the possibility of the adapter - * interacting with a stopped or released {@link MediaCodec}. + * @see MediaCodec#setOnFrameRenderedListener */ - void shutdown(); + @RequiresApi(23) + void setOnFrameRenderedListener(OnFrameRenderedListener listener, Handler handler); - /** Returns the {@link MediaCodec} instance of this adapter. */ - MediaCodec getCodec(); + /** + * Dynamically sets the output surface of a {@link MediaCodec}. + * + * @see MediaCodec#setOutputSurface(Surface) + */ + @RequiresApi(23) + void setOutputSurface(Surface surface); + + /** + * Communicate additional parameter changes to the {@link MediaCodec} instance. + * + * @see MediaCodec#setParameters(Bundle) + */ + @RequiresApi(19) + void setParameters(Bundle params); + + /** + * Specifies the scaling mode to use, if a surface has been specified in a previous call to {@link + * #configure}. + * + * @see MediaCodec#setVideoScalingMode(int) + */ + void setVideoScalingMode(@C.VideoScalingMode int scalingMode); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecAsyncCallback.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecAsyncCallback.java deleted file mode 100644 index 65f0c266a9b..00000000000 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecAsyncCallback.java +++ /dev/null @@ -1,157 +0,0 @@ -/* - * Copyright (C) 2019 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package com.google.android.exoplayer2.mediacodec; - -import android.media.MediaCodec; -import android.media.MediaFormat; -import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; -import androidx.annotation.VisibleForTesting; -import com.google.android.exoplayer2.util.IntArrayQueue; -import java.util.ArrayDeque; - -/** Handles the asynchronous callbacks from {@link android.media.MediaCodec.Callback}. */ -@RequiresApi(21) -/* package */ final class MediaCodecAsyncCallback extends MediaCodec.Callback { - private final IntArrayQueue availableInputBuffers; - private final IntArrayQueue availableOutputBuffers; - private final ArrayDeque bufferInfos; - private final ArrayDeque formats; - @Nullable private MediaFormat currentFormat; - @Nullable private MediaFormat pendingOutputFormat; - @Nullable private IllegalStateException mediaCodecException; - - /** Creates a new MediaCodecAsyncCallback. */ - public MediaCodecAsyncCallback() { - availableInputBuffers = new IntArrayQueue(); - availableOutputBuffers = new IntArrayQueue(); - bufferInfos = new ArrayDeque<>(); - formats = new ArrayDeque<>(); - } - - /** - * Returns the next available input buffer index or {@link MediaCodec#INFO_TRY_AGAIN_LATER} if no - * such buffer exists. - */ - public int dequeueInputBufferIndex() { - return availableInputBuffers.isEmpty() - ? MediaCodec.INFO_TRY_AGAIN_LATER - : availableInputBuffers.remove(); - } - - /** - * Returns the next available output buffer index. If the next available output is a MediaFormat - * change, it will return {@link MediaCodec#INFO_OUTPUT_FORMAT_CHANGED} and you should call {@link - * #getOutputFormat()} to get the format. If there is no available output, this method will return - * {@link MediaCodec#INFO_TRY_AGAIN_LATER}. - */ - public int dequeueOutputBufferIndex(MediaCodec.BufferInfo bufferInfo) { - if (availableOutputBuffers.isEmpty()) { - return MediaCodec.INFO_TRY_AGAIN_LATER; - } else { - int bufferIndex = availableOutputBuffers.remove(); - if (bufferIndex >= 0) { - MediaCodec.BufferInfo nextBufferInfo = bufferInfos.remove(); - bufferInfo.set( - nextBufferInfo.offset, - nextBufferInfo.size, - nextBufferInfo.presentationTimeUs, - nextBufferInfo.flags); - } else if (bufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { - currentFormat = formats.remove(); - } - return bufferIndex; - } - } - - /** - * Returns the {@link MediaFormat} signalled by the underlying {@link MediaCodec}. - * - *

    Call this after {@link #dequeueOutputBufferIndex} returned {@link - * MediaCodec#INFO_OUTPUT_FORMAT_CHANGED}. - * - * @throws IllegalStateException If called before {@link #dequeueOutputBufferIndex} has returned - * {@link MediaCodec#INFO_OUTPUT_FORMAT_CHANGED}. - */ - public MediaFormat getOutputFormat() throws IllegalStateException { - if (currentFormat == null) { - throw new IllegalStateException(); - } - return currentFormat; - } - - /** - * Checks and throws an {@link IllegalStateException} if an error was previously set on this - * instance via {@link #onError}. - */ - public void maybeThrowMediaCodecException() throws IllegalStateException { - IllegalStateException exception = mediaCodecException; - mediaCodecException = null; - if (exception != null) { - throw exception; - } - } - - /** - * Flushes the MediaCodecAsyncCallback. This method removes all available input and output buffers - * and any error that was previously set. - */ - public void flush() { - pendingOutputFormat = formats.isEmpty() ? null : formats.getLast(); - availableInputBuffers.clear(); - availableOutputBuffers.clear(); - bufferInfos.clear(); - formats.clear(); - mediaCodecException = null; - } - - @Override - public void onInputBufferAvailable(MediaCodec mediaCodec, int index) { - availableInputBuffers.add(index); - } - - @Override - public void onOutputBufferAvailable( - MediaCodec mediaCodec, int index, MediaCodec.BufferInfo bufferInfo) { - if (pendingOutputFormat != null) { - addOutputFormat(pendingOutputFormat); - pendingOutputFormat = null; - } - availableOutputBuffers.add(index); - bufferInfos.add(bufferInfo); - } - - @Override - public void onError(MediaCodec mediaCodec, MediaCodec.CodecException e) { - onMediaCodecError(e); - } - - @Override - public void onOutputFormatChanged(MediaCodec mediaCodec, MediaFormat mediaFormat) { - addOutputFormat(mediaFormat); - pendingOutputFormat = null; - } - - @VisibleForTesting() - void onMediaCodecError(IllegalStateException e) { - mediaCodecException = e; - } - - private void addOutputFormat(MediaFormat mediaFormat) { - availableOutputBuffers.add(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); - formats.add(mediaFormat); - } -} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecInfo.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecInfo.java index 404066e96d8..99396af2f4d 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecInfo.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecInfo.java @@ -15,6 +15,20 @@ */ package com.google.android.exoplayer2.mediacodec; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_AUDIO_CHANNEL_COUNT_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_AUDIO_ENCODING_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_AUDIO_SAMPLE_RATE_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_INITIALIZATION_DATA_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_MIME_TYPE_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_COLOR_INFO_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_RESOLUTION_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_ROTATION_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_WORKAROUND; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_NO; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITHOUT_RECONFIGURATION; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITH_FLUSH; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITH_RECONFIGURATION; + import android.graphics.Point; import android.media.MediaCodec; import android.media.MediaCodecInfo.AudioCapabilities; @@ -24,7 +38,11 @@ import android.util.Pair; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DecoderDiscardReasons; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DecoderReuseResult; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.MimeTypes; @@ -44,8 +62,8 @@ public final class MediaCodecInfo { /** * The name of the decoder. - *

    - * May be passed to {@link MediaCodec#createByCodecName(String)} to create an instance of the + * + *

    May be passed to {@link MediaCodec#createByCodecName(String)} to create an instance of the * decoder. */ public final String name; @@ -152,11 +170,16 @@ public static MediaCodecInfo newInstance( hardwareAccelerated, softwareOnly, vendor, - forceDisableAdaptive, - forceSecure); + /* adaptive= */ !forceDisableAdaptive + && capabilities != null + && isAdaptive(capabilities) + && !needsDisableAdaptationWorkaround(name), + /* tunneling= */ capabilities != null && isTunneling(capabilities), + /* secure= */ forceSecure || (capabilities != null && isSecure(capabilities))); } - private MediaCodecInfo( + @VisibleForTesting + /* package */ MediaCodecInfo( String name, String mimeType, String codecMimeType, @@ -164,8 +187,9 @@ private MediaCodecInfo( boolean hardwareAccelerated, boolean softwareOnly, boolean vendor, - boolean forceDisableAdaptive, - boolean forceSecure) { + boolean adaptive, + boolean tunneling, + boolean secure) { this.name = Assertions.checkNotNull(name); this.mimeType = mimeType; this.codecMimeType = codecMimeType; @@ -173,9 +197,9 @@ private MediaCodecInfo( this.hardwareAccelerated = hardwareAccelerated; this.softwareOnly = softwareOnly; this.vendor = vendor; - adaptive = !forceDisableAdaptive && capabilities != null && isAdaptive(capabilities); - tunneling = capabilities != null && isTunneling(capabilities); - secure = forceSecure || (capabilities != null && isSecure(capabilities)); + this.adaptive = adaptive; + this.tunneling = tunneling; + this.secure = secure; isVideo = MimeTypes.isVideo(mimeType); } @@ -274,8 +298,16 @@ public boolean isCodecSupported(Format format) { // which may not be widely supported. See https://github.com/google/ExoPlayer/issues/5145. return true; } - for (CodecProfileLevel capabilities : getProfileLevels()) { - if (capabilities.profile == profile && capabilities.level >= level) { + + CodecProfileLevel[] profileLevels = getProfileLevels(); + if (Util.SDK_INT <= 23 && MimeTypes.VIDEO_VP9.equals(mimeType) && profileLevels.length == 0) { + // Some older devices don't report profile levels for VP9. Estimate them using other data in + // the codec capabilities. + profileLevels = estimateLegacyVp9ProfileLevels(capabilities); + } + + for (CodecProfileLevel profileLevel : profileLevels) { + if (profileLevel.profile == profile && profileLevel.level >= level) { return true; } } @@ -296,11 +328,12 @@ public boolean isHdr10PlusOutOfBandMetadataSupported() { } /** - * Returns whether it may be possible to adapt to playing a different format when the codec is - * configured to play media in the specified {@code format}. For adaptation to succeed, the codec - * must also be configured with appropriate maximum values and {@link - * #isSeamlessAdaptationSupported(Format, Format, boolean)} must return {@code true} for the - * old/new formats. + * Returns whether it may be possible to adapt an instance of this decoder to playing a different + * format when the codec is configured to play media in the specified {@code format}. + * + *

    For adaptation to succeed, the codec must also be configured with appropriate maximum values + * and {@link #isSeamlessAdaptationSupported(Format, Format, boolean)} must return {@code true} + * for the old/new formats. * * @param format The format of media for which the decoder will be configured. * @return Whether adaptation may be possible @@ -309,53 +342,129 @@ public boolean isSeamlessAdaptationSupported(Format format) { if (isVideo) { return adaptive; } else { - Pair codecProfileLevel = MediaCodecUtil.getCodecProfileAndLevel(format); - return codecProfileLevel != null && codecProfileLevel.first == CodecProfileLevel.AACObjectXHE; + Pair profileLevel = MediaCodecUtil.getCodecProfileAndLevel(format); + return profileLevel != null && profileLevel.first == CodecProfileLevel.AACObjectXHE; } } /** - * Returns whether it is possible to adapt the decoder seamlessly from {@code oldFormat} to {@code - * newFormat}. If {@code newFormat} may not be completely populated, pass {@code false} for {@code - * isNewFormatComplete}. + * Returns whether it is possible to adapt an instance of this decoder seamlessly from {@code + * oldFormat} to {@code newFormat}. If {@code newFormat} may not be completely populated, pass + * {@code false} for {@code isNewFormatComplete}. + * + *

    For adaptation to succeed, the codec must also be configured with maximum values that are + * compatible with the new format. * * @param oldFormat The format being decoded. * @param newFormat The new format. * @param isNewFormatComplete Whether {@code newFormat} is populated with format-specific * metadata. * @return Whether it is possible to adapt the decoder seamlessly. + * @deprecated Use {@link #canReuseCodec}. */ + @Deprecated public boolean isSeamlessAdaptationSupported( Format oldFormat, Format newFormat, boolean isNewFormatComplete) { + if (!isNewFormatComplete && oldFormat.colorInfo != null && newFormat.colorInfo == null) { + newFormat = newFormat.buildUpon().setColorInfo(oldFormat.colorInfo).build(); + } + @DecoderReuseResult int reuseResult = canReuseCodec(oldFormat, newFormat).result; + return reuseResult == REUSE_RESULT_YES_WITH_RECONFIGURATION + || reuseResult == REUSE_RESULT_YES_WITHOUT_RECONFIGURATION; + } + + /** + * Evaluates whether it's possible to reuse an instance of this decoder that's currently decoding + * {@code oldFormat} to decode {@code newFormat} instead. + * + *

    For adaptation to succeed, the codec must also be configured with maximum values that are + * compatible with the new format. + * + * @param oldFormat The format being decoded. + * @param newFormat The new format. + * @return The result of the evaluation. + */ + public DecoderReuseEvaluation canReuseCodec(Format oldFormat, Format newFormat) { + @DecoderDiscardReasons int discardReasons = 0; + if (!Util.areEqual(oldFormat.sampleMimeType, newFormat.sampleMimeType)) { + discardReasons |= DISCARD_REASON_MIME_TYPE_CHANGED; + } + if (isVideo) { - return Assertions.checkNotNull(oldFormat.sampleMimeType).equals(newFormat.sampleMimeType) - && oldFormat.rotationDegrees == newFormat.rotationDegrees - && (adaptive - || (oldFormat.width == newFormat.width && oldFormat.height == newFormat.height)) - && ((!isNewFormatComplete && newFormat.colorInfo == null) - || Util.areEqual(oldFormat.colorInfo, newFormat.colorInfo)); + if (oldFormat.rotationDegrees != newFormat.rotationDegrees) { + discardReasons |= DISCARD_REASON_VIDEO_ROTATION_CHANGED; + } + if (!adaptive + && (oldFormat.width != newFormat.width || oldFormat.height != newFormat.height)) { + discardReasons |= DISCARD_REASON_VIDEO_RESOLUTION_CHANGED; + } + if (!Util.areEqual(oldFormat.colorInfo, newFormat.colorInfo)) { + discardReasons |= DISCARD_REASON_VIDEO_COLOR_INFO_CHANGED; + } + if (needsAdaptationReconfigureWorkaround(name) + && !oldFormat.initializationDataEquals(newFormat)) { + discardReasons |= DISCARD_REASON_WORKAROUND; + } + + if (discardReasons == 0) { + return new DecoderReuseEvaluation( + name, + oldFormat, + newFormat, + oldFormat.initializationDataEquals(newFormat) + ? REUSE_RESULT_YES_WITHOUT_RECONFIGURATION + : REUSE_RESULT_YES_WITH_RECONFIGURATION, + /* discardReasons= */ 0); + } } else { - if (!MimeTypes.AUDIO_AAC.equals(mimeType) - || !Assertions.checkNotNull(oldFormat.sampleMimeType).equals(newFormat.sampleMimeType) - || oldFormat.channelCount != newFormat.channelCount - || oldFormat.sampleRate != newFormat.sampleRate) { - return false; + if (oldFormat.channelCount != newFormat.channelCount) { + discardReasons |= DISCARD_REASON_AUDIO_CHANNEL_COUNT_CHANGED; } - // Check the codec profile levels support adaptation. - @Nullable - Pair oldCodecProfileLevel = - MediaCodecUtil.getCodecProfileAndLevel(oldFormat); - @Nullable - Pair newCodecProfileLevel = - MediaCodecUtil.getCodecProfileAndLevel(newFormat); - if (oldCodecProfileLevel == null || newCodecProfileLevel == null) { - return false; + if (oldFormat.sampleRate != newFormat.sampleRate) { + discardReasons |= DISCARD_REASON_AUDIO_SAMPLE_RATE_CHANGED; + } + if (oldFormat.pcmEncoding != newFormat.pcmEncoding) { + discardReasons |= DISCARD_REASON_AUDIO_ENCODING_CHANGED; + } + + // Check whether we're adapting between two xHE-AAC formats, for which adaptation is possible + // without reconfiguration or flushing. + if (discardReasons == 0 && MimeTypes.AUDIO_AAC.equals(mimeType)) { + @Nullable + Pair oldCodecProfileLevel = + MediaCodecUtil.getCodecProfileAndLevel(oldFormat); + @Nullable + Pair newCodecProfileLevel = + MediaCodecUtil.getCodecProfileAndLevel(newFormat); + if (oldCodecProfileLevel != null && newCodecProfileLevel != null) { + int oldProfile = oldCodecProfileLevel.first; + int newProfile = newCodecProfileLevel.first; + if (oldProfile == CodecProfileLevel.AACObjectXHE + && newProfile == CodecProfileLevel.AACObjectXHE) { + return new DecoderReuseEvaluation( + name, + oldFormat, + newFormat, + REUSE_RESULT_YES_WITHOUT_RECONFIGURATION, + /* discardReasons= */ 0); + } + } + } + + if (!oldFormat.initializationDataEquals(newFormat)) { + discardReasons |= DISCARD_REASON_INITIALIZATION_DATA_CHANGED; + } + if (needsAdaptationFlushWorkaround(mimeType)) { + discardReasons |= DISCARD_REASON_WORKAROUND; + } + + if (discardReasons == 0) { + return new DecoderReuseEvaluation( + name, oldFormat, newFormat, REUSE_RESULT_YES_WITH_FLUSH, /* discardReasons= */ 0); } - int oldProfile = oldCodecProfileLevel.first; - int newProfile = newCodecProfileLevel.first; - return oldProfile == CodecProfileLevel.AACObjectXHE - && newProfile == CodecProfileLevel.AACObjectXHE; } + + return new DecoderReuseEvaluation(name, oldFormat, newFormat, REUSE_RESULT_NO, discardReasons); } /** @@ -382,7 +491,7 @@ public boolean isVideoSizeAndRateSupportedV21(int width, int height, double fram } if (!areSizeAndRateSupportedV21(videoCapabilities, width, height, frameRate)) { if (width >= height - || !enableRotatedVerticalResolutionWorkaround(name) + || !needsRotatedVerticalResolutionWorkaround(name) || !areSizeAndRateSupportedV21(videoCapabilities, height, width, frameRate)) { logNoSupport("sizeAndRate.support, " + width + "x" + height + "x" + frameRate); return false; @@ -577,6 +686,100 @@ private static int getMaxSupportedInstancesV23(CodecCapabilities capabilities) { return capabilities.getMaxSupportedInstances(); } + /** + * Called on devices with {@link Util#SDK_INT} 23 and below, for VP9 decoders whose {@link + * CodecCapabilities} do not correctly report profile levels. The returned {@link + * CodecProfileLevel CodecProfileLevels} are estimated based on other data in the {@link + * CodecCapabilities}. + * + * @param capabilities The {@link CodecCapabilities} for a VP9 decoder, or {@code null} if not + * known. + * @return The estimated {@link CodecProfileLevel CodecProfileLevels} for the decoder. + */ + private static CodecProfileLevel[] estimateLegacyVp9ProfileLevels( + @Nullable CodecCapabilities capabilities) { + int maxBitrate = 0; + if (capabilities != null) { + @Nullable VideoCapabilities videoCapabilities = capabilities.getVideoCapabilities(); + if (videoCapabilities != null) { + maxBitrate = videoCapabilities.getBitrateRange().getUpper(); + } + } + + // Values taken from https://www.webmproject.org/vp9/levels. + int level; + if (maxBitrate >= 180_000_000) { + level = CodecProfileLevel.VP9Level52; + } else if (maxBitrate >= 120_000_000) { + level = CodecProfileLevel.VP9Level51; + } else if (maxBitrate >= 60_000_000) { + level = CodecProfileLevel.VP9Level5; + } else if (maxBitrate >= 30_000_000) { + level = CodecProfileLevel.VP9Level41; + } else if (maxBitrate >= 18_000_000) { + level = CodecProfileLevel.VP9Level4; + } else if (maxBitrate >= 12_000_000) { + level = CodecProfileLevel.VP9Level31; + } else if (maxBitrate >= 7_200_000) { + level = CodecProfileLevel.VP9Level3; + } else if (maxBitrate >= 3_600_000) { + level = CodecProfileLevel.VP9Level21; + } else if (maxBitrate >= 1_800_000) { + level = CodecProfileLevel.VP9Level2; + } else if (maxBitrate >= 800_000) { + level = CodecProfileLevel.VP9Level11; + } else { // Assume level 1 is always supported. + level = CodecProfileLevel.VP9Level1; + } + + CodecProfileLevel profileLevel = new CodecProfileLevel(); + // Since this method is for legacy devices only, assume that only profile 0 is supported. + profileLevel.profile = CodecProfileLevel.VP9Profile0; + profileLevel.level = level; + + return new CodecProfileLevel[] {profileLevel}; + } + + /** + * Returns whether the decoder is known to fail when adapting, despite advertising itself as an + * adaptive decoder. + * + * @param name The decoder name. + * @return True if the decoder is known to fail when adapting. + */ + private static boolean needsDisableAdaptationWorkaround(String name) { + return Util.SDK_INT <= 22 + && ("ODROID-XU3".equals(Util.MODEL) || "Nexus 10".equals(Util.MODEL)) + && ("OMX.Exynos.AVC.Decoder".equals(name) || "OMX.Exynos.AVC.Decoder.secure".equals(name)); + } + + /** + * Returns whether the decoder is known to fail when an attempt is made to reconfigure it with a + * new format's configuration data. + * + * @param name The name of the decoder. + * @return Whether the decoder is known to fail when an attempt is made to reconfigure it with a + * new format's configuration data. + */ + private static boolean needsAdaptationReconfigureWorkaround(String name) { + return Util.MODEL.startsWith("SM-T230") && "OMX.MARVELL.VIDEO.HW.CODA7542DECODER".equals(name); + } + + /** + * Returns whether the decoder is known to behave incorrectly if flushed to adapt to a new format. + * + * @param mimeType The name of the MIME type. + * @return Whether the decoder is known to to behave incorrectly if flushed to adapt to a new + * format. + */ + private static boolean needsAdaptationFlushWorkaround(String mimeType) { + // For Opus, we don't flush and reuse the codec because the decoder may discard samples after + // flushing, which would result in audio being dropped just after a stream change (see + // [Internal: b/143450854]). For other formats, we allow reuse after flushing if the codec + // initialization data is unchanged. + return MimeTypes.AUDIO_OPUS.equals(mimeType); + } + /** * Capabilities are known to be inaccurately reported for vertical resolutions on some devices. * [Internal ref: b/31387661]. When this workaround is enabled, we also check whether the @@ -586,7 +789,7 @@ private static int getMaxSupportedInstancesV23(CodecCapabilities capabilities) { * @param name The name of the codec. * @return Whether to enable the workaround. */ - private static final boolean enableRotatedVerticalResolutionWorkaround(String name) { + private static final boolean needsRotatedVerticalResolutionWorkaround(String name) { if ("OMX.MTK.VIDEO.DECODER.HEVC".equals(name) && "mcv5a".equals(Util.DEVICE)) { // See https://github.com/google/ExoPlayer/issues/6612. return false; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecInputBufferEnqueuer.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecInputBufferEnqueuer.java deleted file mode 100644 index 34a1ccc6bab..00000000000 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecInputBufferEnqueuer.java +++ /dev/null @@ -1,56 +0,0 @@ -/* - * Copyright (C) 2020 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.google.android.exoplayer2.mediacodec; - -import android.media.MediaCodec; -import com.google.android.exoplayer2.decoder.CryptoInfo; - -/** Abstracts operations to enqueue input buffer on a {@link android.media.MediaCodec}. */ -interface MediaCodecInputBufferEnqueuer { - - /** - * Starts this instance. - * - *

    Call this method after creating an instance. - */ - void start(); - - /** - * Submits an input buffer for decoding. - * - * @see android.media.MediaCodec#queueInputBuffer - */ - void queueInputBuffer(int index, int offset, int size, long presentationTimeUs, int flags); - - /** - * Submits an input buffer that potentially contains encrypted data for decoding. - * - *

    Note: This method behaves as {@link MediaCodec#queueSecureInputBuffer} with the difference - * that {@code info} is of type {@link CryptoInfo} and not {@link - * android.media.MediaCodec.CryptoInfo}. - * - * @see android.media.MediaCodec#queueSecureInputBuffer - */ - void queueSecureInputBuffer( - int index, int offset, CryptoInfo info, long presentationTimeUs, int flags); - - /** Flushes the instance. */ - void flush(); - - /** Shut down the instance. Make sure to call this method to release its internal resources. */ - void shutdown(); -} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecRenderer.java index 75ee7f5c818..1e4506c7957 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecRenderer.java @@ -15,6 +15,15 @@ */ package com.google.android.exoplayer2.mediacodec; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_DRM_SESSION_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_OPERATING_RATE_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_REUSE_NOT_IMPLEMENTED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_WORKAROUND; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_NO; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITHOUT_RECONFIGURATION; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITH_FLUSH; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITH_RECONFIGURATION; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; import static com.google.android.exoplayer2.util.Assertions.checkState; import static java.lang.Math.max; @@ -39,6 +48,8 @@ import com.google.android.exoplayer2.FormatHolder; import com.google.android.exoplayer2.decoder.DecoderCounters; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DecoderDiscardReasons; import com.google.android.exoplayer2.drm.DrmSession; import com.google.android.exoplayer2.drm.DrmSession.DrmSessionException; import com.google.android.exoplayer2.drm.ExoMediaCrypto; @@ -54,10 +65,8 @@ import com.google.android.exoplayer2.util.TraceUtil; import com.google.android.exoplayer2.util.Util; import java.lang.annotation.Documented; -import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; -import java.lang.annotation.Target; import java.nio.ByteBuffer; import java.nio.ByteOrder; import java.util.ArrayDeque; @@ -69,44 +78,6 @@ */ public abstract class MediaCodecRenderer extends BaseRenderer { - /** - * The modes to operate the {@link MediaCodec}. - * - *

    Allowed values: - * - *

      - *
    • {@link #OPERATION_MODE_SYNCHRONOUS} - *
    • {@link #OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD} - *
    • {@link #OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING} - *
    - */ - @Documented - @Retention(RetentionPolicy.SOURCE) - @Target({ElementType.TYPE_PARAMETER, ElementType.TYPE_USE}) - @IntDef({ - OPERATION_MODE_SYNCHRONOUS, - OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD, - OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING, - }) - public @interface MediaCodecOperationMode {} - - // TODO: Refactor these constants once internal evaluation completed. - // Do not assign values 1, 3 and 5 to a new operation mode until the evaluation is completed, - // otherwise existing clients may operate one of the dropped modes. - // [Internal ref: b/132684114] - /** Operates the {@link MediaCodec} in synchronous mode. */ - public static final int OPERATION_MODE_SYNCHRONOUS = 0; - /** - * Operates the {@link MediaCodec} in asynchronous mode and routes {@link MediaCodec.Callback} - * callbacks to a dedicated thread. - */ - public static final int OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD = 2; - /** - * Same as {@link #OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD}, and offloads queueing to another - * thread. - */ - public static final int OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING = 4; - /** Thrown when a failure occurs instantiating a decoder. */ public static class DecoderInitializationException extends Exception { @@ -232,31 +203,6 @@ private static String buildCustomDiagnosticInfo(int errorCode) { // pending output streams that have fewer frames than the codec latency. private static final int MAX_PENDING_OUTPUT_STREAM_OFFSET_COUNT = 10; - /** - * The possible return values for {@link #canKeepCodec(MediaCodec, MediaCodecInfo, Format, - * Format)}. - */ - @Documented - @Retention(RetentionPolicy.SOURCE) - @IntDef({ - KEEP_CODEC_RESULT_NO, - KEEP_CODEC_RESULT_YES_WITH_FLUSH, - KEEP_CODEC_RESULT_YES_WITH_RECONFIGURATION, - KEEP_CODEC_RESULT_YES_WITHOUT_RECONFIGURATION - }) - protected @interface KeepCodecResult {} - /** The codec cannot be kept. */ - protected static final int KEEP_CODEC_RESULT_NO = 0; - /** The codec can be kept, but must be flushed. */ - protected static final int KEEP_CODEC_RESULT_YES_WITH_FLUSH = 1; - /** - * The codec can be kept. It does not need to be flushed, but must be reconfigured by prefixing - * the next input buffer with the new format's configuration data. - */ - protected static final int KEEP_CODEC_RESULT_YES_WITH_RECONFIGURATION = 2; - /** The codec can be kept. It does not need to be flushed and no reconfiguration is required. */ - protected static final int KEEP_CODEC_RESULT_YES_WITHOUT_RECONFIGURATION = 3; - @Documented @Retention(RetentionPolicy.SOURCE) @IntDef({ @@ -295,7 +241,7 @@ private static String buildCustomDiagnosticInfo(int errorCode) { @IntDef({ DRAIN_ACTION_NONE, DRAIN_ACTION_FLUSH, - DRAIN_ACTION_UPDATE_DRM_SESSION, + DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION, DRAIN_ACTION_REINITIALIZE }) private @interface DrainAction {} @@ -304,7 +250,7 @@ private static String buildCustomDiagnosticInfo(int errorCode) { /** The codec should be flushed. */ private static final int DRAIN_ACTION_FLUSH = 1; /** The codec should be flushed and updated to use the pending DRM session. */ - private static final int DRAIN_ACTION_UPDATE_DRM_SESSION = 2; + private static final int DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION = 2; /** The codec should be reinitialized. */ private static final int DRAIN_ACTION_REINITIALIZE = 3; @@ -344,11 +290,13 @@ private static String buildCustomDiagnosticInfo(int errorCode) { private static final int ADAPTATION_WORKAROUND_SLICE_WIDTH_HEIGHT = 32; + private final MediaCodecAdapter.Factory codecAdapterFactory; private final MediaCodecSelector mediaCodecSelector; private final boolean enableDecoderFallback; private final float assumedMinimumCodecOperatingRate; - private final DecoderInputBuffer buffer; private final DecoderInputBuffer flagsOnlyBuffer; + private final DecoderInputBuffer buffer; + private final DecoderInputBuffer bypassSampleBuffer; private final BatchBuffer bypassBatchBuffer; private final TimedValueQueue formatQueue; private final ArrayList decodeOnlyPresentationTimestamps; @@ -364,9 +312,9 @@ private static String buildCustomDiagnosticInfo(int errorCode) { @Nullable private MediaCrypto mediaCrypto; private boolean mediaCryptoRequiresSecureDecoder; private long renderTimeLimitMs; - private float operatingRate; - @Nullable private MediaCodec codec; - @Nullable private MediaCodecAdapter codecAdapter; + private float currentPlaybackSpeed; + private float targetPlaybackSpeed; + @Nullable private MediaCodecAdapter codec; @Nullable private Format codecInputFormat; @Nullable private MediaFormat codecOutputMediaFormat; private boolean codecOutputMediaFormatChanged; @@ -375,19 +323,17 @@ private static String buildCustomDiagnosticInfo(int errorCode) { @Nullable private DecoderInitializationException preferredDecoderInitializationException; @Nullable private MediaCodecInfo codecInfo; @AdaptationWorkaroundMode private int codecAdaptationWorkaroundMode; - private boolean codecNeedsReconfigureWorkaround; private boolean codecNeedsDiscardToSpsWorkaround; private boolean codecNeedsFlushWorkaround; private boolean codecNeedsSosFlushWorkaround; private boolean codecNeedsEosFlushWorkaround; private boolean codecNeedsEosOutputExceptionWorkaround; + private boolean codecNeedsEosBufferTimestampWorkaround; private boolean codecNeedsMonoChannelCountWorkaround; private boolean codecNeedsAdaptationWorkaroundBuffer; private boolean shouldSkipAdaptationWorkaroundOutputBuffer; private boolean codecNeedsEosPropagation; @Nullable private C2Mp3TimestampTracker c2Mp3TimestampTracker; - private ByteBuffer[] inputBuffers; - private ByteBuffer[] outputBuffers; private long codecHotswapDeadlineMs; private int inputIndex; private int outputIndex; @@ -395,6 +341,7 @@ private static String buildCustomDiagnosticInfo(int errorCode) { private boolean isDecodeOnlyOutputBuffer; private boolean isLastOutputBuffer; private boolean bypassEnabled; + private boolean bypassSampleBufferPending; private boolean bypassDrainAndReinitialize; private boolean codecReconfigured; @ReconfigurationState private int codecReconfigurationState; @@ -409,7 +356,9 @@ private static String buildCustomDiagnosticInfo(int errorCode) { private boolean outputStreamEnded; private boolean waitingForFirstSampleInFormat; private boolean pendingOutputEndOfStream; - @MediaCodecOperationMode private int mediaCodecOperationMode; + private boolean enableAsynchronousBufferQueueing; + private boolean forceAsyncQueueingSynchronizationWorkaround; + private boolean enableSynchronizeCodecInteractionsWithQueueing; @Nullable private ExoPlaybackException pendingPlaybackException; protected DecoderCounters decoderCounters; private long outputStreamStartPositionUs; @@ -429,31 +378,36 @@ private static String buildCustomDiagnosticInfo(int errorCode) { */ public MediaCodecRenderer( int trackType, + MediaCodecAdapter.Factory codecAdapterFactory, MediaCodecSelector mediaCodecSelector, boolean enableDecoderFallback, float assumedMinimumCodecOperatingRate) { super(trackType); - this.mediaCodecSelector = Assertions.checkNotNull(mediaCodecSelector); + this.codecAdapterFactory = codecAdapterFactory; + this.mediaCodecSelector = checkNotNull(mediaCodecSelector); this.enableDecoderFallback = enableDecoderFallback; this.assumedMinimumCodecOperatingRate = assumedMinimumCodecOperatingRate; - buffer = new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DISABLED); flagsOnlyBuffer = DecoderInputBuffer.newFlagsOnlyInstance(); + buffer = new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DISABLED); + bypassSampleBuffer = new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + bypassBatchBuffer = new BatchBuffer(); formatQueue = new TimedValueQueue<>(); decodeOnlyPresentationTimestamps = new ArrayList<>(); outputBufferInfo = new MediaCodec.BufferInfo(); - operatingRate = 1f; - mediaCodecOperationMode = OPERATION_MODE_SYNCHRONOUS; + currentPlaybackSpeed = 1f; + targetPlaybackSpeed = 1f; renderTimeLimitMs = C.TIME_UNSET; pendingOutputStreamStartPositionsUs = new long[MAX_PENDING_OUTPUT_STREAM_OFFSET_COUNT]; pendingOutputStreamOffsetsUs = new long[MAX_PENDING_OUTPUT_STREAM_OFFSET_COUNT]; pendingOutputStreamSwitchTimesUs = new long[MAX_PENDING_OUTPUT_STREAM_OFFSET_COUNT]; outputStreamStartPositionUs = C.TIME_UNSET; outputStreamOffsetUs = C.TIME_UNSET; - bypassBatchBuffer = new BatchBuffer(); - bypassBatchBuffer.ensureSpaceForWrite(/* length= */ 0); // MediaCodec outputs audio buffers in native endian: // https://developer.android.com/reference/android/media/MediaCodec#raw-audio-buffers // and code called from MediaCodecAudioRenderer.processOutputBuffer expects this endianness. + // Call ensureSpaceForWrite to make sure the buffer has non-null data, and set the expected + // endianness. + bypassBatchBuffer.ensureSpaceForWrite(/* length= */ 0); bypassBatchBuffer.data.order(ByteOrder.nativeOrder()); resetCodecStateForRelease(); } @@ -472,28 +426,43 @@ public void setRenderTimeLimitMs(long renderTimeLimitMs) { } /** - * Set the mode of operation of the underlying {@link MediaCodec}. + * Enable asynchronous input buffer queueing. + * + *

    Operates the underlying {@link MediaCodec} in asynchronous mode and submits input buffers + * from a separate thread to unblock the playback thread. * *

    This method is experimental, and will be renamed or removed in a future release. It should * only be called before the renderer is used. + */ + public void experimentalSetAsynchronousBufferQueueingEnabled(boolean enabled) { + enableAsynchronousBufferQueueing = enabled; + } + + /** + * Enable the asynchronous queueing synchronization workaround. + * + *

    When enabled, the queueing threads for {@link MediaCodec} instance will synchronize on a + * shared lock when submitting buffers to the respective {@link MediaCodec}. * - * @param mode The mode of the MediaCodec. The supported modes are: - *

      - *
    • {@link #OPERATION_MODE_SYNCHRONOUS}: The {@link MediaCodec} will operate in - * synchronous mode. - *
    • {@link #OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD}: The {@link MediaCodec} will - * operate in asynchronous mode and {@link MediaCodec.Callback} callbacks will be routed - * to a dedicated thread. This mode requires API level ≥ 23; if the API level is ≤ - * 22, the operation mode will be set to {@link #OPERATION_MODE_SYNCHRONOUS}. - *
    • {@link #OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING}: Same as - * {@link #OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD} and, in addition, input buffers - * will be submitted to the {@link MediaCodec} in a separate thread. - *
    - * By default, the operation mode is set to {@link - * MediaCodecRenderer#OPERATION_MODE_SYNCHRONOUS}. + *

    This method is experimental, and will be renamed or removed in a future release. It should + * only be called before the renderer is used. */ - public void experimentalSetMediaCodecOperationMode(@MediaCodecOperationMode int mode) { - mediaCodecOperationMode = mode; + public void experimentalSetForceAsyncQueueingSynchronizationWorkaround(boolean enabled) { + this.forceAsyncQueueingSynchronizationWorkaround = enabled; + } + + /** + * Enable synchronizing codec interactions with asynchronous buffer queueing. + * + *

    When enabled, codec interactions will wait until all input buffers pending for asynchronous + * queueing are submitted to the {@link MediaCodec} first. This method is effective only if {@link + * #experimentalSetAsynchronousBufferQueueingEnabled asynchronous buffer queueing} is enabled. + * + *

    This method is experimental, and will be renamed or removed in a future release. It should + * only be called before the renderer is used. + */ + public void experimentalSetSynchronizeCodecInteractionsWithQueueingEnabled(boolean enabled) { + enableSynchronizeCodecInteractionsWithQueueing = enabled; } @Override @@ -543,7 +512,7 @@ protected abstract List getDecoderInfos( * Configures a newly created {@link MediaCodec}. * * @param codecInfo Information about the {@link MediaCodec} being configured. - * @param codecAdapter The {@link MediaCodecAdapter} to configure. + * @param codec The {@link MediaCodecAdapter} to configure. * @param format The {@link Format} for which the codec is being configured. * @param crypto For drm protected playbacks, a {@link MediaCrypto} to use for decryption. * @param codecOperatingRate The codec operating rate, or {@link #CODEC_OPERATING_RATE_UNSET} if @@ -551,7 +520,7 @@ protected abstract List getDecoderInfos( */ protected abstract void configureCodec( MediaCodecInfo codecInfo, - MediaCodecAdapter codecAdapter, + MediaCodecAdapter codec, Format format, @Nullable MediaCrypto crypto, float codecOperatingRate); @@ -577,8 +546,8 @@ protected final void maybeInitCodecOrBypass() throws ExoPlaybackException { if (sessionMediaCrypto == null) { @Nullable DrmSessionException drmError = codecDrmSession.getError(); if (drmError != null) { - // Continue for now. We may be able to avoid failure if the session recovers, or if a - // new input format causes the session to be replaced before it's used. + // Continue for now. We may be able to avoid failure if a new input format causes the + // session to be replaced without it having been used. } else { // The drm session isn't open yet. return; @@ -677,17 +646,7 @@ protected final void updateOutputFormatForTime(long presentationTimeUs) } @Nullable - protected Format getInputFormat() { - return inputFormat; - } - - @Nullable - protected final Format getOutputFormat() { - return outputFormat; - } - - @Nullable - protected final MediaCodec getCodec() { + protected final MediaCodecAdapter getCodec() { return codec; } @@ -736,7 +695,9 @@ protected void onPositionReset(long positionUs, boolean joining) throws ExoPlayb outputStreamEnded = false; pendingOutputEndOfStream = false; if (bypassEnabled) { - bypassBatchBuffer.flush(); + bypassBatchBuffer.clear(); + bypassSampleBuffer.clear(); + bypassSampleBufferPending = false; } else { flushOrReinitializeCodec(); } @@ -756,12 +717,14 @@ protected void onPositionReset(long positionUs, boolean joining) throws ExoPlayb } @Override - public void setOperatingRate(float operatingRate) throws ExoPlaybackException { - this.operatingRate = operatingRate; + public void setPlaybackSpeed(float currentPlaybackSpeed, float targetPlaybackSpeed) + throws ExoPlaybackException { + this.currentPlaybackSpeed = currentPlaybackSpeed; + this.targetPlaybackSpeed = targetPlaybackSpeed; if (codec != null && codecDrainAction != DRAIN_ACTION_REINITIALIZE && getState() != STATE_DISABLED) { - updateCodecOperatingRate(); + updateCodecOperatingRate(codecInputFormat); } } @@ -792,21 +755,20 @@ protected void onReset() { private void disableBypass() { bypassDrainAndReinitialize = false; bypassBatchBuffer.clear(); + bypassSampleBuffer.clear(); + bypassSampleBufferPending = false; bypassEnabled = false; } protected void releaseCodec() { try { - if (codecAdapter != null) { - codecAdapter.shutdown(); - } if (codec != null) { - decoderCounters.decoderReleaseCount++; codec.release(); + decoderCounters.decoderReleaseCount++; + onCodecReleased(codecInfo.name); } } finally { codec = null; - codecAdapter = null; try { if (mediaCrypto != null) { mediaCrypto.release(); @@ -915,12 +877,17 @@ protected boolean flushOrReleaseCodec() { releaseCodec(); return true; } + flushCodec(); + return false; + } + + /** Flushes the codec. */ + private void flushCodec() { try { - codecAdapter.flush(); + codec.flush(); } finally { resetCodecStateForFlush(); } - return false; } /** Resets the renderer internal state after a codec flush. */ @@ -970,17 +937,16 @@ protected void resetCodecStateForRelease() { codecHasOutputMediaFormat = false; codecOperatingRate = CODEC_OPERATING_RATE_UNSET; codecAdaptationWorkaroundMode = ADAPTATION_WORKAROUND_MODE_NEVER; - codecNeedsReconfigureWorkaround = false; codecNeedsDiscardToSpsWorkaround = false; codecNeedsFlushWorkaround = false; codecNeedsSosFlushWorkaround = false; codecNeedsEosFlushWorkaround = false; codecNeedsEosOutputExceptionWorkaround = false; + codecNeedsEosBufferTimestampWorkaround = false; codecNeedsMonoChannelCountWorkaround = false; codecNeedsEosPropagation = false; codecReconfigured = false; codecReconfigurationState = RECONFIGURATION_STATE_NONE; - resetCodecBuffers(); mediaCryptoRequiresSecureDecoder = false; } @@ -1104,9 +1070,9 @@ private void initBypass(Format format) { && !MimeTypes.AUDIO_MPEG.equals(mimeType) && !MimeTypes.AUDIO_OPUS.equals(mimeType)) { // TODO(b/154746451): Batching provokes frame drops in non offload. - bypassBatchBuffer.setMaxAccessUnitCount(1); + bypassBatchBuffer.setMaxSampleCount(1); } else { - bypassBatchBuffer.setMaxAccessUnitCount(BatchBuffer.DEFAULT_BATCH_SIZE_ACCESS_UNITS); + bypassBatchBuffer.setMaxSampleCount(BatchBuffer.DEFAULT_MAX_SAMPLE_COUNT); } bypassEnabled = true; } @@ -1114,35 +1080,31 @@ private void initBypass(Format format) { private void initCodec(MediaCodecInfo codecInfo, MediaCrypto crypto) throws Exception { long codecInitializingTimestamp; long codecInitializedTimestamp; - MediaCodec codec = null; + @Nullable MediaCodecAdapter codecAdapter = null; String codecName = codecInfo.name; float codecOperatingRate = Util.SDK_INT < 23 ? CODEC_OPERATING_RATE_UNSET - : getCodecOperatingRateV23(operatingRate, inputFormat, getStreamFormats()); + : getCodecOperatingRateV23(targetPlaybackSpeed, inputFormat, getStreamFormats()); if (codecOperatingRate <= assumedMinimumCodecOperatingRate) { codecOperatingRate = CODEC_OPERATING_RATE_UNSET; } - MediaCodecAdapter codecAdapter = null; try { codecInitializingTimestamp = SystemClock.elapsedRealtime(); TraceUtil.beginSection("createCodec:" + codecName); - codec = MediaCodec.createByCodecName(codecName); - if (mediaCodecOperationMode == OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD - && Util.SDK_INT >= 23) { - codecAdapter = new AsynchronousMediaCodecAdapter(codec, getTrackType()); - } else if (mediaCodecOperationMode - == OPERATION_MODE_ASYNCHRONOUS_DEDICATED_THREAD_ASYNCHRONOUS_QUEUEING - && Util.SDK_INT >= 23) { + MediaCodec codec = MediaCodec.createByCodecName(codecName); + if (enableAsynchronousBufferQueueing && Util.SDK_INT >= 23) { codecAdapter = - new AsynchronousMediaCodecAdapter( - codec, /* enableAsynchronousQueueing= */ true, getTrackType()); + new AsynchronousMediaCodecAdapter.Factory( + getTrackType(), + forceAsyncQueueingSynchronizationWorkaround, + enableSynchronizeCodecInteractionsWithQueueing) + .createAdapter(codec); } else { - codecAdapter = new SynchronousMediaCodecAdapter(codec); + codecAdapter = codecAdapterFactory.createAdapter(codec); } - TraceUtil.endSection(); TraceUtil.beginSection("configureCodec"); configureCodec(codecInfo, codecAdapter, inputFormat, crypto, codecOperatingRate); @@ -1151,31 +1113,25 @@ private void initCodec(MediaCodecInfo codecInfo, MediaCrypto crypto) throws Exce codecAdapter.start(); TraceUtil.endSection(); codecInitializedTimestamp = SystemClock.elapsedRealtime(); - getCodecBuffers(codec); } catch (Exception e) { if (codecAdapter != null) { - codecAdapter.shutdown(); - } - if (codec != null) { - resetCodecBuffers(); - codec.release(); + codecAdapter.release(); } throw e; } - this.codec = codec; - this.codecAdapter = codecAdapter; + this.codec = codecAdapter; this.codecInfo = codecInfo; this.codecOperatingRate = codecOperatingRate; codecInputFormat = inputFormat; codecAdaptationWorkaroundMode = codecAdaptationWorkaroundMode(codecName); - codecNeedsReconfigureWorkaround = codecNeedsReconfigureWorkaround(codecName); codecNeedsDiscardToSpsWorkaround = codecNeedsDiscardToSpsWorkaround(codecName, codecInputFormat); codecNeedsFlushWorkaround = codecNeedsFlushWorkaround(codecName); codecNeedsSosFlushWorkaround = codecNeedsSosFlushWorkaround(codecName); codecNeedsEosFlushWorkaround = codecNeedsEosFlushWorkaround(codecName); codecNeedsEosOutputExceptionWorkaround = codecNeedsEosOutputExceptionWorkaround(codecName); + codecNeedsEosBufferTimestampWorkaround = codecNeedsEosBufferTimestampWorkaround(codecName); codecNeedsMonoChannelCountWorkaround = codecNeedsMonoChannelCountWorkaround(codecName, codecInputFormat); codecNeedsEosPropagation = @@ -1198,37 +1154,6 @@ private boolean shouldContinueRendering(long renderStartTimeMs) { || SystemClock.elapsedRealtime() - renderStartTimeMs < renderTimeLimitMs; } - private void getCodecBuffers(MediaCodec codec) { - if (Util.SDK_INT < 21) { - inputBuffers = codec.getInputBuffers(); - outputBuffers = codec.getOutputBuffers(); - } - } - - private void resetCodecBuffers() { - if (Util.SDK_INT < 21) { - inputBuffers = null; - outputBuffers = null; - } - } - - private ByteBuffer getInputBuffer(int inputIndex) { - if (Util.SDK_INT >= 21) { - return codec.getInputBuffer(inputIndex); - } else { - return inputBuffers[inputIndex]; - } - } - - @Nullable - private ByteBuffer getOutputBuffer(int outputIndex) { - if (Util.SDK_INT >= 21) { - return codec.getOutputBuffer(outputIndex); - } else { - return outputBuffers[outputIndex]; - } - } - private boolean hasOutputBuffer() { return outputIndex >= 0; } @@ -1263,11 +1188,11 @@ private boolean feedInputBuffer() throws ExoPlaybackException { } if (inputIndex < 0) { - inputIndex = codecAdapter.dequeueInputBufferIndex(); + inputIndex = codec.dequeueInputBufferIndex(); if (inputIndex < 0) { return false; } - buffer.data = getInputBuffer(inputIndex); + buffer.data = codec.getInputBuffer(inputIndex); buffer.clear(); } @@ -1278,7 +1203,7 @@ private boolean feedInputBuffer() throws ExoPlaybackException { // Do nothing. } else { codecReceivedEos = true; - codecAdapter.queueInputBuffer(inputIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM); + codec.queueInputBuffer(inputIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM); resetInputBuffer(); } codecDrainState = DRAIN_STATE_WAIT_END_OF_STREAM; @@ -1288,7 +1213,7 @@ private boolean feedInputBuffer() throws ExoPlaybackException { if (codecNeedsAdaptationWorkaroundBuffer) { codecNeedsAdaptationWorkaroundBuffer = false; buffer.data.put(ADAPTATION_WORKAROUND_BUFFER); - codecAdapter.queueInputBuffer(inputIndex, 0, ADAPTATION_WORKAROUND_BUFFER.length, 0, 0); + codec.queueInputBuffer(inputIndex, 0, ADAPTATION_WORKAROUND_BUFFER.length, 0, 0); resetInputBuffer(); codecReceivedBuffers = true; return true; @@ -1347,7 +1272,7 @@ private boolean feedInputBuffer() throws ExoPlaybackException { // Do nothing. } else { codecReceivedEos = true; - codecAdapter.queueInputBuffer( + codec.queueInputBuffer( inputIndex, /* offset= */ 0, /* size= */ 0, @@ -1418,10 +1343,10 @@ private boolean feedInputBuffer() throws ExoPlaybackException { onQueueInputBuffer(buffer); try { if (bufferEncrypted) { - codecAdapter.queueSecureInputBuffer( + codec.queueSecureInputBuffer( inputIndex, /* offset= */ 0, buffer.cryptoInfo, presentationTimeUs, /* flags= */ 0); } else { - codecAdapter.queueInputBuffer( + codec.queueInputBuffer( inputIndex, /* offset= */ 0, buffer.data.limit(), presentationTimeUs, /* flags= */ 0); } } catch (CryptoException e) { @@ -1450,16 +1375,31 @@ protected void onCodecInitialized(String name, long initializedTimestampMs, // Do nothing. } + /** + * Called when a {@link MediaCodec} has been released. + * + *

    The default implementation is a no-op. + * + * @param name The name of the codec that was released. + */ + protected void onCodecReleased(String name) { + // Do nothing. + } + /** * Called when a new {@link Format} is read from the upstream {@link MediaPeriod}. * * @param formatHolder A {@link FormatHolder} that holds the new {@link Format}. * @throws ExoPlaybackException If an error occurs re-initializing the {@link MediaCodec}. + * @return The result of the evaluation to determine whether the existing decoder instance can be + * reused for the new format, or {@code null} if the renderer did not have a decoder. */ @CallSuper - protected void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybackException { + @Nullable + protected DecoderReuseEvaluation onInputFormatChanged(FormatHolder formatHolder) + throws ExoPlaybackException { waitingForFirstSampleInFormat = true; - Format newFormat = Assertions.checkNotNull(formatHolder.format); + Format newFormat = checkNotNull(formatHolder.format); if (newFormat.sampleMimeType == null) { // If the new format is invalid, it is either a media bug or it is not intended to be played. // See also https://github.com/google/ExoPlayer/issues/8283. @@ -1470,7 +1410,7 @@ protected void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybac if (bypassEnabled) { bypassDrainAndReinitialize = true; - return; // Need to drain batch buffer first. + return null; // Need to drain batch buffer first. } if (codec == null) { @@ -1478,67 +1418,91 @@ protected void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybac availableCodecInfos = null; } maybeInitCodecOrBypass(); - return; + return null; } - // We have an existing codec that we may need to reconfigure or re-initialize or release it to - // switch to bypass. If the existing codec instance is being kept then its operating rate - // may need to be updated. + // We have an existing codec that we may need to reconfigure, re-initialize, or release to + // switch to bypass. If the existing codec instance is kept then its operating rate and DRM + // session may need to be updated. - if ((sourceDrmSession == null && codecDrmSession != null) - || (sourceDrmSession != null && codecDrmSession == null) - || (sourceDrmSession != codecDrmSession - && !codecInfo.secure - && maybeRequiresSecureDecoder(sourceDrmSession, newFormat)) - || (Util.SDK_INT < 23 && sourceDrmSession != codecDrmSession)) { - // We might need to switch between the clear and protected output paths, or we're using DRM - // prior to API level 23 where the codec needs to be re-initialized to switch to the new DRM - // session. - drainAndReinitializeCodec(); - return; - } + // Copy the current codec and codecInfo to local variables so they remain accessible if the + // member variables are updated during the logic below. + MediaCodecAdapter codec = this.codec; + MediaCodecInfo codecInfo = this.codecInfo; - switch (canKeepCodec(codec, codecInfo, codecInputFormat, newFormat)) { - case KEEP_CODEC_RESULT_NO: + Format oldFormat = codecInputFormat; + if (drmNeedsCodecReinitialization(codecInfo, newFormat, codecDrmSession, sourceDrmSession)) { + drainAndReinitializeCodec(); + return new DecoderReuseEvaluation( + codecInfo.name, + oldFormat, + newFormat, + REUSE_RESULT_NO, + DISCARD_REASON_DRM_SESSION_CHANGED); + } + boolean drainAndUpdateCodecDrmSession = sourceDrmSession != codecDrmSession; + Assertions.checkState(!drainAndUpdateCodecDrmSession || Util.SDK_INT >= 23); + + DecoderReuseEvaluation evaluation = canReuseCodec(codecInfo, oldFormat, newFormat); + @DecoderDiscardReasons int overridingDiscardReasons = 0; + switch (evaluation.result) { + case REUSE_RESULT_NO: drainAndReinitializeCodec(); break; - case KEEP_CODEC_RESULT_YES_WITH_FLUSH: - codecInputFormat = newFormat; - updateCodecOperatingRate(); - if (sourceDrmSession != codecDrmSession) { - drainAndUpdateCodecDrmSession(); + case REUSE_RESULT_YES_WITH_FLUSH: + if (!updateCodecOperatingRate(newFormat)) { + overridingDiscardReasons |= DISCARD_REASON_OPERATING_RATE_CHANGED; } else { - drainAndFlushCodec(); + codecInputFormat = newFormat; + if (drainAndUpdateCodecDrmSession) { + if (!drainAndUpdateCodecDrmSessionV23()) { + overridingDiscardReasons |= DISCARD_REASON_WORKAROUND; + } + } else if (!drainAndFlushCodec()) { + overridingDiscardReasons |= DISCARD_REASON_WORKAROUND; + } } break; - case KEEP_CODEC_RESULT_YES_WITH_RECONFIGURATION: - if (codecNeedsReconfigureWorkaround) { - drainAndReinitializeCodec(); + case REUSE_RESULT_YES_WITH_RECONFIGURATION: + if (!updateCodecOperatingRate(newFormat)) { + overridingDiscardReasons |= DISCARD_REASON_OPERATING_RATE_CHANGED; } else { codecReconfigured = true; codecReconfigurationState = RECONFIGURATION_STATE_WRITE_PENDING; codecNeedsAdaptationWorkaroundBuffer = codecAdaptationWorkaroundMode == ADAPTATION_WORKAROUND_MODE_ALWAYS || (codecAdaptationWorkaroundMode == ADAPTATION_WORKAROUND_MODE_SAME_RESOLUTION - && newFormat.width == codecInputFormat.width - && newFormat.height == codecInputFormat.height); + && newFormat.width == oldFormat.width + && newFormat.height == oldFormat.height); codecInputFormat = newFormat; - updateCodecOperatingRate(); - if (sourceDrmSession != codecDrmSession) { - drainAndUpdateCodecDrmSession(); + if (drainAndUpdateCodecDrmSession && !drainAndUpdateCodecDrmSessionV23()) { + overridingDiscardReasons |= DISCARD_REASON_WORKAROUND; } } break; - case KEEP_CODEC_RESULT_YES_WITHOUT_RECONFIGURATION: - codecInputFormat = newFormat; - updateCodecOperatingRate(); - if (sourceDrmSession != codecDrmSession) { - drainAndUpdateCodecDrmSession(); + case REUSE_RESULT_YES_WITHOUT_RECONFIGURATION: + if (!updateCodecOperatingRate(newFormat)) { + overridingDiscardReasons |= DISCARD_REASON_OPERATING_RATE_CHANGED; + } else { + codecInputFormat = newFormat; + if (drainAndUpdateCodecDrmSession && !drainAndUpdateCodecDrmSessionV23()) { + overridingDiscardReasons |= DISCARD_REASON_WORKAROUND; + } } break; default: throw new IllegalStateException(); // Never happens. } + + if (evaluation.result != REUSE_RESULT_NO + && (this.codec != codec || codecDrainAction == DRAIN_ACTION_REINITIALIZE)) { + // Initial evaluation indicated reuse was possible, but codec re-initialization was triggered. + // The reasons are indicated by overridingDiscardReasons. + return new DecoderReuseEvaluation( + codecInfo.name, oldFormat, newFormat, REUSE_RESULT_NO, overridingDiscardReasons); + } + + return evaluation; } /** @@ -1630,20 +1594,24 @@ protected void onProcessedStreamChange() { } /** - * Determines whether the existing {@link MediaCodec} can be kept for a new {@link Format}, and if + * Evaluates whether the existing {@link MediaCodec} can be kept for a new {@link Format}, and if * it can whether it requires reconfiguration. * - *

    The default implementation returns {@link #KEEP_CODEC_RESULT_NO}. + *

    The default implementation does not allow decoder reuse. * - * @param codec The existing {@link MediaCodec} instance. * @param codecInfo A {@link MediaCodecInfo} describing the decoder. * @param oldFormat The {@link Format} for which the existing instance is configured. * @param newFormat The new {@link Format}. - * @return Whether the instance can be kept, and if it can whether it requires reconfiguration. + * @return The result of the evaluation. */ - protected @KeepCodecResult int canKeepCodec( - MediaCodec codec, MediaCodecInfo codecInfo, Format oldFormat, Format newFormat) { - return KEEP_CODEC_RESULT_NO; + protected DecoderReuseEvaluation canReuseCodec( + MediaCodecInfo codecInfo, Format oldFormat, Format newFormat) { + return new DecoderReuseEvaluation( + codecInfo.name, + oldFormat, + newFormat, + REUSE_RESULT_NO, + DISCARD_REASON_REUSE_NOT_IMPLEMENTED); } @Override @@ -1660,9 +1628,9 @@ public boolean isReady() { && SystemClock.elapsedRealtime() < codecHotswapDeadlineMs)); } - /** Returns the renderer operating rate, as set by {@link #setOperatingRate}. */ - protected float getOperatingRate() { - return operatingRate; + /** Returns the current playback speed, as set by {@link #setPlaybackSpeed}. */ + protected float getPlaybackSpeed() { + return currentPlaybackSpeed; } /** Returns the operating rate used by the current codec */ @@ -1671,40 +1639,47 @@ protected float getCodecOperatingRate() { } /** - * Returns the {@link MediaFormat#KEY_OPERATING_RATE} value for a given renderer operating rate, - * current {@link Format} and set of possible stream formats. + * Returns the {@link MediaFormat#KEY_OPERATING_RATE} value for a given playback speed, current + * {@link Format} and set of possible stream formats. * *

    The default implementation returns {@link #CODEC_OPERATING_RATE_UNSET}. * - * @param operatingRate The renderer operating rate. + * @param targetPlaybackSpeed The target factor by which playback should be sped up. This may be + * different from the current playback speed, for example, if the speed is temporarily + * adjusted for live playback. * @param format The {@link Format} for which the codec is being configured. * @param streamFormats The possible stream formats. * @return The codec operating rate, or {@link #CODEC_OPERATING_RATE_UNSET} if no codec operating * rate should be set. */ protected float getCodecOperatingRateV23( - float operatingRate, Format format, Format[] streamFormats) { + float targetPlaybackSpeed, Format format, Format[] streamFormats) { return CODEC_OPERATING_RATE_UNSET; } /** - * Updates the codec operating rate. + * Updates the codec operating rate, or triggers codec release and re-initialization if a + * previously set operating rate needs to be cleared. * + * @param format The {@link Format} for which the operating rate should be configured. * @throws ExoPlaybackException If an error occurs releasing or initializing a codec. + * @return False if codec release and re-initialization was triggered. True in all other cases. */ - private void updateCodecOperatingRate() throws ExoPlaybackException { + private boolean updateCodecOperatingRate(Format format) throws ExoPlaybackException { if (Util.SDK_INT < 23) { - return; + return true; } float newCodecOperatingRate = - getCodecOperatingRateV23(operatingRate, codecInputFormat, getStreamFormats()); + getCodecOperatingRateV23(targetPlaybackSpeed, format, getStreamFormats()); if (codecOperatingRate == newCodecOperatingRate) { // No change. + return true; } else if (newCodecOperatingRate == CODEC_OPERATING_RATE_UNSET) { // The only way to clear the operating rate is to instantiate a new codec instance. See // [Internal ref: b/111543954]. drainAndReinitializeCodec(); + return false; } else if (codecOperatingRate != CODEC_OPERATING_RATE_UNSET || newCodecOperatingRate > assumedMinimumCodecOperatingRate) { // We need to set the operating rate, either because we've set it previously or because it's @@ -1713,36 +1688,56 @@ private void updateCodecOperatingRate() throws ExoPlaybackException { codecParameters.putFloat(MediaFormat.KEY_OPERATING_RATE, newCodecOperatingRate); codec.setParameters(codecParameters); codecOperatingRate = newCodecOperatingRate; + return true; } + + return true; } - /** Starts draining the codec for flush. */ - private void drainAndFlushCodec() { + /** + * Starts draining the codec for a flush, or to release and re-initialize the codec if flushing + * will not be possible. If no buffers have been queued to the codec then this method is a no-op. + * + * @return False if codec release and re-initialization was triggered due to the need to apply a + * flush workaround. True in all other cases. + */ + private boolean drainAndFlushCodec() { if (codecReceivedBuffers) { codecDrainState = DRAIN_STATE_SIGNAL_END_OF_STREAM; - codecDrainAction = DRAIN_ACTION_FLUSH; + if (codecNeedsFlushWorkaround || codecNeedsEosFlushWorkaround) { + codecDrainAction = DRAIN_ACTION_REINITIALIZE; + return false; + } else { + codecDrainAction = DRAIN_ACTION_FLUSH; + } } + return true; } /** - * Starts draining the codec to update its DRM session. The update may occur immediately if no - * buffers have been queued to the codec. + * Starts draining the codec to flush it and update its DRM session, or to release and + * re-initialize the codec if flushing will not be possible. If no buffers have been queued to the + * codec then this method updates the DRM session immediately without flushing the codec. * * @throws ExoPlaybackException If an error occurs updating the codec's DRM session. + * @return False if codec release and re-initialization was triggered due to the need to apply a + * flush workaround. True in all other cases. */ - private void drainAndUpdateCodecDrmSession() throws ExoPlaybackException { - if (Util.SDK_INT < 23) { - // The codec needs to be re-initialized to switch to the source DRM session. - drainAndReinitializeCodec(); - return; - } + @TargetApi(23) // Only called when SDK_INT >= 23, but lint isn't clever enough to know. + private boolean drainAndUpdateCodecDrmSessionV23() throws ExoPlaybackException { if (codecReceivedBuffers) { codecDrainState = DRAIN_STATE_SIGNAL_END_OF_STREAM; - codecDrainAction = DRAIN_ACTION_UPDATE_DRM_SESSION; + if (codecNeedsFlushWorkaround || codecNeedsEosFlushWorkaround) { + codecDrainAction = DRAIN_ACTION_REINITIALIZE; + return false; + } else { + codecDrainAction = DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION; + } } else { // Nothing has been queued to the decoder, so we can do the update immediately. - updateDrmSessionOrReinitializeCodecV23(); + updateDrmSessionV23(); } + return true; } /** @@ -1771,7 +1766,7 @@ private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs) int outputIndex; if (codecNeedsEosOutputExceptionWorkaround && codecReceivedEos) { try { - outputIndex = codecAdapter.dequeueOutputBufferIndex(outputBufferInfo); + outputIndex = codec.dequeueOutputBufferIndex(outputBufferInfo); } catch (IllegalStateException e) { processEndOfStream(); if (outputStreamEnded) { @@ -1781,18 +1776,15 @@ private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs) return false; } } else { - outputIndex = codecAdapter.dequeueOutputBufferIndex(outputBufferInfo); + outputIndex = codec.dequeueOutputBufferIndex(outputBufferInfo); } if (outputIndex < 0) { if (outputIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED /* (-2) */) { processOutputMediaFormatChanged(); return true; - } else if (outputIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED /* (-3) */) { - processOutputBuffersChanged(); - return true; } - /* MediaCodec.INFO_TRY_AGAIN_LATER (-1) or unknown negative return value */ + // MediaCodec.INFO_TRY_AGAIN_LATER (-1) or unknown negative return value. if (codecNeedsEosPropagation && (inputStreamEnded || codecDrainState == DRAIN_STATE_WAIT_END_OF_STREAM)) { processEndOfStream(); @@ -1813,7 +1805,7 @@ private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs) } this.outputIndex = outputIndex; - outputBuffer = getOutputBuffer(outputIndex); + outputBuffer = codec.getOutputBuffer(outputIndex); // The dequeued buffer is a media buffer. Do some initial setup. // It will be processed by calling processOutputBuffer (possibly multiple times). @@ -1821,6 +1813,12 @@ private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs) outputBuffer.position(outputBufferInfo.offset); outputBuffer.limit(outputBufferInfo.offset + outputBufferInfo.size); } + if (codecNeedsEosBufferTimestampWorkaround + && outputBufferInfo.presentationTimeUs == 0 + && (outputBufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0 + && largestQueuedPresentationTimeUs != C.TIME_UNSET) { + outputBufferInfo.presentationTimeUs = largestQueuedPresentationTimeUs; + } isDecodeOnlyOutputBuffer = isDecodeOnlyBuffer(outputBufferInfo.presentationTimeUs); isLastOutputBuffer = lastBufferInStreamPresentationTimeUs == outputBufferInfo.presentationTimeUs; @@ -1883,7 +1881,7 @@ private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs) /** Processes a change in the decoder output {@link MediaFormat}. */ private void processOutputMediaFormatChanged() { codecHasOutputMediaFormat = true; - MediaFormat mediaFormat = codecAdapter.getOutputFormat(); + MediaFormat mediaFormat = codec.getOutputFormat(); if (codecAdaptationWorkaroundMode != ADAPTATION_WORKAROUND_MODE_NEVER && mediaFormat.getInteger(MediaFormat.KEY_WIDTH) == ADAPTATION_WORKAROUND_SLICE_WIDTH_HEIGHT && mediaFormat.getInteger(MediaFormat.KEY_HEIGHT) @@ -1899,15 +1897,6 @@ private void processOutputMediaFormatChanged() { codecOutputMediaFormatChanged = true; } - /** - * Processes a change in the output buffers. - */ - private void processOutputBuffersChanged() { - if (Util.SDK_INT < 21) { - outputBuffers = codec.getOutputBuffers(); - } - } - /** * Processes an output media buffer. * @@ -1926,7 +1915,8 @@ private void processOutputBuffersChanged() { * iteration of the rendering loop. * @param elapsedRealtimeUs {@link SystemClock#elapsedRealtime()} in microseconds, measured at the * start of the current iteration of the rendering loop. - * @param codec The {@link MediaCodec} instance, or null in bypass mode were no codec is used. + * @param codec The {@link MediaCodecAdapter} instance, or null in bypass mode were no codec is + * used. * @param buffer The output buffer to process, or null if the buffer data is not made available to * the application layer (see {@link MediaCodec#getOutputBuffer(int)}). This {@code buffer} * can only be null for video data. Note that the buffer data can still be rendered in this @@ -1946,7 +1936,7 @@ private void processOutputBuffersChanged() { protected abstract boolean processOutputBuffer( long positionUs, long elapsedRealtimeUs, - @Nullable MediaCodec codec, + @Nullable MediaCodecAdapter codec, @Nullable ByteBuffer buffer, int bufferIndex, int bufferFlags, @@ -1979,11 +1969,12 @@ private void processEndOfStream() throws ExoPlaybackException { case DRAIN_ACTION_REINITIALIZE: reinitializeCodec(); break; - case DRAIN_ACTION_UPDATE_DRM_SESSION: - updateDrmSessionOrReinitializeCodecV23(); + case DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION: + flushCodec(); + updateDrmSessionV23(); break; case DRAIN_ACTION_FLUSH: - flushOrReinitializeCodec(); + flushCodec(); break; case DRAIN_ACTION_NONE: default: @@ -2001,22 +1992,10 @@ protected final void setPendingOutputEndOfStream() { pendingOutputEndOfStream = true; } - /** Returns the largest queued input presentation time, in microseconds. */ - protected final long getLargestQueuedPresentationTimeUs() { - return largestQueuedPresentationTimeUs; - } - - /** - * Returns the start position of the output {@link SampleStream}, in renderer time microseconds. - */ - protected final long getOutputStreamStartPositionUs() { - return outputStreamStartPositionUs; - } - /** * Returns the offset that should be subtracted from {@code bufferPresentationTimeUs} in {@link - * #processOutputBuffer(long, long, MediaCodec, ByteBuffer, int, int, int, long, boolean, boolean, - * Format)} to get the playback position with respect to the media. + * #processOutputBuffer(long, long, MediaCodecAdapter, ByteBuffer, int, int, int, long, boolean, + * boolean, Format)} to get the playback position with respect to the media. */ protected final long getOutputStreamOffsetUs() { return outputStreamOffsetUs; @@ -2029,23 +2008,71 @@ protected static boolean supportsFormatDrm(Format format) { } /** - * Returns whether a {@link DrmSession} may require a secure decoder for a given {@link Format}. - * - * @param drmSession The {@link DrmSession}. - * @param format The {@link Format}. - * @return Whether a secure decoder may be required. + * Returns whether it's necessary to re-initialize the codec to handle a DRM change. If {@code + * false} is returned then either {@code oldSession == newSession} (i.e., there was no change), or + * it's possible to update the existing codec using MediaCrypto.setMediaDrmSession. */ - private boolean maybeRequiresSecureDecoder(DrmSession drmSession, Format format) + private boolean drmNeedsCodecReinitialization( + MediaCodecInfo codecInfo, + Format newFormat, + @Nullable DrmSession oldSession, + @Nullable DrmSession newSession) throws ExoPlaybackException { - // MediaCrypto type is checked during track selection. - @Nullable FrameworkMediaCrypto sessionMediaCrypto = getFrameworkMediaCrypto(drmSession); - if (sessionMediaCrypto == null) { - // We'd only expect this to happen if the CDM from which the pending session is obtained needs + if (oldSession == newSession) { + // No need to re-initialize if the old and new sessions are the same. + return false; + } + + // Note: At least one of oldSession and newSession are non-null. + + if (newSession == null || oldSession == null) { + // Changing from DRM to no DRM and vice-versa always requires re-initialization. + return true; + } + + // Note: Both oldSession and newSession are non-null, and they are different sessions. + + if (Util.SDK_INT < 23) { + // MediaCrypto.setMediaDrmSession is only available from API level 23, so re-initialization is + // required to switch to newSession on older API levels. + return true; + } + if (C.PLAYREADY_UUID.equals(oldSession.getSchemeUuid()) + || C.PLAYREADY_UUID.equals(newSession.getSchemeUuid())) { + // The PlayReady CDM does not support MediaCrypto.setMediaDrmSession, either as the old or new + // session. + // TODO: Add an API check once [Internal ref: b/128835874] is fixed. + return true; + } + @Nullable FrameworkMediaCrypto newMediaCrypto = getFrameworkMediaCrypto(newSession); + if (newMediaCrypto == null) { + // We'd only expect this to happen if the CDM from which newSession is obtained needs // provisioning. This is unlikely to happen (it probably requires a switch from one DRM scheme - // to another, where the new CDM hasn't been used before and needs provisioning). Assume that - // a secure decoder may be required. + // to another, where the new CDM hasn't been used before and needs provisioning). It would be + // possible to handle this case without codec re-initialization, but it would require the + // re-use code path to be able to wait for provisioning to finish before calling + // MediaCrypto.setMediaDrmSession. The extra complexity is not warranted given how unlikely + // the case is to occur, so we re-initialize in this case. + return true; + } + if (!codecInfo.secure && maybeRequiresSecureDecoder(newMediaCrypto, newFormat)) { + // Re-initialization is required because newSession might require switching to the secure + // output path. return true; } + + return false; + } + + /** + * Returns whether a {@link DrmSession} may require a secure decoder for a given {@link Format}. + * + * @param sessionMediaCrypto The {@link DrmSession}'s {@link FrameworkMediaCrypto}. + * @param format The {@link Format}. + * @return Whether a secure decoder may be required. + */ + private boolean maybeRequiresSecureDecoder( + FrameworkMediaCrypto sessionMediaCrypto, Format format) { if (sessionMediaCrypto.forceAllowInsecureDecoderComponents) { return false; } @@ -2082,33 +2109,9 @@ private boolean isDecodeOnlyBuffer(long presentationTimeUs) { } @RequiresApi(23) - private void updateDrmSessionOrReinitializeCodecV23() throws ExoPlaybackException { - @Nullable FrameworkMediaCrypto sessionMediaCrypto = getFrameworkMediaCrypto(sourceDrmSession); - if (sessionMediaCrypto == null) { - // We'd only expect this to happen if the CDM from which the pending session is obtained needs - // provisioning. This is unlikely to happen (it probably requires a switch from one DRM scheme - // to another, where the new CDM hasn't been used before and needs provisioning). It would be - // possible to handle this case more efficiently (i.e. with a new renderer state that waits - // for provisioning to finish and then calls mediaCrypto.setMediaDrmSession), but the extra - // complexity is not warranted given how unlikely the case is to occur. - reinitializeCodec(); - return; - } - if (C.PLAYREADY_UUID.equals(sessionMediaCrypto.uuid)) { - // The PlayReady CDM does not implement setMediaDrmSession. - // TODO: Add API check once [Internal ref: b/128835874] is fixed. - reinitializeCodec(); - return; - } - - if (flushOrReinitializeCodec()) { - // The codec was reinitialized. The new codec will be using the new DRM session, so there's - // nothing more to do. - return; - } - + private void updateDrmSessionV23() throws ExoPlaybackException { try { - mediaCrypto.setMediaDrmSession(sessionMediaCrypto.sessionId); + mediaCrypto.setMediaDrmSession(getFrameworkMediaCrypto(sourceDrmSession).sessionId); } catch (MediaCryptoException e) { throw createRendererException(e, inputFormat); } @@ -2145,103 +2148,106 @@ private FrameworkMediaCrypto getFrameworkMediaCrypto(DrmSession drmSession) */ private boolean bypassRender(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackException { - BatchBuffer batchBuffer = bypassBatchBuffer; - // Process any data in the batch buffer. + // Process any batched data. checkState(!outputStreamEnded); - if (!batchBuffer.isEmpty()) { // Optimisation: Do not process buffer if empty. + if (bypassBatchBuffer.hasSamples()) { if (processOutputBuffer( positionUs, elapsedRealtimeUs, /* codec= */ null, - batchBuffer.data, + bypassBatchBuffer.data, outputIndex, /* bufferFlags= */ 0, - batchBuffer.getAccessUnitCount(), - batchBuffer.getFirstAccessUnitTimeUs(), - batchBuffer.isDecodeOnly(), - batchBuffer.isEndOfStream(), + bypassBatchBuffer.getSampleCount(), + bypassBatchBuffer.getFirstSampleTimeUs(), + bypassBatchBuffer.isDecodeOnly(), + bypassBatchBuffer.isEndOfStream(), outputFormat)) { - onProcessedOutputBuffer(batchBuffer.getLastAccessUnitTimeUs()); + // The batch buffer has been fully processed. + onProcessedOutputBuffer(bypassBatchBuffer.getLastSampleTimeUs()); + bypassBatchBuffer.clear(); } else { - // Could not process the whole buffer. Try again later. + // Could not process the whole batch buffer. Try again later. return false; } } - // Process the end of stream, if it has been reached. - if (batchBuffer.isEndOfStream()) { + + // Process end of stream, if reached. + if (inputStreamEnded) { outputStreamEnded = true; return false; } - batchBuffer.batchWasConsumed(); + + if (bypassSampleBufferPending) { + Assertions.checkState(bypassBatchBuffer.append(bypassSampleBuffer)); + bypassSampleBufferPending = false; + } if (bypassDrainAndReinitialize) { - if (!batchBuffer.isEmpty()) { - return true; // Drain the batch buffer before propagating the format change. + if (bypassBatchBuffer.hasSamples()) { + // This can only happen if bypassSampleBufferPending was true above. Return true to try and + // immediately process the sample, which has now been appended to the batch buffer. + return true; } - disableBypass(); // The new format might require a codec. + // The new format might require using a codec rather than bypass. + disableBypass(); bypassDrainAndReinitialize = false; maybeInitCodecOrBypass(); if (!bypassEnabled) { - return false; // The new format is not supported in codec bypass. + // We're no longer in bypass mode. + return false; } } - // Now refill the empty buffer for the next iteration. - checkState(!inputStreamEnded); - FormatHolder formatHolder = getFormatHolder(); - boolean formatChange = readBatchFromSource(formatHolder, batchBuffer); - - if (!batchBuffer.isEmpty() && waitingForFirstSampleInFormat) { - // This is the first buffer in a new format, the output format must be updated. - outputFormat = Assertions.checkNotNull(inputFormat); - onOutputFormatChanged(outputFormat, /* mediaFormat= */ null); - waitingForFirstSampleInFormat = false; - } - - if (formatChange) { - onInputFormatChanged(formatHolder); - } + // Read from the input, appending any sample buffers to the batch buffer. + bypassRead(); - boolean haveDataToProcess = false; - if (batchBuffer.isEndOfStream()) { - inputStreamEnded = true; - haveDataToProcess = true; - } - if (!batchBuffer.isEmpty()) { - batchBuffer.flip(); - haveDataToProcess = true; + if (bypassBatchBuffer.hasSamples()) { + bypassBatchBuffer.flip(); } - return haveDataToProcess; + // We can make more progress if we have batched data, an EOS, or a re-initialization to process + // (note that one or more of the code blocks above will be executed during the next call). + return bypassBatchBuffer.hasSamples() || inputStreamEnded || bypassDrainAndReinitialize; } - /** - * Fills the buffer with multiple access unit from the source. Has otherwise the same semantic as - * {@link #readSource(FormatHolder, DecoderInputBuffer, boolean)}. Will stop early on format - * change, EOS or source starvation. - * - * @return If the format has changed. - */ - private boolean readBatchFromSource(FormatHolder formatHolder, BatchBuffer batchBuffer) { - while (!batchBuffer.isFull() && !batchBuffer.isEndOfStream()) { + private void bypassRead() throws ExoPlaybackException { + checkState(!inputStreamEnded); + FormatHolder formatHolder = getFormatHolder(); + bypassSampleBuffer.clear(); + while (true) { + bypassSampleBuffer.clear(); @SampleStream.ReadDataResult - int result = - readSource( - formatHolder, batchBuffer.getNextAccessUnitBuffer(), /* formatRequired= */ false); + int result = readSource(formatHolder, bypassSampleBuffer, /* formatRequired= */ false); switch (result) { case C.RESULT_FORMAT_READ: - return true; + onInputFormatChanged(formatHolder); + return; case C.RESULT_NOTHING_READ: - return false; + return; case C.RESULT_BUFFER_READ: - batchBuffer.commitNextAccessUnit(); + if (bypassSampleBuffer.isEndOfStream()) { + inputStreamEnded = true; + return; + } + if (waitingForFirstSampleInFormat) { + // This is the first buffer in a new format, the output format must be updated. + outputFormat = checkNotNull(inputFormat); + onOutputFormatChanged(outputFormat, /* mediaFormat= */ null); + waitingForFirstSampleInFormat = false; + } + // Try to append the buffer to the batch buffer. + bypassSampleBuffer.flip(); + if (!bypassBatchBuffer.append(bypassSampleBuffer)) { + bypassSampleBufferPending = true; + return; + } break; default: - throw new IllegalStateException(); // Unsupported result + throw new IllegalStateException(); } } - return false; } private static boolean isMediaCodecException(IllegalStateException error) { @@ -2304,21 +2310,6 @@ private static boolean codecNeedsFlushWorkaround(String name) { } } - /** - * Returns whether the decoder is known to fail when an attempt is made to reconfigure it with a - * new format's configuration data. - * - *

    When enabled, the workaround will always release and recreate the decoder, rather than - * attempting to reconfigure the existing instance. - * - * @param name The name of the decoder. - * @return True if the decoder is known to fail when an attempt is made to reconfigure it with a - * new format's configuration data. - */ - private static boolean codecNeedsReconfigureWorkaround(String name) { - return Util.MODEL.startsWith("SM-T230") && "OMX.MARVELL.VIDEO.HW.CODA7542DECODER".equals(name); - } - /** * Returns whether the decoder is an H.264/AVC decoder known to fail if NAL units are queued * before the codec specific data. @@ -2335,6 +2326,23 @@ private static boolean codecNeedsDiscardToSpsWorkaround(String name, Format form && "OMX.MTK.VIDEO.DECODER.AVC".equals(name); } + /** + * Returns whether the decoder is known to behave incorrectly if flushed prior to having output a + * {@link MediaFormat}. + * + *

    If true is returned, the renderer will work around the issue by instantiating a new decoder + * when this case occurs. + * + *

    See [Internal: b/141097367]. + * + * @param name The name of the decoder. + * @return True if the decoder is known to behave incorrectly if flushed prior to having output a + * {@link MediaFormat}. False otherwise. + */ + private static boolean codecNeedsSosFlushWorkaround(String name) { + return Util.SDK_INT == 29 && "c2.android.aac.decoder".equals(name); + } + /** * Returns whether the decoder is known to handle the propagation of the {@link * MediaCodec#BUFFER_FLAG_END_OF_STREAM} flag incorrectly on the host device. @@ -2379,12 +2387,30 @@ private static boolean codecNeedsEosFlushWorkaround(String name) { } /** - * Returns whether the decoder may throw an {@link IllegalStateException} from - * {@link MediaCodec#dequeueOutputBuffer(MediaCodec.BufferInfo, long)} or - * {@link MediaCodec#releaseOutputBuffer(int, boolean)} after receiving an input - * buffer with {@link MediaCodec#BUFFER_FLAG_END_OF_STREAM} set. - *

    - * See [Internal: b/17933838]. + * Returns whether the decoder may output a non-empty buffer with timestamp 0 as the end of stream + * buffer. + * + *

    See GitHub issue #5045. + */ + private static boolean codecNeedsEosBufferTimestampWorkaround(String codecName) { + return Util.SDK_INT < 21 + && "OMX.SEC.mp3.dec".equals(codecName) + && "samsung".equals(Util.MANUFACTURER) + && (Util.DEVICE.startsWith("baffin") + || Util.DEVICE.startsWith("grand") + || Util.DEVICE.startsWith("fortuna") + || Util.DEVICE.startsWith("gprimelte") + || Util.DEVICE.startsWith("j2y18lte") + || Util.DEVICE.startsWith("ms01")); + } + + /** + * Returns whether the decoder may throw an {@link IllegalStateException} from {@link + * MediaCodec#dequeueOutputBuffer(MediaCodec.BufferInfo, long)} or {@link + * MediaCodec#releaseOutputBuffer(int, boolean)} after receiving an input buffer with {@link + * MediaCodec#BUFFER_FLAG_END_OF_STREAM} set. + * + *

    See [Internal: b/17933838]. * * @param name The name of the decoder. * @return True if the decoder may throw an exception after receiving an end-of-stream buffer. @@ -2411,21 +2437,4 @@ private static boolean codecNeedsMonoChannelCountWorkaround(String name, Format return Util.SDK_INT <= 18 && format.channelCount == 1 && "OMX.MTK.AUDIO.DECODER.MP3".equals(name); } - - /** - * Returns whether the decoder is known to behave incorrectly if flushed prior to having output a - * {@link MediaFormat}. - * - *

    If true is returned, the renderer will work around the issue by instantiating a new decoder - * when this case occurs. - * - *

    See [Internal: b/141097367]. - * - * @param name The name of the decoder. - * @return True if the decoder is known to behave incorrectly if flushed prior to having output a - * {@link MediaFormat}. False otherwise. - */ - private static boolean codecNeedsSosFlushWorkaround(String name) { - return Util.SDK_INT == 29 && "c2.android.aac.decoder".equals(name); - } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecUtil.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecUtil.java index 64eb0bb8374..bcaf8a07b15 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecUtil.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecUtil.java @@ -24,6 +24,7 @@ import android.text.TextUtils; import android.util.Pair; import androidx.annotation.CheckResult; +import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import com.google.android.exoplayer2.C; @@ -63,6 +64,7 @@ private DecoderQueryException(Throwable cause) { private static final String TAG = "MediaCodecUtil"; private static final Pattern PROFILE_PATTERN = Pattern.compile("^\\D?(\\d+)$"); + @GuardedBy("MediaCodecUtil.class") private static final HashMap> decoderInfosCache = new HashMap<>(); // Codecs to constant mappings. @@ -105,6 +107,15 @@ public static void warmDecoderInfoCache(String mimeType, boolean secure, boolean } } + /** + * Clears the codec cache. + * + *

    This method should only be called in tests. + */ + public static synchronized void clearDecoderInfoCache() { + decoderInfosCache.clear(); + } + /** * Returns information about a decoder that will only decrypt data, without decoding it. * @@ -134,7 +145,7 @@ public static MediaCodecInfo getDecoderInfo(String mimeType, boolean secure, boo return decoderInfos.isEmpty() ? null : decoderInfos.get(0); } - /** + /* * Returns all {@link MediaCodecInfo}s for the given mime type, in the order given by {@link * MediaCodecList}. * @@ -312,7 +323,6 @@ private static ArrayList getDecoderInfosInternal( boolean hardwareAccelerated = isHardwareAccelerated(codecInfo); boolean softwareOnly = isSoftwareOnly(codecInfo); boolean vendor = isVendor(codecInfo); - boolean forceDisableAdaptive = codecNeedsDisableAdaptationWorkaround(name); if ((secureDecodersExplicit && key.secure == secureSupported) || (!secureDecodersExplicit && !key.secure)) { decoderInfos.add( @@ -324,7 +334,7 @@ private static ArrayList getDecoderInfosInternal( hardwareAccelerated, softwareOnly, vendor, - forceDisableAdaptive, + /* forceDisableAdaptive= */ false, /* forceSecure= */ false)); } else if (!secureDecodersExplicit && secureSupported) { decoderInfos.add( @@ -336,7 +346,7 @@ private static ArrayList getDecoderInfosInternal( hardwareAccelerated, softwareOnly, vendor, - forceDisableAdaptive, + /* forceDisableAdaptive= */ false, /* forceSecure= */ true)); // It only makes sense to have one synthesized secure decoder, return immediately. return decoderInfos; @@ -651,19 +661,6 @@ private static boolean isVendorV29(android.media.MediaCodecInfo codecInfo) { return codecInfo.isVendor(); } - /** - * Returns whether the decoder is known to fail when adapting, despite advertising itself as an - * adaptive decoder. - * - * @param name The decoder name. - * @return True if the decoder is known to fail when adapting. - */ - private static boolean codecNeedsDisableAdaptationWorkaround(String name) { - return Util.SDK_INT <= 22 - && ("ODROID-XU3".equals(Util.MODEL) || "Nexus 10".equals(Util.MODEL)) - && ("OMX.Exynos.AVC.Decoder".equals(name) || "OMX.Exynos.AVC.Decoder.secure".equals(name)); - } - @Nullable private static Pair getDolbyVisionProfileAndLevel( String codec, String[] parts) { @@ -1275,6 +1272,7 @@ private static Integer dolbyVisionStringToLevel(@Nullable String levelString) { if (levelString == null) { return null; } + // TODO (Internal: b/179261323): use framework constants for levels 10 to 13. switch (levelString) { case "01": return CodecProfileLevel.DolbyVisionLevelHd24; @@ -1294,6 +1292,14 @@ private static Integer dolbyVisionStringToLevel(@Nullable String levelString) { return CodecProfileLevel.DolbyVisionLevelUhd48; case "09": return CodecProfileLevel.DolbyVisionLevelUhd60; + case "10": + return 0x200; + case "11": + return 0x400; + case "12": + return 0x800; + case "13": + return 0x1000; default: return null; } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/SynchronousMediaCodecAdapter.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/SynchronousMediaCodecAdapter.java index f5138e90f06..db1401cd525 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/SynchronousMediaCodecAdapter.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/SynchronousMediaCodecAdapter.java @@ -16,21 +16,39 @@ package com.google.android.exoplayer2.mediacodec; +import static com.google.android.exoplayer2.util.Util.castNonNull; + import android.media.MediaCodec; import android.media.MediaCrypto; import android.media.MediaFormat; +import android.os.Bundle; +import android.os.Handler; import android.view.Surface; import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.decoder.CryptoInfo; +import com.google.android.exoplayer2.util.Util; +import java.nio.ByteBuffer; /** * A {@link MediaCodecAdapter} that operates the underlying {@link MediaCodec} in synchronous mode. */ -/* package */ final class SynchronousMediaCodecAdapter implements MediaCodecAdapter { +public final class SynchronousMediaCodecAdapter implements MediaCodecAdapter { + + /** A factory for {@link SynchronousMediaCodecAdapter} instances. */ + public static final class Factory implements MediaCodecAdapter.Factory { + @Override + public MediaCodecAdapter createAdapter(MediaCodec codec) { + return new SynchronousMediaCodecAdapter(codec); + } + } private final MediaCodec codec; + @Nullable private ByteBuffer[] inputByteBuffers; + @Nullable private ByteBuffer[] outputByteBuffers; - public SynchronousMediaCodecAdapter(MediaCodec mediaCodec) { + private SynchronousMediaCodecAdapter(MediaCodec mediaCodec) { this.codec = mediaCodec; } @@ -46,6 +64,10 @@ public void configure( @Override public void start() { codec.start(); + if (Util.SDK_INT < 21) { + inputByteBuffers = codec.getInputBuffers(); + outputByteBuffers = codec.getOutputBuffers(); + } } @Override @@ -55,7 +77,15 @@ public int dequeueInputBufferIndex() { @Override public int dequeueOutputBufferIndex(MediaCodec.BufferInfo bufferInfo) { - return codec.dequeueOutputBuffer(bufferInfo, 0); + int index; + do { + index = codec.dequeueOutputBuffer(bufferInfo, 0); + if (index == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED && Util.SDK_INT < 21) { + outputByteBuffers = codec.getOutputBuffers(); + } + } while (index == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED); + + return index; } @Override @@ -63,6 +93,26 @@ public MediaFormat getOutputFormat() { return codec.getOutputFormat(); } + @Override + @Nullable + public ByteBuffer getInputBuffer(int index) { + if (Util.SDK_INT >= 21) { + return codec.getInputBuffer(index); + } else { + return castNonNull(inputByteBuffers)[index]; + } + } + + @Override + @Nullable + public ByteBuffer getOutputBuffer(int index) { + if (Util.SDK_INT >= 21) { + return codec.getOutputBuffer(index); + } else { + return castNonNull(outputByteBuffers)[index]; + } + } + @Override public void queueInputBuffer( int index, int offset, int size, long presentationTimeUs, int flags) { @@ -76,16 +126,53 @@ public void queueSecureInputBuffer( index, offset, info.getFrameworkCryptoInfo(), presentationTimeUs, flags); } + @Override + public void releaseOutputBuffer(int index, boolean render) { + codec.releaseOutputBuffer(index, render); + } + + @Override + @RequiresApi(21) + public void releaseOutputBuffer(int index, long renderTimeStampNs) { + codec.releaseOutputBuffer(index, renderTimeStampNs); + } + @Override public void flush() { codec.flush(); } @Override - public void shutdown() {} + public void release() { + inputByteBuffers = null; + outputByteBuffers = null; + codec.release(); + } + + @Override + @RequiresApi(23) + public void setOnFrameRenderedListener(OnFrameRenderedListener listener, Handler handler) { + codec.setOnFrameRenderedListener( + (codec, presentationTimeUs, nanoTime) -> + listener.onFrameRendered( + SynchronousMediaCodecAdapter.this, presentationTimeUs, nanoTime), + handler); + } + + @Override + @RequiresApi(23) + public void setOutputSurface(Surface surface) { + codec.setOutputSurface(surface); + } + + @Override + @RequiresApi(19) + public void setParameters(Bundle params) { + codec.setParameters(params); + } @Override - public MediaCodec getCodec() { - return codec; + public void setVideoScalingMode(@C.VideoScalingMode int scalingMode) { + codec.setVideoScalingMode(scalingMode); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/SynchronousMediaCodecBufferEnqueuer.java b/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/SynchronousMediaCodecBufferEnqueuer.java deleted file mode 100644 index f16748f8fc5..00000000000 --- a/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/SynchronousMediaCodecBufferEnqueuer.java +++ /dev/null @@ -1,59 +0,0 @@ -/* - * Copyright (C) 2020 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.google.android.exoplayer2.mediacodec; - -import android.media.MediaCodec; -import com.google.android.exoplayer2.decoder.CryptoInfo; - -/** - * A {@link MediaCodecInputBufferEnqueuer} that forwards queueing methods directly to {@link - * MediaCodec}. - */ -class SynchronousMediaCodecBufferEnqueuer implements MediaCodecInputBufferEnqueuer { - private final MediaCodec codec; - - /** - * Creates an instance that queues input buffers on the specified {@link MediaCodec}. - * - * @param codec The {@link MediaCodec} to submit input buffers to. - */ - SynchronousMediaCodecBufferEnqueuer(MediaCodec codec) { - this.codec = codec; - } - - @Override - public void start() {} - - @Override - public void queueInputBuffer( - int index, int offset, int size, long presentationTimeUs, int flags) { - codec.queueInputBuffer(index, offset, size, presentationTimeUs, flags); - } - - @Override - public void queueSecureInputBuffer( - int index, int offset, CryptoInfo info, long presentationTimeUs, int flags) { - codec.queueSecureInputBuffer( - index, offset, info.getFrameworkCryptoInfo(), presentationTimeUs, flags); - } - - @Override - public void flush() {} - - @Override - public void shutdown() {} -} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/metadata/MetadataRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/metadata/MetadataRenderer.java index d2b75635b1f..5ae7cca66c6 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/metadata/MetadataRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/metadata/MetadataRenderer.java @@ -103,9 +103,9 @@ public String getName() { public int supportsFormat(Format format) { if (decoderFactory.supportsFormat(format)) { return RendererCapabilities.create( - format.exoMediaCryptoType == null ? FORMAT_HANDLED : FORMAT_UNSUPPORTED_DRM); + format.exoMediaCryptoType == null ? C.FORMAT_HANDLED : C.FORMAT_UNSUPPORTED_DRM); } else { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/offline/DefaultDownloadIndex.java b/library/core/src/main/java/com/google/android/exoplayer2/offline/DefaultDownloadIndex.java index 3a18d4da10d..2a62b0602c9 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/offline/DefaultDownloadIndex.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/offline/DefaultDownloadIndex.java @@ -403,7 +403,7 @@ private Cursor getCursor(String selection, @Nullable String[] selectionArgs) } @VisibleForTesting - /* package*/ static String encodeStreamKeys(List streamKeys) { + /* package */ static String encodeStreamKeys(List streamKeys) { StringBuilder stringBuilder = new StringBuilder(); for (int i = 0; i < streamKeys.size(); i++) { StreamKey streamKey = streamKeys.get(i); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/offline/DownloadCursor.java b/library/core/src/main/java/com/google/android/exoplayer2/offline/DownloadCursor.java index a1822fca979..ce0c84d7004 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/offline/DownloadCursor.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/offline/DownloadCursor.java @@ -28,7 +28,7 @@ public interface DownloadCursor extends Closeable { /** * Returns the current position of the cursor in the download set. The value is zero-based. When - * the download set is first returned the cursor will be at positon -1, which is before the first + * the download set is first returned the cursor will be at position -1, which is before the first * download. After the last download is returned another call to next() will leave the cursor past * the last entry, at a position of count(). * diff --git a/library/core/src/main/java/com/google/android/exoplayer2/offline/DownloadHelper.java b/library/core/src/main/java/com/google/android/exoplayer2/offline/DownloadHelper.java index ba8a799381a..27ff0a7956a 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/offline/DownloadHelper.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/offline/DownloadHelper.java @@ -48,8 +48,8 @@ import com.google.android.exoplayer2.trackselection.DefaultTrackSelector; import com.google.android.exoplayer2.trackselection.DefaultTrackSelector.Parameters; import com.google.android.exoplayer2.trackselection.DefaultTrackSelector.SelectionOverride; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.trackselection.MappingTrackSelector.MappedTrackInfo; -import com.google.android.exoplayer2.trackselection.TrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelectorResult; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.BandwidthMeter; @@ -341,7 +341,7 @@ public static DownloadHelper forMediaItem(Context context, MediaItem mediaItem) * streams. This argument is required for adaptive streams and ignored for progressive * streams. * @return A {@link DownloadHelper}. - * @throws IllegalStateException If the the corresponding module is missing for DASH, HLS or + * @throws IllegalStateException If the corresponding module is missing for DASH, HLS or * SmoothStreaming media items. * @throws IllegalArgumentException If the {@code dataSourceFactory} is null for adaptive streams. */ @@ -370,7 +370,7 @@ public static DownloadHelper forMediaItem( * streams. This argument is required for adaptive streams and ignored for progressive * streams. * @return A {@link DownloadHelper}. - * @throws IllegalStateException If the the corresponding module is missing for DASH, HLS or + * @throws IllegalStateException If the corresponding module is missing for DASH, HLS or * SmoothStreaming media items. * @throws IllegalArgumentException If the {@code dataSourceFactory} is null for adaptive streams. */ @@ -401,7 +401,7 @@ public static DownloadHelper forMediaItem( * @param drmSessionManager An optional {@link DrmSessionManager}. Used to help determine which * tracks can be selected. * @return A {@link DownloadHelper}. - * @throws IllegalStateException If the the corresponding module is missing for DASH, HLS or + * @throws IllegalStateException If the corresponding module is missing for DASH, HLS or * SmoothStreaming media items. * @throws IllegalArgumentException If the {@code dataSourceFactory} is null for adaptive streams. */ @@ -465,8 +465,9 @@ public static MediaSource createMediaSource( private @MonotonicNonNull MediaPreparer mediaPreparer; private TrackGroupArray @MonotonicNonNull [] trackGroupArrays; private MappedTrackInfo @MonotonicNonNull [] mappedTrackInfos; - private List @MonotonicNonNull [][] trackSelectionsByPeriodAndRenderer; - private List @MonotonicNonNull [][] immutableTrackSelectionsByPeriodAndRenderer; + private List @MonotonicNonNull [][] trackSelectionsByPeriodAndRenderer; + private List @MonotonicNonNull [][] + immutableTrackSelectionsByPeriodAndRenderer; /** * Creates download helper. @@ -573,14 +574,14 @@ public MappedTrackInfo getMappedTrackInfo(int periodIndex) { } /** - * Returns all {@link TrackSelection track selections} for a period and renderer. Must not be + * Returns all {@link ExoTrackSelection track selections} for a period and renderer. Must not be * called until after preparation completes. * * @param periodIndex The period index. * @param rendererIndex The renderer index. - * @return A list of selected {@link TrackSelection track selections}. + * @return A list of selected {@link ExoTrackSelection track selections}. */ - public List getTrackSelections(int periodIndex, int rendererIndex) { + public List getTrackSelections(int periodIndex, int rendererIndex) { assertPreparedWithMedia(); return immutableTrackSelectionsByPeriodAndRenderer[periodIndex][rendererIndex]; } @@ -751,7 +752,7 @@ public DownloadRequest getDownloadRequest(String id, @Nullable byte[] data) { } assertPreparedWithMedia(); List streamKeys = new ArrayList<>(); - List allSelections = new ArrayList<>(); + List allSelections = new ArrayList<>(); int periodCount = trackSelectionsByPeriodAndRenderer.length; for (int periodIndex = 0; periodIndex < periodCount; periodIndex++) { allSelections.clear(); @@ -773,9 +774,9 @@ private void onMediaPrepared() { int periodCount = mediaPreparer.mediaPeriods.length; int rendererCount = rendererCapabilities.length; trackSelectionsByPeriodAndRenderer = - (List[][]) new List[periodCount][rendererCount]; + (List[][]) new List[periodCount][rendererCount]; immutableTrackSelectionsByPeriodAndRenderer = - (List[][]) new List[periodCount][rendererCount]; + (List[][]) new List[periodCount][rendererCount]; for (int i = 0; i < periodCount; i++) { for (int j = 0; j < rendererCount; j++) { trackSelectionsByPeriodAndRenderer[i][j] = new ArrayList<>(); @@ -847,15 +848,15 @@ private TrackSelectorResult runTrackSelection(int periodIndex) { new MediaPeriodId(mediaPreparer.timeline.getUidOfPeriod(periodIndex)), mediaPreparer.timeline); for (int i = 0; i < trackSelectorResult.length; i++) { - @Nullable TrackSelection newSelection = trackSelectorResult.selections.get(i); + @Nullable ExoTrackSelection newSelection = trackSelectorResult.selections[i]; if (newSelection == null) { continue; } - List existingSelectionList = + List existingSelectionList = trackSelectionsByPeriodAndRenderer[periodIndex][i]; boolean mergedWithExistingSelection = false; for (int j = 0; j < existingSelectionList.size(); j++) { - TrackSelection existingSelection = existingSelectionList.get(j); + ExoTrackSelection existingSelection = existingSelectionList.get(j); if (existingSelection.getTrackGroup() == newSelection.getTrackGroup()) { // Merge with existing selection. scratchSet.clear(); @@ -1003,7 +1004,7 @@ public void onSourceInfoRefreshed(MediaSource source, Timeline timeline) { // Ignore dynamic updates. return; } - if (timeline.getWindow(/* windowIndex= */ 0, new Timeline.Window()).isLive) { + if (timeline.getWindow(/* windowIndex= */ 0, new Timeline.Window()).isLive()) { downloadHelperHandler .obtainMessage( DOWNLOAD_HELPER_CALLBACK_MESSAGE_FAILED, @@ -1066,12 +1067,15 @@ private boolean handleDownloadHelperCallbackMessage(Message msg) { private static final class DownloadTrackSelection extends BaseTrackSelection { - private static final class Factory implements TrackSelection.Factory { + private static final class Factory implements ExoTrackSelection.Factory { @Override - public @NullableType TrackSelection[] createTrackSelections( - @NullableType Definition[] definitions, BandwidthMeter bandwidthMeter) { - @NullableType TrackSelection[] selections = new TrackSelection[definitions.length]; + public @NullableType ExoTrackSelection[] createTrackSelections( + @NullableType Definition[] definitions, + BandwidthMeter bandwidthMeter, + MediaPeriodId mediaPeriodId, + Timeline timeline) { + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[definitions.length]; for (int i = 0; i < definitions.length; i++) { selections[i] = definitions[i] == null diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/ClippingMediaPeriod.java b/library/core/src/main/java/com/google/android/exoplayer2/source/ClippingMediaPeriod.java index 7bb6a83add2..650e055f0b3 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/ClippingMediaPeriod.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/ClippingMediaPeriod.java @@ -21,7 +21,7 @@ import com.google.android.exoplayer2.FormatHolder; import com.google.android.exoplayer2.SeekParameters; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.Util; @@ -34,9 +34,7 @@ */ public final class ClippingMediaPeriod implements MediaPeriod, MediaPeriod.Callback { - /** - * The {@link MediaPeriod} wrapped by this clipping media period. - */ + /** The {@link MediaPeriod} wrapped by this clipping media period. */ public final MediaPeriod mediaPeriod; @Nullable private MediaPeriod.Callback callback; @@ -98,7 +96,7 @@ public TrackGroupArray getTrackGroups() { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, @@ -250,7 +248,7 @@ private SeekParameters clipSeekParameters(long positionUs, SeekParameters seekPa } private static boolean shouldKeepInitialDiscontinuity( - long startUs, @NullableType TrackSelection[] selections) { + long startUs, @NullableType ExoTrackSelection[] selections) { // If the clipping start position is non-zero, the clipping sample streams will adjust // timestamps on buffers they read from the unclipped sample streams. These adjusted buffer // timestamps can be negative, because sample streams provide buffers starting at a key-frame, @@ -261,7 +259,7 @@ private static boolean shouldKeepInitialDiscontinuity( // However, for tracks where all samples are sync samples, we assume they have random access // seek behaviour and do not need an initial discontinuity to reset the renderer. if (startUs != 0) { - for (TrackSelection trackSelection : selections) { + for (ExoTrackSelection trackSelection : selections) { if (trackSelection != null) { Format selectedFormat = trackSelection.getSelectedFormat(); if (!MimeTypes.allSamplesAreSyncSamples( @@ -274,9 +272,7 @@ private static boolean shouldKeepInitialDiscontinuity( return false; } - /** - * Wraps a {@link SampleStream} and clips its samples. - */ + /** Wraps a {@link SampleStream} and clips its samples. */ private final class ClippingSampleStream implements SampleStream { public final SampleStream childStream; @@ -302,8 +298,8 @@ public void maybeThrowError() throws IOException { } @Override - public int readData(FormatHolder formatHolder, DecoderInputBuffer buffer, - boolean requireFormat) { + public int readData( + FormatHolder formatHolder, DecoderInputBuffer buffer, boolean requireFormat) { if (isPendingInitialDiscontinuity()) { return C.RESULT_NOTHING_READ; } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/DefaultMediaSourceFactory.java b/library/core/src/main/java/com/google/android/exoplayer2/source/DefaultMediaSourceFactory.java index b3955d50c37..31aad16b024 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/DefaultMediaSourceFactory.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/DefaultMediaSourceFactory.java @@ -15,13 +15,16 @@ */ package com.google.android.exoplayer2.source; +import static com.google.android.exoplayer2.util.Util.castNonNull; + import android.content.Context; -import android.net.Uri; +import android.util.Pair; import android.util.SparseArray; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.drm.DrmSessionManager; +import com.google.android.exoplayer2.drm.DrmSessionManagerProvider; import com.google.android.exoplayer2.extractor.DefaultExtractorsFactory; import com.google.android.exoplayer2.extractor.ExtractorsFactory; import com.google.android.exoplayer2.offline.StreamKey; @@ -72,41 +75,44 @@ * *

    Ad support for media items with ad tag URIs

    * - *

    To support media items with {@link MediaItem.PlaybackProperties#adTagUri ad tag URIs}, {@link - * #setAdsLoaderProvider} and {@link #setAdViewProvider} need to be called to configure the factory - * with the required providers. + *

    To support media items with {@link MediaItem.PlaybackProperties#adsConfiguration ads + * configuration}, {@link #setAdsLoaderProvider} and {@link #setAdViewProvider} need to be called to + * configure the factory with the required providers. */ public final class DefaultMediaSourceFactory implements MediaSourceFactory { /** * Provides {@link AdsLoader} instances for media items that have {@link - * MediaItem.PlaybackProperties#adTagUri ad tag URIs}. + * MediaItem.PlaybackProperties#adsConfiguration ad tag URIs}. */ public interface AdsLoaderProvider { /** - * Returns an {@link AdsLoader} for the given {@link MediaItem.PlaybackProperties#adTagUri ad - * tag URI}, or null if no ads loader is available for the given ad tag URI. + * Returns an {@link AdsLoader} for the given {@link + * MediaItem.PlaybackProperties#adsConfiguration ads configuration}, or {@code null} if no ads + * loader is available for the given ads configuration. * *

    This method is called each time a {@link MediaSource} is created from a {@link MediaItem} - * that defines an {@link MediaItem.PlaybackProperties#adTagUri ad tag URI}. + * that defines an {@link MediaItem.PlaybackProperties#adsConfiguration ads configuration}. */ @Nullable - AdsLoader getAdsLoader(Uri adTagUri); + AdsLoader getAdsLoader(MediaItem.AdsConfiguration adsConfiguration); } private static final String TAG = "DefaultMediaSourceFactory"; - private final MediaSourceDrmHelper mediaSourceDrmHelper; private final DataSource.Factory dataSourceFactory; private final SparseArray mediaSourceFactories; @C.ContentType private final int[] supportedTypes; @Nullable private AdsLoaderProvider adsLoaderProvider; @Nullable private AdViewProvider adViewProvider; - @Nullable private DrmSessionManager drmSessionManager; - @Nullable private List streamKeys; @Nullable private LoadErrorHandlingPolicy loadErrorHandlingPolicy; + private long liveTargetOffsetMs; + private long liveMinOffsetMs; + private long liveMaxOffsetMs; + private float liveMinSpeed; + private float liveMaxSpeed; /** * Creates a new instance. @@ -149,17 +155,21 @@ public DefaultMediaSourceFactory(DataSource.Factory dataSourceFactory) { public DefaultMediaSourceFactory( DataSource.Factory dataSourceFactory, ExtractorsFactory extractorsFactory) { this.dataSourceFactory = dataSourceFactory; - mediaSourceDrmHelper = new MediaSourceDrmHelper(); mediaSourceFactories = loadDelegates(dataSourceFactory, extractorsFactory); supportedTypes = new int[mediaSourceFactories.size()]; for (int i = 0; i < mediaSourceFactories.size(); i++) { supportedTypes[i] = mediaSourceFactories.keyAt(i); } + liveTargetOffsetMs = C.TIME_UNSET; + liveMinOffsetMs = C.TIME_UNSET; + liveMaxOffsetMs = C.TIME_UNSET; + liveMinSpeed = C.RATE_UNSET; + liveMaxSpeed = C.RATE_UNSET; } /** * Sets the {@link AdsLoaderProvider} that provides {@link AdsLoader} instances for media items - * that have {@link MediaItem.PlaybackProperties#adTagUri ad tag URIs}. + * that have {@link MediaItem.PlaybackProperties#adsConfiguration ads configurations}. * * @param adsLoaderProvider A provider for {@link AdsLoader} instances. * @return This factory, for convenience. @@ -181,23 +191,101 @@ public DefaultMediaSourceFactory setAdViewProvider(@Nullable AdViewProvider adVi return this; } + /** + * Sets the target live offset for live streams, in milliseconds. + * + * @param liveTargetOffsetMs The target live offset, in milliseconds, or {@link C#TIME_UNSET} to + * use the media-defined default. + * @return This factory, for convenience. + */ + public DefaultMediaSourceFactory setLiveTargetOffsetMs(long liveTargetOffsetMs) { + this.liveTargetOffsetMs = liveTargetOffsetMs; + return this; + } + + /** + * Sets the minimum offset from the live edge for live streams, in milliseconds. + * + * @param liveMinOffsetMs The minimum allowed live offset, in milliseconds, or {@link + * C#TIME_UNSET} to use the media-defined default. + * @return This factory, for convenience. + */ + public DefaultMediaSourceFactory setLiveMinOffsetMs(long liveMinOffsetMs) { + this.liveMinOffsetMs = liveMinOffsetMs; + return this; + } + + /** + * Sets the maximum offset from the live edge for live streams, in milliseconds. + * + * @param liveMaxOffsetMs The maximum allowed live offset, in milliseconds, or {@link + * C#TIME_UNSET} to use the media-defined default. + * @return This factory, for convenience. + */ + public DefaultMediaSourceFactory setLiveMaxOffsetMs(long liveMaxOffsetMs) { + this.liveMaxOffsetMs = liveMaxOffsetMs; + return this; + } + + /** + * Sets the minimum playback speed for live streams. + * + * @param minSpeed The minimum factor by which playback can be sped up for live streams, or {@link + * C#RATE_UNSET} to use the media-defined default. + * @return This factory, for convenience. + */ + public DefaultMediaSourceFactory setLiveMinSpeed(float minSpeed) { + this.liveMinSpeed = minSpeed; + return this; + } + + /** + * Sets the maximum playback speed for live streams. + * + * @param maxSpeed The maximum factor by which playback can be sped up for live streams, or {@link + * C#RATE_UNSET} to use the media-defined default. + * @return This factory, for convenience. + */ + public DefaultMediaSourceFactory setLiveMaxSpeed(float maxSpeed) { + this.liveMaxSpeed = maxSpeed; + return this; + } + + @SuppressWarnings("deprecation") // Calling through to the same deprecated method. @Override public DefaultMediaSourceFactory setDrmHttpDataSourceFactory( @Nullable HttpDataSource.Factory drmHttpDataSourceFactory) { - mediaSourceDrmHelper.setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + for (int i = 0; i < mediaSourceFactories.size(); i++) { + mediaSourceFactories.valueAt(i).setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + } return this; } + @SuppressWarnings("deprecation") // Calling through to the same deprecated method. @Override public DefaultMediaSourceFactory setDrmUserAgent(@Nullable String userAgent) { - mediaSourceDrmHelper.setDrmUserAgent(userAgent); + for (int i = 0; i < mediaSourceFactories.size(); i++) { + mediaSourceFactories.valueAt(i).setDrmUserAgent(userAgent); + } return this; } + @SuppressWarnings("deprecation") // Calling through to the same deprecated method. @Override public DefaultMediaSourceFactory setDrmSessionManager( @Nullable DrmSessionManager drmSessionManager) { - this.drmSessionManager = drmSessionManager; + for (int i = 0; i < mediaSourceFactories.size(); i++) { + mediaSourceFactories.valueAt(i).setDrmSessionManager(drmSessionManager); + } + return this; + } + + @Override + public DefaultMediaSourceFactory setDrmSessionManagerProvider( + @Nullable DrmSessionManagerProvider drmSessionManagerProvider) { + for (int i = 0; i < mediaSourceFactories.size(); i++) { + mediaSourceFactories.valueAt(i).setDrmSessionManagerProvider(drmSessionManagerProvider); + } return this; } @@ -205,6 +293,9 @@ public DefaultMediaSourceFactory setDrmSessionManager( public DefaultMediaSourceFactory setLoadErrorHandlingPolicy( @Nullable LoadErrorHandlingPolicy loadErrorHandlingPolicy) { this.loadErrorHandlingPolicy = loadErrorHandlingPolicy; + for (int i = 0; i < mediaSourceFactories.size(); i++) { + mediaSourceFactories.valueAt(i).setLoadErrorHandlingPolicy(loadErrorHandlingPolicy); + } return this; } @@ -212,11 +303,13 @@ public DefaultMediaSourceFactory setLoadErrorHandlingPolicy( * @deprecated Use {@link MediaItem.Builder#setStreamKeys(List)} and {@link * #createMediaSource(MediaItem)} instead. */ - @SuppressWarnings("deprecation") + @SuppressWarnings("deprecation") // Calling through to the same deprecated method. @Deprecated @Override public DefaultMediaSourceFactory setStreamKeys(@Nullable List streamKeys) { - this.streamKeys = streamKeys != null && !streamKeys.isEmpty() ? streamKeys : null; + for (int i = 0; i < mediaSourceFactories.size(); i++) { + mediaSourceFactories.valueAt(i).setStreamKeys(streamKeys); + } return this; } @@ -225,7 +318,6 @@ public int[] getSupportedTypes() { return Arrays.copyOf(supportedTypes, supportedTypes.length); } - @SuppressWarnings("deprecation") @Override public MediaSource createMediaSource(MediaItem mediaItem) { Assertions.checkNotNull(mediaItem.playbackProperties); @@ -236,17 +328,46 @@ public MediaSource createMediaSource(MediaItem mediaItem) { @Nullable MediaSourceFactory mediaSourceFactory = mediaSourceFactories.get(type); Assertions.checkNotNull( mediaSourceFactory, "No suitable media source factory found for content type: " + type); - mediaSourceFactory.setDrmSessionManager( - drmSessionManager != null ? drmSessionManager : mediaSourceDrmHelper.create(mediaItem)); - mediaSourceFactory.setStreamKeys( - !mediaItem.playbackProperties.streamKeys.isEmpty() - ? mediaItem.playbackProperties.streamKeys - : streamKeys); - mediaSourceFactory.setLoadErrorHandlingPolicy(loadErrorHandlingPolicy); + // Make sure to retain the very same media item instance, if no value needs to be overridden. + if ((mediaItem.liveConfiguration.targetOffsetMs == C.TIME_UNSET + && liveTargetOffsetMs != C.TIME_UNSET) + || (mediaItem.liveConfiguration.minPlaybackSpeed == C.RATE_UNSET + && liveMinSpeed != C.RATE_UNSET) + || (mediaItem.liveConfiguration.maxPlaybackSpeed == C.RATE_UNSET + && liveMaxSpeed != C.RATE_UNSET) + || (mediaItem.liveConfiguration.minOffsetMs == C.TIME_UNSET + && liveMinOffsetMs != C.TIME_UNSET) + || (mediaItem.liveConfiguration.maxOffsetMs == C.TIME_UNSET + && liveMaxOffsetMs != C.TIME_UNSET)) { + mediaItem = + mediaItem + .buildUpon() + .setLiveTargetOffsetMs( + mediaItem.liveConfiguration.targetOffsetMs == C.TIME_UNSET + ? liveTargetOffsetMs + : mediaItem.liveConfiguration.targetOffsetMs) + .setLiveMinPlaybackSpeed( + mediaItem.liveConfiguration.minPlaybackSpeed == C.RATE_UNSET + ? liveMinSpeed + : mediaItem.liveConfiguration.minPlaybackSpeed) + .setLiveMaxPlaybackSpeed( + mediaItem.liveConfiguration.maxPlaybackSpeed == C.RATE_UNSET + ? liveMaxSpeed + : mediaItem.liveConfiguration.maxPlaybackSpeed) + .setLiveMinOffsetMs( + mediaItem.liveConfiguration.minOffsetMs == C.TIME_UNSET + ? liveMinOffsetMs + : mediaItem.liveConfiguration.minOffsetMs) + .setLiveMaxOffsetMs( + mediaItem.liveConfiguration.maxOffsetMs == C.TIME_UNSET + ? liveMaxOffsetMs + : mediaItem.liveConfiguration.maxOffsetMs) + .build(); + } MediaSource mediaSource = mediaSourceFactory.createMediaSource(mediaItem); - List subtitles = mediaItem.playbackProperties.subtitles; + List subtitles = castNonNull(mediaItem.playbackProperties).subtitles; if (!subtitles.isEmpty()) { MediaSource[] mediaSources = new MediaSource[subtitles.size() + 1]; mediaSources[0] = mediaSource; @@ -282,8 +403,9 @@ private static MediaSource maybeClipMediaSource(MediaItem mediaItem, MediaSource private MediaSource maybeWrapWithAdsMediaSource(MediaItem mediaItem, MediaSource mediaSource) { Assertions.checkNotNull(mediaItem.playbackProperties); - @Nullable Uri adTagUri = mediaItem.playbackProperties.adTagUri; - if (adTagUri == null) { + @Nullable + MediaItem.AdsConfiguration adsConfiguration = mediaItem.playbackProperties.adsConfiguration; + if (adsConfiguration == null) { return mediaSource; } AdsLoaderProvider adsLoaderProvider = this.adsLoaderProvider; @@ -295,14 +417,17 @@ private MediaSource maybeWrapWithAdsMediaSource(MediaItem mediaItem, MediaSource + " setAdViewProvider."); return mediaSource; } - @Nullable AdsLoader adsLoader = adsLoaderProvider.getAdsLoader(adTagUri); + @Nullable AdsLoader adsLoader = adsLoaderProvider.getAdsLoader(adsConfiguration); if (adsLoader == null) { - Log.w(TAG, "Playing media without ads. No AdsLoader for provided adTagUri"); + Log.w(TAG, "Playing media without ads, as no AdsLoader was provided."); return mediaSource; } return new AdsMediaSource( mediaSource, - new DataSpec(adTagUri), + new DataSpec(adsConfiguration.adTagUri), + /* adsId= */ adsConfiguration.adsId != null + ? adsConfiguration.adsId + : Pair.create(mediaItem.mediaId, adsConfiguration.adTagUri), /* adMediaSourceFactory= */ this, adsLoader, adViewProvider); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/ExtractorMediaSource.java b/library/core/src/main/java/com/google/android/exoplayer2/source/ExtractorMediaSource.java index 38146c92b24..c2fa35275ca 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/ExtractorMediaSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/ExtractorMediaSource.java @@ -23,6 +23,7 @@ import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.drm.DrmSessionManager; +import com.google.android.exoplayer2.drm.DrmSessionManagerProvider; import com.google.android.exoplayer2.extractor.DefaultExtractorsFactory; import com.google.android.exoplayer2.extractor.Extractor; import com.google.android.exoplayer2.extractor.ExtractorsFactory; @@ -122,18 +123,10 @@ public Factory setTag(@Nullable Object tag) { return this; } - /** @deprecated Use {@link #setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)} instead. */ - @Deprecated - public Factory setMinLoadableRetryCount(int minLoadableRetryCount) { - return setLoadErrorHandlingPolicy(new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount)); - } - /** * Sets the {@link LoadErrorHandlingPolicy}. The default value is created by calling {@link * DefaultLoadErrorHandlingPolicy#DefaultLoadErrorHandlingPolicy()}. * - *

    Calling this method overrides any calls to {@link #setMinLoadableRetryCount(int)}. - * * @param loadErrorHandlingPolicy A {@link LoadErrorHandlingPolicy}. * @return This factory, for convenience. */ @@ -162,6 +155,18 @@ public Factory setContinueLoadingCheckIntervalBytes(int continueLoadingCheckInte return this; } + /** + * @deprecated Use {@link + * ProgressiveMediaSource.Factory#setDrmSessionManagerProvider(DrmSessionManagerProvider)} + * instead. + */ + @Deprecated + @Override + public Factory setDrmSessionManagerProvider( + @Nullable DrmSessionManagerProvider drmSessionManagerProvider) { + throw new UnsupportedOperationException(); + } + /** @deprecated Use {@link ProgressiveMediaSource.Factory#setDrmSessionManager} instead. */ @Deprecated @Override @@ -214,20 +219,6 @@ public ExtractorMediaSource createMediaSource(MediaItem mediaItem) { mediaItem.playbackProperties.tag != null ? mediaItem.playbackProperties.tag : tag); } - /** - * @deprecated Use {@link #createMediaSource(MediaItem)} and {@link #addEventListener(Handler, - * MediaSourceEventListener)} instead. - */ - @Deprecated - public ExtractorMediaSource createMediaSource( - Uri uri, @Nullable Handler eventHandler, @Nullable MediaSourceEventListener eventListener) { - ExtractorMediaSource mediaSource = createMediaSource(uri); - if (eventHandler != null && eventListener != null) { - mediaSource.addEventListener(eventHandler, eventListener); - } - return mediaSource; - } - @Override public int[] getSupportedTypes() { return new int[] {C.TYPE_OTHER}; @@ -346,7 +337,7 @@ private ExtractorMediaSource( .build(), dataSourceFactory, extractorsFactory, - DrmSessionManager.getDummyDrmSessionManager(), + DrmSessionManager.DRM_UNSUPPORTED, loadableLoadErrorHandlingPolicy, continueLoadingCheckIntervalBytes); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MaskingMediaPeriod.java b/library/core/src/main/java/com/google/android/exoplayer2/source/MaskingMediaPeriod.java index a69835532f4..2151119abf5 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/MaskingMediaPeriod.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/MaskingMediaPeriod.java @@ -23,7 +23,7 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.SeekParameters; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import java.io.IOException; import org.checkerframework.checker.nullness.compatqual.NullableType; @@ -173,7 +173,7 @@ public TrackGroupArray getTrackGroups() { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MaskingMediaSource.java b/library/core/src/main/java/com/google/android/exoplayer2/source/MaskingMediaSource.java index 5d2e1c6fb7b..f9cf5af50db 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/MaskingMediaSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/MaskingMediaSource.java @@ -374,7 +374,7 @@ public Window getWindow(int windowIndex, Window window, long defaultPositionProj /* isSeekable= */ false, // Dynamic window to indicate pending timeline updates. /* isDynamic= */ true, - /* isLive= */ false, + /* liveConfiguration= */ null, /* defaultPositionUs= */ 0, /* durationUs= */ C.TIME_UNSET, /* firstPeriodIndex= */ 0, diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaParserExtractorAdapter.java b/library/core/src/main/java/com/google/android/exoplayer2/source/MediaParserExtractorAdapter.java new file mode 100644 index 00000000000..6cb20c9fbe7 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/MediaParserExtractorAdapter.java @@ -0,0 +1,121 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source; + +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_EAGERLY_EXPOSE_TRACK_TYPE; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_INCLUDE_SUPPLEMENTAL_DATA; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_IN_BAND_CRYPTO_INFO; + +import android.annotation.SuppressLint; +import android.media.MediaParser; +import android.media.MediaParser.SeekPoint; +import android.net.Uri; +import android.util.Pair; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.extractor.Extractor; +import com.google.android.exoplayer2.extractor.ExtractorOutput; +import com.google.android.exoplayer2.extractor.PositionHolder; +import com.google.android.exoplayer2.source.mediaparser.InputReaderAdapterV30; +import com.google.android.exoplayer2.source.mediaparser.OutputConsumerAdapterV30; +import com.google.android.exoplayer2.upstream.DataReader; +import java.io.IOException; +import java.util.List; +import java.util.Map; + +/** {@link ProgressiveMediaExtractor} implemented on top of the platform's {@link MediaParser}. */ +@RequiresApi(30) +/* package */ final class MediaParserExtractorAdapter implements ProgressiveMediaExtractor { + + private final OutputConsumerAdapterV30 outputConsumerAdapter; + private final InputReaderAdapterV30 inputReaderAdapter; + private final MediaParser mediaParser; + private String parserName; + + @SuppressLint("WrongConstant") + public MediaParserExtractorAdapter() { + // TODO: Add support for injecting the desired extractor list. + outputConsumerAdapter = new OutputConsumerAdapterV30(); + inputReaderAdapter = new InputReaderAdapterV30(); + mediaParser = MediaParser.create(outputConsumerAdapter); + mediaParser.setParameter(PARAMETER_EAGERLY_EXPOSE_TRACK_TYPE, true); + mediaParser.setParameter(PARAMETER_IN_BAND_CRYPTO_INFO, true); + mediaParser.setParameter(PARAMETER_INCLUDE_SUPPLEMENTAL_DATA, true); + parserName = MediaParser.PARSER_NAME_UNKNOWN; + } + + @Override + public void init( + DataReader dataReader, + Uri uri, + Map> responseHeaders, + long position, + long length, + ExtractorOutput output) + throws IOException { + outputConsumerAdapter.setExtractorOutput(output); + inputReaderAdapter.setDataReader(dataReader, length); + inputReaderAdapter.setCurrentPosition(position); + String currentParserName = mediaParser.getParserName(); + if (MediaParser.PARSER_NAME_UNKNOWN.equals(currentParserName)) { + // We need to sniff. + mediaParser.advance(inputReaderAdapter); + parserName = mediaParser.getParserName(); + outputConsumerAdapter.setSelectedParserName(parserName); + } else if (!currentParserName.equals(parserName)) { + // The parser was created by name. + parserName = mediaParser.getParserName(); + outputConsumerAdapter.setSelectedParserName(parserName); + } else { + // The parser implementation has already been selected. Do nothing. + } + } + + @Override + public void release() { + mediaParser.release(); + } + + @Override + public void disableSeekingOnMp3Streams() { + if (MediaParser.PARSER_NAME_MP3.equals(parserName)) { + outputConsumerAdapter.disableSeeking(); + } + } + + @Override + public long getCurrentInputPosition() { + return inputReaderAdapter.getPosition(); + } + + @Override + public void seek(long position, long seekTimeUs) { + inputReaderAdapter.setCurrentPosition(position); + Pair seekPoints = outputConsumerAdapter.getSeekPoints(seekTimeUs); + mediaParser.seek(seekPoints.second.position == position ? seekPoints.second : seekPoints.first); + } + + @Override + public int read(PositionHolder positionHolder) throws IOException { + boolean shouldContinue = mediaParser.advance(inputReaderAdapter); + positionHolder.position = inputReaderAdapter.getAndResetSeekPosition(); + return !shouldContinue + ? Extractor.RESULT_END_OF_INPUT + : positionHolder.position != C.POSITION_UNSET + ? Extractor.RESULT_SEEK + : Extractor.RESULT_CONTINUE; + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaPeriod.java b/library/core/src/main/java/com/google/android/exoplayer2/source/MediaPeriod.java index 39b207e2641..bcbf95a431b 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaPeriod.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/MediaPeriod.java @@ -21,7 +21,7 @@ import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.offline.StreamKey; import com.google.android.exoplayer2.source.MediaSource.MediaSourceCaller; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import java.io.IOException; import java.util.Collections; import java.util.List; @@ -29,21 +29,23 @@ /** * Loads media corresponding to a {@link Timeline.Period}, and allows that media to be read. All - * methods are called on the player's internal playback thread, as described in the - * {@link ExoPlayer} Javadoc. + * methods are called on the player's internal playback thread, as described in the {@link + * ExoPlayer} Javadoc. + * + *

    A {@link MediaPeriod} may only able to provide one {@link SampleStream} corresponding to a + * group at any given time, however this {@link SampleStream} may adapt between multiple tracks + * within the group. */ public interface MediaPeriod extends SequenceableLoader { - /** - * A callback to be notified of {@link MediaPeriod} events. - */ + /** A callback to be notified of {@link MediaPeriod} events. */ interface Callback extends SequenceableLoader.Callback { /** * Called when preparation completes. * *

    Called on the playback thread. After invoking this method, the {@link MediaPeriod} can - * expect for {@link #selectTracks(TrackSelection[], boolean[], SampleStream[], boolean[], + * expect for {@link #selectTracks(ExoTrackSelection[], boolean[], SampleStream[], boolean[], * long)} to be called with the initial track selection. * * @param mediaPeriod The prepared {@link MediaPeriod}. @@ -88,17 +90,17 @@ interface Callback extends SequenceableLoader.Callback { /** * Returns a list of {@link StreamKey StreamKeys} which allow to filter the media in this period - * to load only the parts needed to play the provided {@link TrackSelection TrackSelections}. + * to load only the parts needed to play the provided {@link ExoTrackSelection TrackSelections}. * *

    This method is only called after the period has been prepared. * - * @param trackSelections The {@link TrackSelection TrackSelections} describing the tracks for + * @param trackSelections The {@link ExoTrackSelection TrackSelections} describing the tracks for * which stream keys are requested. * @return The corresponding {@link StreamKey StreamKeys} for the selected tracks, or an empty * list if filtering is not possible and the entire media needs to be loaded to play the * selected tracks. */ - default List getStreamKeys(List trackSelections) { + default List getStreamKeys(List trackSelections) { return Collections.emptyList(); } @@ -113,8 +115,8 @@ default List getStreamKeys(List trackSelections) { * corresponding flag in {@code streamResetFlags} will be set to true. This flag will also be set * if a new sample stream is created. * - *

    Note that previously passed {@link TrackSelection TrackSelections} are no longer valid, and - * any references to them must be updated to point to the new selections. + *

    Note that previously passed {@link ExoTrackSelection TrackSelections} are no longer valid, + * and any references to them must be updated to point to the new selections. * *

    This method is only called after the period has been prepared. * @@ -133,7 +135,7 @@ default List getStreamKeys(List trackSelections) { * @return The actual position at which the tracks were enabled, in microseconds. */ long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSource.java b/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSource.java index 94a9f82030c..f6f2ab496d8 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSource.java @@ -17,7 +17,6 @@ import android.os.Handler; import androidx.annotation.Nullable; -import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.drm.DrmSessionEventListener; @@ -61,155 +60,52 @@ interface MediaSourceCaller { void onSourceInfoRefreshed(MediaSource source, Timeline timeline); } - /** Identifier for a {@link MediaPeriod}. */ - final class MediaPeriodId { - - /** The unique id of the timeline period. */ - public final Object periodUid; - - /** - * If the media period is in an ad group, the index of the ad group in the period. - * {@link C#INDEX_UNSET} otherwise. - */ - public final int adGroupIndex; - - /** - * If the media period is in an ad group, the index of the ad in its ad group in the period. - * {@link C#INDEX_UNSET} otherwise. - */ - public final int adIndexInAdGroup; - - /** - * The sequence number of the window in the buffered sequence of windows this media period is - * part of. {@link C#INDEX_UNSET} if the media period id is not part of a buffered sequence of - * windows. - */ - public final long windowSequenceNumber; - - /** - * The index of the next ad group to which the media period's content is clipped, or {@link - * C#INDEX_UNSET} if there is no following ad group or if this media period is an ad. - */ - public final int nextAdGroupIndex; + // TODO(b/172315872) Delete when all clients have been migrated to base class. + /** + * Identifier for a {@link MediaPeriod}. + * + *

    Extends for backward-compatibility {@link + * com.google.android.exoplayer2.source.MediaPeriodId}. + */ + final class MediaPeriodId extends com.google.android.exoplayer2.source.MediaPeriodId { - /** - * Creates a media period identifier for a period which is not part of a buffered sequence of - * windows. - * - * @param periodUid The unique id of the timeline period. - */ + /** See {@link com.google.android.exoplayer2.source.MediaPeriodId#MediaPeriodId(Object)}. */ public MediaPeriodId(Object periodUid) { - this(periodUid, /* windowSequenceNumber= */ C.INDEX_UNSET); + super(periodUid); } /** - * Creates a media period identifier for the specified period in the timeline. - * - * @param periodUid The unique id of the timeline period. - * @param windowSequenceNumber The sequence number of the window in the buffered sequence of - * windows this media period is part of. + * See {@link com.google.android.exoplayer2.source.MediaPeriodId#MediaPeriodId(Object, long)}. */ public MediaPeriodId(Object periodUid, long windowSequenceNumber) { - this( - periodUid, - /* adGroupIndex= */ C.INDEX_UNSET, - /* adIndexInAdGroup= */ C.INDEX_UNSET, - windowSequenceNumber, - /* nextAdGroupIndex= */ C.INDEX_UNSET); + super(periodUid, windowSequenceNumber); } /** - * Creates a media period identifier for the specified clipped period in the timeline. - * - * @param periodUid The unique id of the timeline period. - * @param windowSequenceNumber The sequence number of the window in the buffered sequence of - * windows this media period is part of. - * @param nextAdGroupIndex The index of the next ad group to which the media period's content is - * clipped. + * See {@link com.google.android.exoplayer2.source.MediaPeriodId#MediaPeriodId(Object, long, + * int)}. */ public MediaPeriodId(Object periodUid, long windowSequenceNumber, int nextAdGroupIndex) { - this( - periodUid, - /* adGroupIndex= */ C.INDEX_UNSET, - /* adIndexInAdGroup= */ C.INDEX_UNSET, - windowSequenceNumber, - nextAdGroupIndex); + super(periodUid, windowSequenceNumber, nextAdGroupIndex); } /** - * Creates a media period identifier that identifies an ad within an ad group at the specified - * timeline period. - * - * @param periodUid The unique id of the timeline period that contains the ad group. - * @param adGroupIndex The index of the ad group. - * @param adIndexInAdGroup The index of the ad in the ad group. - * @param windowSequenceNumber The sequence number of the window in the buffered sequence of - * windows this media period is part of. + * See {@link com.google.android.exoplayer2.source.MediaPeriodId#MediaPeriodId(Object, int, int, + * long)}. */ public MediaPeriodId( Object periodUid, int adGroupIndex, int adIndexInAdGroup, long windowSequenceNumber) { - this( - periodUid, - adGroupIndex, - adIndexInAdGroup, - windowSequenceNumber, - /* nextAdGroupIndex= */ C.INDEX_UNSET); + super(periodUid, adGroupIndex, adIndexInAdGroup, windowSequenceNumber); } - private MediaPeriodId( - Object periodUid, - int adGroupIndex, - int adIndexInAdGroup, - long windowSequenceNumber, - int nextAdGroupIndex) { - this.periodUid = periodUid; - this.adGroupIndex = adGroupIndex; - this.adIndexInAdGroup = adIndexInAdGroup; - this.windowSequenceNumber = windowSequenceNumber; - this.nextAdGroupIndex = nextAdGroupIndex; + /** Wraps an {@link com.google.android.exoplayer2.source.MediaPeriodId} into a MediaPeriodId. */ + public MediaPeriodId(com.google.android.exoplayer2.source.MediaPeriodId mediaPeriodId) { + super(mediaPeriodId); } - /** Returns a copy of this period identifier but with {@code newPeriodUid} as its period uid. */ + /** See {@link com.google.android.exoplayer2.source.MediaPeriodId#copyWithPeriodUid(Object)}. */ public MediaPeriodId copyWithPeriodUid(Object newPeriodUid) { - return periodUid.equals(newPeriodUid) - ? this - : new MediaPeriodId( - newPeriodUid, adGroupIndex, adIndexInAdGroup, windowSequenceNumber, nextAdGroupIndex); - } - - /** - * Returns whether this period identifier identifies an ad in an ad group in a period. - */ - public boolean isAd() { - return adGroupIndex != C.INDEX_UNSET; - } - - @Override - public boolean equals(@Nullable Object obj) { - if (this == obj) { - return true; - } - if (obj == null || getClass() != obj.getClass()) { - return false; - } - - MediaPeriodId periodId = (MediaPeriodId) obj; - return periodUid.equals(periodId.periodUid) - && adGroupIndex == periodId.adGroupIndex - && adIndexInAdGroup == periodId.adIndexInAdGroup - && windowSequenceNumber == periodId.windowSequenceNumber - && nextAdGroupIndex == periodId.nextAdGroupIndex; - } - - @Override - public int hashCode() { - int result = 17; - result = 31 * result + periodUid.hashCode(); - result = 31 * result + adGroupIndex; - result = 31 * result + adIndexInAdGroup; - result = 31 * result + (int) windowSequenceNumber; - result = 31 * result + nextAdGroupIndex; - return result; + return new MediaPeriodId(super.copyWithPeriodUid(newPeriodUid)); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSourceFactory.java b/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSourceFactory.java index 204220e334e..7242c2a2148 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSourceFactory.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/MediaSourceFactory.java @@ -20,7 +20,9 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.drm.DefaultDrmSessionManager; +import com.google.android.exoplayer2.drm.DefaultDrmSessionManagerProvider; import com.google.android.exoplayer2.drm.DrmSessionManager; +import com.google.android.exoplayer2.drm.DrmSessionManagerProvider; import com.google.android.exoplayer2.drm.HttpMediaDrmCallback; import com.google.android.exoplayer2.offline.StreamKey; import com.google.android.exoplayer2.upstream.DefaultHttpDataSourceFactory; @@ -56,41 +58,80 @@ default MediaSourceFactory setStreamKeys(@Nullable List streamKeys) { return this; } + /** + * Sets the {@link DrmSessionManagerProvider} used to obtain a {@link DrmSessionManager} for a + * {@link MediaItem}. + * + *

    If not set, {@link DefaultDrmSessionManagerProvider} is used. + * + *

    If set, calls to the following (deprecated) methods are ignored: + * + *

      + *
    • {@link #setDrmUserAgent(String)} + *
    • {@link #setDrmHttpDataSourceFactory(HttpDataSource.Factory)} + *
    + * + * @return This factory, for convenience. + */ + MediaSourceFactory setDrmSessionManagerProvider( + @Nullable DrmSessionManagerProvider drmSessionManagerProvider); + /** * Sets the {@link DrmSessionManager} to use for all media items regardless of their {@link * MediaItem.DrmConfiguration}. * + *

    Calling this with a non-null {@code drmSessionManager} is equivalent to calling {@code + * setDrmSessionManagerProvider(unusedMediaItem -> drmSessionManager)}. + * * @param drmSessionManager The {@link DrmSessionManager}, or {@code null} to use the {@link * DefaultDrmSessionManager}. * @return This factory, for convenience. + * @deprecated Use {@link #setDrmSessionManagerProvider(DrmSessionManagerProvider)} and pass an + * implementation that always returns the same instance. */ + @Deprecated MediaSourceFactory setDrmSessionManager(@Nullable DrmSessionManager drmSessionManager); /** * Sets the {@link HttpDataSource.Factory} to be used for creating {@link HttpMediaDrmCallback * HttpMediaDrmCallbacks} to execute key and provisioning requests over HTTP. * - *

    In case a {@link DrmSessionManager} has been set by {@link - * #setDrmSessionManager(DrmSessionManager)}, this data source factory is ignored. + *

    Calls to this method are ignored if either a {@link + * #setDrmSessionManagerProvider(DrmSessionManagerProvider) DrmSessionManager provider} or {@link + * #setDrmSessionManager(DrmSessionManager) concrete DrmSessionManager} are provided. * * @param drmHttpDataSourceFactory The HTTP data source factory, or {@code null} to use {@link * DefaultHttpDataSourceFactory}. * @return This factory, for convenience. + * @deprecated Use {@link #setDrmSessionManagerProvider(DrmSessionManagerProvider)} and pass an + * implementation that configures the returned {@link DrmSessionManager} with the desired + * {@link HttpDataSource.Factory}. */ + @Deprecated MediaSourceFactory setDrmHttpDataSourceFactory( @Nullable HttpDataSource.Factory drmHttpDataSourceFactory); /** * Sets the optional user agent to be used for DRM requests. * - *

    In case a factory has been set by {@link - * #setDrmHttpDataSourceFactory(HttpDataSource.Factory)} or a {@link DrmSessionManager} has been - * set by {@link #setDrmSessionManager(DrmSessionManager)}, this user agent is ignored. + *

    Calls to this method are ignored if any of the following are provided: + * + *

      + *
    • A {@link #setDrmSessionManagerProvider(DrmSessionManagerProvider) DrmSessionManager + * provider}. + *
    • A {@link #setDrmSessionManager(DrmSessionManager) concrete DrmSessionManager}. + *
    • A {@link #setDrmHttpDataSourceFactory(HttpDataSource.Factory) DRM + * HttpDataSource.Factory}. + *
    * * @param userAgent The user agent to be used for DRM requests, or {@code null} to use the * default. * @return This factory, for convenience. + * @deprecated Use {@link #setDrmSessionManagerProvider(DrmSessionManagerProvider)} and pass an + * implementation that configures the returned {@link DrmSessionManager} with the desired + * {@code userAgent}. */ + @Deprecated MediaSourceFactory setDrmUserAgent(@Nullable String userAgent); /** diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MergingMediaPeriod.java b/library/core/src/main/java/com/google/android/exoplayer2/source/MergingMediaPeriod.java index 0dae1ad6f94..860d9a3b953 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/MergingMediaPeriod.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/MergingMediaPeriod.java @@ -23,7 +23,7 @@ import com.google.android.exoplayer2.SeekParameters; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; import com.google.android.exoplayer2.offline.StreamKey; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.util.Assertions; import java.io.IOException; import java.util.ArrayList; @@ -33,9 +33,7 @@ import org.checkerframework.checker.nullness.compatqual.NullableType; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; -/** - * Merges multiple {@link MediaPeriod}s. - */ +/** Merges multiple {@link MediaPeriod}s. */ /* package */ final class MergingMediaPeriod implements MediaPeriod, MediaPeriod.Callback { private final MediaPeriod[] periods; @@ -100,7 +98,7 @@ public TrackGroupArray getTrackGroups() { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, @@ -126,15 +124,16 @@ public long selectTracks( // Select tracks for each child, copying the resulting streams back into a new streams array. @NullableType SampleStream[] newStreams = new SampleStream[selections.length]; @NullableType SampleStream[] childStreams = new SampleStream[selections.length]; - @NullableType TrackSelection[] childSelections = new TrackSelection[selections.length]; + @NullableType ExoTrackSelection[] childSelections = new ExoTrackSelection[selections.length]; ArrayList enabledPeriodsList = new ArrayList<>(periods.length); for (int i = 0; i < periods.length; i++) { for (int j = 0; j < selections.length; j++) { childStreams[j] = streamChildIndices[j] == i ? streams[j] : null; childSelections[j] = selectionChildIndices[j] == i ? selections[j] : null; } - long selectPositionUs = periods[i].selectTracks(childSelections, mayRetainStreamFlags, - childStreams, streamResetFlags, positionUs); + long selectPositionUs = + periods[i].selectTracks( + childSelections, mayRetainStreamFlags, childStreams, streamResetFlags, positionUs); if (i == 0) { positionUs = selectPositionUs; } else if (selectPositionUs != positionUs) { @@ -314,13 +313,13 @@ public TrackGroupArray getTrackGroups() { } @Override - public List getStreamKeys(List trackSelections) { + public List getStreamKeys(List trackSelections) { return mediaPeriod.getStreamKeys(trackSelections); } @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/MergingMediaSource.java b/library/core/src/main/java/com/google/android/exoplayer2/source/MergingMediaSource.java index 8df7a639c6f..3a9bfe4f918 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/MergingMediaSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/MergingMediaSource.java @@ -15,12 +15,18 @@ */ package com.google.android.exoplayer2.source; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static java.lang.Math.min; + import androidx.annotation.IntDef; import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.TransferListener; +import com.google.common.collect.Multimap; +import com.google.common.collect.MultimapBuilder; import java.io.IOException; import java.lang.annotation.Documented; import java.lang.annotation.Retention; @@ -28,6 +34,8 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; +import java.util.HashMap; +import java.util.Map; /** * Merges multiple {@link MediaSource}s. @@ -70,21 +78,26 @@ public IllegalMergeException(@Reason int reason) { new MediaItem.Builder().setMediaId("MergingMediaSource").build(); private final boolean adjustPeriodTimeOffsets; + private final boolean clipDurations; private final MediaSource[] mediaSources; private final Timeline[] timelines; private final ArrayList pendingTimelineSources; private final CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory; + private final Map clippedDurationsUs; + private final Multimap clippedMediaPeriods; private int periodCount; private long[][] periodTimeOffsetsUs; + @Nullable private IllegalMergeException mergeError; /** * Creates a merging media source. * - *

    Offsets between the timestamps in the media sources will not be adjusted. + *

    Neither offsets between the timestamps in the media sources nor the durations of the media + * sources will be adjusted. * - * @param mediaSources The {@link MediaSource}s to merge. + * @param mediaSources The {@link MediaSource MediaSources} to merge. */ public MergingMediaSource(MediaSource... mediaSources) { this(/* adjustPeriodTimeOffsets= */ false, mediaSources); @@ -93,12 +106,14 @@ public MergingMediaSource(MediaSource... mediaSources) { /** * Creates a merging media source. * + *

    Durations of the media sources will not be adjusted. + * * @param adjustPeriodTimeOffsets Whether to adjust timestamps of the merged media sources to all * start at the same time. - * @param mediaSources The {@link MediaSource}s to merge. + * @param mediaSources The {@link MediaSource MediaSources} to merge. */ public MergingMediaSource(boolean adjustPeriodTimeOffsets, MediaSource... mediaSources) { - this(adjustPeriodTimeOffsets, new DefaultCompositeSequenceableLoaderFactory(), mediaSources); + this(adjustPeriodTimeOffsets, /* clipDurations= */ false, mediaSources); } /** @@ -106,22 +121,46 @@ public MergingMediaSource(boolean adjustPeriodTimeOffsets, MediaSource... mediaS * * @param adjustPeriodTimeOffsets Whether to adjust timestamps of the merged media sources to all * start at the same time. + * @param clipDurations Whether to clip the durations of the media sources to match the shortest + * duration. + * @param mediaSources The {@link MediaSource MediaSources} to merge. + */ + public MergingMediaSource( + boolean adjustPeriodTimeOffsets, boolean clipDurations, MediaSource... mediaSources) { + this( + adjustPeriodTimeOffsets, + clipDurations, + new DefaultCompositeSequenceableLoaderFactory(), + mediaSources); + } + + /** + * Creates a merging media source. + * + * @param adjustPeriodTimeOffsets Whether to adjust timestamps of the merged media sources to all + * start at the same time. + * @param clipDurations Whether to clip the durations of the media sources to match the shortest + * duration. * @param compositeSequenceableLoaderFactory A factory to create composite {@link * SequenceableLoader}s for when this media source loads data from multiple streams (video, * audio etc...). - * @param mediaSources The {@link MediaSource}s to merge. + * @param mediaSources The {@link MediaSource MediaSources} to merge. */ public MergingMediaSource( boolean adjustPeriodTimeOffsets, + boolean clipDurations, CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory, MediaSource... mediaSources) { this.adjustPeriodTimeOffsets = adjustPeriodTimeOffsets; + this.clipDurations = clipDurations; this.mediaSources = mediaSources; this.compositeSequenceableLoaderFactory = compositeSequenceableLoaderFactory; pendingTimelineSources = new ArrayList<>(Arrays.asList(mediaSources)); periodCount = PERIOD_COUNT_UNSET; timelines = new Timeline[mediaSources.length]; periodTimeOffsetsUs = new long[0][]; + clippedDurationsUs = new HashMap<>(); + clippedMediaPeriods = MultimapBuilder.hashKeys().arrayListValues().build(); } /** @@ -167,12 +206,33 @@ public MediaPeriod createPeriod(MediaPeriodId id, Allocator allocator, long star mediaSources[i].createPeriod( childMediaPeriodId, allocator, startPositionUs - periodTimeOffsetsUs[periodIndex][i]); } - return new MergingMediaPeriod( - compositeSequenceableLoaderFactory, periodTimeOffsetsUs[periodIndex], periods); + MediaPeriod mediaPeriod = + new MergingMediaPeriod( + compositeSequenceableLoaderFactory, periodTimeOffsetsUs[periodIndex], periods); + if (clipDurations) { + mediaPeriod = + new ClippingMediaPeriod( + mediaPeriod, + /* enableInitialDiscontinuity= */ true, + /* startUs= */ 0, + /* endUs= */ checkNotNull(clippedDurationsUs.get(id.periodUid))); + clippedMediaPeriods.put(id.periodUid, (ClippingMediaPeriod) mediaPeriod); + } + return mediaPeriod; } @Override public void releasePeriod(MediaPeriod mediaPeriod) { + if (clipDurations) { + ClippingMediaPeriod clippingMediaPeriod = (ClippingMediaPeriod) mediaPeriod; + for (Map.Entry entry : clippedMediaPeriods.entries()) { + if (entry.getValue().equals(clippingMediaPeriod)) { + clippedMediaPeriods.remove(entry.getKey(), entry.getValue()); + break; + } + } + mediaPeriod = clippingMediaPeriod.mediaPeriod; + } MergingMediaPeriod mergingPeriod = (MergingMediaPeriod) mediaPeriod; for (int i = 0; i < mediaSources.length; i++) { mediaSources[i].releasePeriod(mergingPeriod.getChildPeriod(i)); @@ -210,7 +270,12 @@ protected void onChildSourceInfoRefreshed( if (adjustPeriodTimeOffsets) { computePeriodTimeOffsets(); } - refreshSourceInfo(timelines[0]); + Timeline mergedTimeline = timelines[0]; + if (clipDurations) { + updateClippedDuration(); + mergedTimeline = new ClippedTimeline(mergedTimeline, clippedDurationsUs); + } + refreshSourceInfo(mergedTimeline); } } @@ -234,4 +299,72 @@ private void computePeriodTimeOffsets() { } } } + + private void updateClippedDuration() { + Timeline.Period period = new Timeline.Period(); + for (int periodIndex = 0; periodIndex < periodCount; periodIndex++) { + long minDurationUs = C.TIME_END_OF_SOURCE; + for (int timelineIndex = 0; timelineIndex < timelines.length; timelineIndex++) { + long durationUs = timelines[timelineIndex].getPeriod(periodIndex, period).getDurationUs(); + if (durationUs == C.TIME_UNSET) { + continue; + } + long adjustedDurationUs = durationUs + periodTimeOffsetsUs[periodIndex][timelineIndex]; + if (minDurationUs == C.TIME_END_OF_SOURCE || adjustedDurationUs < minDurationUs) { + minDurationUs = adjustedDurationUs; + } + } + Object periodUid = timelines[0].getUidOfPeriod(periodIndex); + clippedDurationsUs.put(periodUid, minDurationUs); + for (ClippingMediaPeriod clippingMediaPeriod : clippedMediaPeriods.get(periodUid)) { + clippingMediaPeriod.updateClipping(/* startUs= */ 0, /* endUs= */ minDurationUs); + } + } + } + + private static final class ClippedTimeline extends ForwardingTimeline { + + private final long[] periodDurationsUs; + private final long[] windowDurationsUs; + + public ClippedTimeline(Timeline timeline, Map clippedDurationsUs) { + super(timeline); + int windowCount = timeline.getWindowCount(); + windowDurationsUs = new long[timeline.getWindowCount()]; + Window window = new Window(); + for (int i = 0; i < windowCount; i++) { + windowDurationsUs[i] = timeline.getWindow(i, window).durationUs; + } + int periodCount = timeline.getPeriodCount(); + periodDurationsUs = new long[periodCount]; + Period period = new Period(); + for (int i = 0; i < periodCount; i++) { + timeline.getPeriod(i, period, /* setIds= */ true); + long clippedDurationUs = checkNotNull(clippedDurationsUs.get(period.uid)); + periodDurationsUs[i] = + clippedDurationUs != C.TIME_END_OF_SOURCE ? clippedDurationUs : period.durationUs; + if (period.durationUs != C.TIME_UNSET) { + windowDurationsUs[period.windowIndex] -= period.durationUs - periodDurationsUs[i]; + } + } + } + + @Override + public Window getWindow(int windowIndex, Window window, long defaultPositionProjectionUs) { + super.getWindow(windowIndex, window, defaultPositionProjectionUs); + window.durationUs = windowDurationsUs[windowIndex]; + window.defaultPositionUs = + window.durationUs == C.TIME_UNSET || window.defaultPositionUs == C.TIME_UNSET + ? window.defaultPositionUs + : min(window.defaultPositionUs, window.durationUs); + return window; + } + + @Override + public Period getPeriod(int periodIndex, Period period, boolean setIds) { + super.getPeriod(periodIndex, period, setIds); + period.durationUs = periodDurationsUs[periodIndex]; + return period; + } + } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/ProgressiveMediaPeriod.java b/library/core/src/main/java/com/google/android/exoplayer2/source/ProgressiveMediaPeriod.java index a5c8ff631d6..f7b88fcab8e 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/ProgressiveMediaPeriod.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/ProgressiveMediaPeriod.java @@ -40,7 +40,7 @@ import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.metadata.icy.IcyHeaders; import com.google.android.exoplayer2.source.SampleQueue.UpstreamFormatChangedListener; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DataSpec; @@ -188,9 +188,7 @@ public ProgressiveMediaPeriod( this.customCacheKey = customCacheKey; this.continueLoadingCheckIntervalBytes = continueLoadingCheckIntervalBytes; loader = new Loader("Loader:ProgressiveMediaPeriod"); - ProgressiveMediaExtractor progressiveMediaExtractor = - new BundledExtractorsAdapter(extractorsFactory); - this.progressiveMediaExtractor = progressiveMediaExtractor; + this.progressiveMediaExtractor = new BundledExtractorsAdapter(extractorsFactory); loadCondition = new ConditionVariable(); maybeFinishPrepareRunnable = this::maybeFinishPrepare; onContinueLoadingRequestedRunnable = @@ -254,7 +252,7 @@ public TrackGroupArray getTrackGroups() { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, @@ -279,7 +277,7 @@ public long selectTracks( // Select new tracks. for (int i = 0; i < selections.length; i++) { if (streams[i] == null && selections[i] != null) { - TrackSelection selection = selections[i]; + ExoTrackSelection selection = selections[i]; Assertions.checkState(selection.length() == 1); Assertions.checkState(selection.getIndexInTrackGroup(0) == 0); int track = tracks.indexOf(selection.getTrackGroup()); @@ -717,7 +715,7 @@ private TrackOutput prepareTrackOutput(TrackId id) { } } SampleQueue trackOutput = - new SampleQueue( + SampleQueue.createWithDrm( allocator, /* playbackLooper= */ handler.getLooper(), drmSessionManager, diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/ProgressiveMediaSource.java b/library/core/src/main/java/com/google/android/exoplayer2/source/ProgressiveMediaSource.java index 19f09fde22b..fe249df6ffd 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/ProgressiveMediaSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/ProgressiveMediaSource.java @@ -22,7 +22,9 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.drm.DefaultDrmSessionManagerProvider; import com.google.android.exoplayer2.drm.DrmSessionManager; +import com.google.android.exoplayer2.drm.DrmSessionManagerProvider; import com.google.android.exoplayer2.extractor.DefaultExtractorsFactory; import com.google.android.exoplayer2.extractor.Extractor; import com.google.android.exoplayer2.extractor.ExtractorsFactory; @@ -51,10 +53,10 @@ public final class ProgressiveMediaSource extends BaseMediaSource public static final class Factory implements MediaSourceFactory { private final DataSource.Factory dataSourceFactory; - private final MediaSourceDrmHelper mediaSourceDrmHelper; private ExtractorsFactory extractorsFactory; - @Nullable private DrmSessionManager drmSessionManager; + private boolean usingCustomDrmSessionManagerProvider; + private DrmSessionManagerProvider drmSessionManagerProvider; private LoadErrorHandlingPolicy loadErrorHandlingPolicy; private int continueLoadingCheckIntervalBytes; @Nullable private String customCacheKey; @@ -79,7 +81,7 @@ public Factory(DataSource.Factory dataSourceFactory) { public Factory(DataSource.Factory dataSourceFactory, ExtractorsFactory extractorsFactory) { this.dataSourceFactory = dataSourceFactory; this.extractorsFactory = extractorsFactory; - mediaSourceDrmHelper = new MediaSourceDrmHelper(); + drmSessionManagerProvider = new DefaultDrmSessionManagerProvider(); loadErrorHandlingPolicy = new DefaultLoadErrorHandlingPolicy(); continueLoadingCheckIntervalBytes = DEFAULT_LOADING_CHECK_INTERVAL_BYTES; } @@ -148,21 +150,42 @@ public Factory setContinueLoadingCheckIntervalBytes(int continueLoadingCheckInte } @Override + public Factory setDrmSessionManagerProvider( + @Nullable DrmSessionManagerProvider drmSessionManagerProvider) { + if (drmSessionManagerProvider != null) { + this.drmSessionManagerProvider = drmSessionManagerProvider; + this.usingCustomDrmSessionManagerProvider = true; + } else { + this.drmSessionManagerProvider = new DefaultDrmSessionManagerProvider(); + this.usingCustomDrmSessionManagerProvider = false; + } + return this; + } + public Factory setDrmSessionManager(@Nullable DrmSessionManager drmSessionManager) { - this.drmSessionManager = drmSessionManager; + if (drmSessionManager == null) { + setDrmSessionManagerProvider(null); + } else { + setDrmSessionManagerProvider(unusedMediaItem -> drmSessionManager); + } return this; } @Override public Factory setDrmHttpDataSourceFactory( @Nullable HttpDataSource.Factory drmHttpDataSourceFactory) { - mediaSourceDrmHelper.setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + if (!usingCustomDrmSessionManagerProvider) { + ((DefaultDrmSessionManagerProvider) drmSessionManagerProvider) + .setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + } return this; } @Override public Factory setDrmUserAgent(@Nullable String userAgent) { - mediaSourceDrmHelper.setDrmUserAgent(userAgent); + if (!usingCustomDrmSessionManagerProvider) { + ((DefaultDrmSessionManagerProvider) drmSessionManagerProvider).setDrmUserAgent(userAgent); + } return this; } @@ -198,7 +221,7 @@ public ProgressiveMediaSource createMediaSource(MediaItem mediaItem) { mediaItem, dataSourceFactory, extractorsFactory, - drmSessionManager != null ? drmSessionManager : mediaSourceDrmHelper.create(mediaItem), + drmSessionManagerProvider.get(mediaItem), loadErrorHandlingPolicy, continueLoadingCheckIntervalBytes); } @@ -336,7 +359,7 @@ private void notifySourceInfoRefreshed() { timelineDurationUs, timelineIsSeekable, /* isDynamic= */ false, - /* isLive= */ timelineIsLive, + /* useLiveConfiguration= */ timelineIsLive, /* manifest= */ null, mediaItem); if (timelineIsPlaceholder) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/SampleDataQueue.java b/library/core/src/main/java/com/google/android/exoplayer2/source/SampleDataQueue.java index 797b5ad30bb..5bc1482e68a 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/SampleDataQueue.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/SampleDataQueue.java @@ -115,39 +115,25 @@ public void rewind() { } /** - * Reads data from the rolling buffer to populate a decoder input buffer. + * Reads data from the rolling buffer to populate a decoder input buffer, and advances the read + * position. * * @param buffer The buffer to populate. * @param extrasHolder The extras holder whose offset should be read and subsequently adjusted. */ public void readToBuffer(DecoderInputBuffer buffer, SampleExtrasHolder extrasHolder) { - // Read encryption data if the sample is encrypted. - if (buffer.isEncrypted()) { - readEncryptionData(buffer, extrasHolder); - } - // Read sample data, extracting supplemental data into a separate buffer if needed. - if (buffer.hasSupplementalData()) { - // If there is supplemental data, the sample data is prefixed by its size. - scratch.reset(4); - readData(extrasHolder.offset, scratch.getData(), 4); - int sampleSize = scratch.readUnsignedIntToInt(); - extrasHolder.offset += 4; - extrasHolder.size -= 4; - - // Write the sample data. - buffer.ensureSpaceForWrite(sampleSize); - readData(extrasHolder.offset, buffer.data, sampleSize); - extrasHolder.offset += sampleSize; - extrasHolder.size -= sampleSize; + readAllocationNode = readSampleData(readAllocationNode, buffer, extrasHolder, scratch); + } - // Write the remaining data as supplemental data. - buffer.resetSupplementalData(extrasHolder.size); - readData(extrasHolder.offset, buffer.supplementalData, extrasHolder.size); - } else { - // Write the sample data. - buffer.ensureSpaceForWrite(extrasHolder.size); - readData(extrasHolder.offset, buffer.data, extrasHolder.size); - } + /** + * Peeks data from the rolling buffer to populate a decoder input buffer, without advancing the + * read position. + * + * @param buffer The buffer to populate. + * @param extrasHolder The extras holder whose offset should be read and subsequently adjusted. + */ + public void peekToBuffer(DecoderInputBuffer buffer, SampleExtrasHolder extrasHolder) { + readSampleData(readAllocationNode, buffer, extrasHolder, scratch); } /** @@ -211,21 +197,128 @@ public void sampleData(ParsableByteArray buffer, int length) { // Private methods. /** - * Reads encryption data for the current sample. + * Clears allocation nodes starting from {@code fromNode}. + * + * @param fromNode The node from which to clear. + */ + private void clearAllocationNodes(AllocationNode fromNode) { + if (!fromNode.wasInitialized) { + return; + } + // Bulk release allocations for performance (it's significantly faster when using + // DefaultAllocator because the allocator's lock only needs to be acquired and released once) + // [Internal: See b/29542039]. + int allocationCount = + (writeAllocationNode.wasInitialized ? 1 : 0) + + ((int) (writeAllocationNode.startPosition - fromNode.startPosition) + / allocationLength); + Allocation[] allocationsToRelease = new Allocation[allocationCount]; + AllocationNode currentNode = fromNode; + for (int i = 0; i < allocationsToRelease.length; i++) { + allocationsToRelease[i] = currentNode.allocation; + currentNode = currentNode.clear(); + } + allocator.release(allocationsToRelease); + } + + /** + * Called before writing sample data to {@link #writeAllocationNode}. May cause {@link + * #writeAllocationNode} to be initialized. + * + * @param length The number of bytes that the caller wishes to write. + * @return The number of bytes that the caller is permitted to write, which may be less than + * {@code length}. + */ + private int preAppend(int length) { + if (!writeAllocationNode.wasInitialized) { + writeAllocationNode.initialize( + allocator.allocate(), + new AllocationNode(writeAllocationNode.endPosition, allocationLength)); + } + return min(length, (int) (writeAllocationNode.endPosition - totalBytesWritten)); + } + + /** + * Called after writing sample data. May cause {@link #writeAllocationNode} to be advanced. + * + * @param length The number of bytes that were written. + */ + private void postAppend(int length) { + totalBytesWritten += length; + if (totalBytesWritten == writeAllocationNode.endPosition) { + writeAllocationNode = writeAllocationNode.next; + } + } + + /** + * Reads data from the rolling buffer to populate a decoder input buffer. + * + * @param allocationNode The first {@link AllocationNode} containing data yet to be read. + * @param buffer The buffer to populate. + * @param extrasHolder The extras holder whose offset should be read and subsequently adjusted. + * @param scratch A scratch {@link ParsableByteArray}. + * @return The first {@link AllocationNode} that contains unread bytes after the last byte that + * the invocation read. + */ + private static AllocationNode readSampleData( + AllocationNode allocationNode, + DecoderInputBuffer buffer, + SampleExtrasHolder extrasHolder, + ParsableByteArray scratch) { + if (buffer.isEncrypted()) { + allocationNode = readEncryptionData(allocationNode, buffer, extrasHolder, scratch); + } + // Read sample data, extracting supplemental data into a separate buffer if needed. + if (buffer.hasSupplementalData()) { + // If there is supplemental data, the sample data is prefixed by its size. + scratch.reset(4); + allocationNode = readData(allocationNode, extrasHolder.offset, scratch.getData(), 4); + int sampleSize = scratch.readUnsignedIntToInt(); + extrasHolder.offset += 4; + extrasHolder.size -= 4; + + // Write the sample data. + buffer.ensureSpaceForWrite(sampleSize); + allocationNode = readData(allocationNode, extrasHolder.offset, buffer.data, sampleSize); + extrasHolder.offset += sampleSize; + extrasHolder.size -= sampleSize; + + // Write the remaining data as supplemental data. + buffer.resetSupplementalData(extrasHolder.size); + allocationNode = + readData(allocationNode, extrasHolder.offset, buffer.supplementalData, extrasHolder.size); + } else { + // Write the sample data. + buffer.ensureSpaceForWrite(extrasHolder.size); + allocationNode = + readData(allocationNode, extrasHolder.offset, buffer.data, extrasHolder.size); + } + return allocationNode; + } + + /** + * Reads encryption data for the sample described by {@code extrasHolder}. * *

    The encryption data is written into {@link DecoderInputBuffer#cryptoInfo}, and {@link * SampleExtrasHolder#size} is adjusted to subtract the number of bytes that were read. The same * value is added to {@link SampleExtrasHolder#offset}. * + * @param allocationNode The first {@link AllocationNode} containing data yet to be read. * @param buffer The buffer into which the encryption data should be written. * @param extrasHolder The extras holder whose offset should be read and subsequently adjusted. + * @param scratch A scratch {@link ParsableByteArray}. + * @return The first {@link AllocationNode} that contains unread bytes after this method returns. */ - private void readEncryptionData(DecoderInputBuffer buffer, SampleExtrasHolder extrasHolder) { + private static AllocationNode readEncryptionData( + AllocationNode allocationNode, + DecoderInputBuffer buffer, + SampleExtrasHolder extrasHolder, + ParsableByteArray scratch) { long offset = extrasHolder.offset; // Read the signal byte. scratch.reset(1); - readData(offset, scratch.getData(), 1); + allocationNode = readData(allocationNode, offset, scratch.getData(), 1); offset++; byte signalByte = scratch.getData()[0]; boolean subsampleEncryption = (signalByte & 0x80) != 0; @@ -239,14 +332,14 @@ private void readEncryptionData(DecoderInputBuffer buffer, SampleExtrasHolder ex // Zero out cryptoInfo.iv so that if ivSize < 16, the remaining bytes are correctly set to 0. Arrays.fill(cryptoInfo.iv, (byte) 0); } - readData(offset, cryptoInfo.iv, ivSize); + allocationNode = readData(allocationNode, offset, cryptoInfo.iv, ivSize); offset += ivSize; // Read the subsample count, if present. int subsampleCount; if (subsampleEncryption) { scratch.reset(2); - readData(offset, scratch.getData(), 2); + allocationNode = readData(allocationNode, offset, scratch.getData(), 2); offset += 2; subsampleCount = scratch.readUnsignedShort(); } else { @@ -265,7 +358,7 @@ private void readEncryptionData(DecoderInputBuffer buffer, SampleExtrasHolder ex if (subsampleEncryption) { int subsampleDataLength = 6 * subsampleCount; scratch.reset(subsampleDataLength); - readData(offset, scratch.getData(), subsampleDataLength); + allocationNode = readData(allocationNode, offset, scratch.getData(), subsampleDataLength); offset += subsampleDataLength; scratch.setPosition(0); for (int i = 0; i < subsampleCount; i++) { @@ -293,120 +386,76 @@ private void readEncryptionData(DecoderInputBuffer buffer, SampleExtrasHolder ex int bytesRead = (int) (offset - extrasHolder.offset); extrasHolder.offset += bytesRead; extrasHolder.size -= bytesRead; + return allocationNode; } /** - * Reads data from the front of the rolling buffer. + * Reads data from {@code allocationNode} and its following nodes. * + * @param allocationNode The first {@link AllocationNode} containing data yet to be read. * @param absolutePosition The absolute position from which data should be read. * @param target The buffer into which data should be written. * @param length The number of bytes to read. + * @return The first {@link AllocationNode} that contains unread bytes after this method returns. */ - private void readData(long absolutePosition, ByteBuffer target, int length) { - advanceReadTo(absolutePosition); + private static AllocationNode readData( + AllocationNode allocationNode, long absolutePosition, ByteBuffer target, int length) { + allocationNode = getNodeContainingPosition(allocationNode, absolutePosition); int remaining = length; while (remaining > 0) { - int toCopy = min(remaining, (int) (readAllocationNode.endPosition - absolutePosition)); - Allocation allocation = readAllocationNode.allocation; - target.put(allocation.data, readAllocationNode.translateOffset(absolutePosition), toCopy); + int toCopy = min(remaining, (int) (allocationNode.endPosition - absolutePosition)); + Allocation allocation = allocationNode.allocation; + target.put(allocation.data, allocationNode.translateOffset(absolutePosition), toCopy); remaining -= toCopy; absolutePosition += toCopy; - if (absolutePosition == readAllocationNode.endPosition) { - readAllocationNode = readAllocationNode.next; + if (absolutePosition == allocationNode.endPosition) { + allocationNode = allocationNode.next; } } + return allocationNode; } /** - * Reads data from the front of the rolling buffer. + * Reads data from {@code allocationNode} and its following nodes. * + * @param allocationNode The first {@link AllocationNode} containing data yet to be read. * @param absolutePosition The absolute position from which data should be read. * @param target The array into which data should be written. * @param length The number of bytes to read. + * @return The first {@link AllocationNode} that contains unread bytes after this method returns. */ - private void readData(long absolutePosition, byte[] target, int length) { - advanceReadTo(absolutePosition); + private static AllocationNode readData( + AllocationNode allocationNode, long absolutePosition, byte[] target, int length) { + allocationNode = getNodeContainingPosition(allocationNode, absolutePosition); int remaining = length; while (remaining > 0) { - int toCopy = min(remaining, (int) (readAllocationNode.endPosition - absolutePosition)); - Allocation allocation = readAllocationNode.allocation; + int toCopy = min(remaining, (int) (allocationNode.endPosition - absolutePosition)); + Allocation allocation = allocationNode.allocation; System.arraycopy( allocation.data, - readAllocationNode.translateOffset(absolutePosition), + allocationNode.translateOffset(absolutePosition), target, length - remaining, toCopy); remaining -= toCopy; absolutePosition += toCopy; - if (absolutePosition == readAllocationNode.endPosition) { - readAllocationNode = readAllocationNode.next; + if (absolutePosition == allocationNode.endPosition) { + allocationNode = allocationNode.next; } } + return allocationNode; } /** - * Advances the read position to the specified absolute position. - * - * @param absolutePosition The position to which {@link #readAllocationNode} should be advanced. + * Returns the {@link AllocationNode} in {@code allocationNode}'s chain which contains the given + * {@code absolutePosition}. */ - private void advanceReadTo(long absolutePosition) { - while (absolutePosition >= readAllocationNode.endPosition) { - readAllocationNode = readAllocationNode.next; - } - } - - /** - * Clears allocation nodes starting from {@code fromNode}. - * - * @param fromNode The node from which to clear. - */ - private void clearAllocationNodes(AllocationNode fromNode) { - if (!fromNode.wasInitialized) { - return; - } - // Bulk release allocations for performance (it's significantly faster when using - // DefaultAllocator because the allocator's lock only needs to be acquired and released once) - // [Internal: See b/29542039]. - int allocationCount = - (writeAllocationNode.wasInitialized ? 1 : 0) - + ((int) (writeAllocationNode.startPosition - fromNode.startPosition) - / allocationLength); - Allocation[] allocationsToRelease = new Allocation[allocationCount]; - AllocationNode currentNode = fromNode; - for (int i = 0; i < allocationsToRelease.length; i++) { - allocationsToRelease[i] = currentNode.allocation; - currentNode = currentNode.clear(); - } - allocator.release(allocationsToRelease); - } - - /** - * Called before writing sample data to {@link #writeAllocationNode}. May cause {@link - * #writeAllocationNode} to be initialized. - * - * @param length The number of bytes that the caller wishes to write. - * @return The number of bytes that the caller is permitted to write, which may be less than - * {@code length}. - */ - private int preAppend(int length) { - if (!writeAllocationNode.wasInitialized) { - writeAllocationNode.initialize( - allocator.allocate(), - new AllocationNode(writeAllocationNode.endPosition, allocationLength)); - } - return min(length, (int) (writeAllocationNode.endPosition - totalBytesWritten)); - } - - /** - * Called after writing sample data. May cause {@link #writeAllocationNode} to be advanced. - * - * @param length The number of bytes that were written. - */ - private void postAppend(int length) { - totalBytesWritten += length; - if (totalBytesWritten == writeAllocationNode.endPosition) { - writeAllocationNode = writeAllocationNode.next; + private static AllocationNode getNodeContainingPosition( + AllocationNode allocationNode, long absolutePosition) { + while (absolutePosition >= allocationNode.endPosition) { + allocationNode = allocationNode.next; } + return allocationNode; } /** A node in a linked list of {@link Allocation}s held by the output. */ diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/SampleQueue.java b/library/core/src/main/java/com/google/android/exoplayer2/source/SampleQueue.java index 824b8dedaf5..77e17c84b1c 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/SampleQueue.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/SampleQueue.java @@ -60,9 +60,9 @@ public interface UpstreamFormatChangedListener { private final SampleDataQueue sampleDataQueue; private final SampleExtrasHolder extrasHolder; - private final Looper playbackLooper; - private final DrmSessionManager drmSessionManager; - private final DrmSessionEventListener.EventDispatcher drmEventDispatcher; + @Nullable private final DrmSessionManager drmSessionManager; + @Nullable private final DrmSessionEventListener.EventDispatcher drmEventDispatcher; + @Nullable private final Looper playbackLooper; @Nullable private UpstreamFormatChangedListener upstreamFormatChangeListener; @Nullable private Format downstreamFormat; @@ -100,7 +100,23 @@ public interface UpstreamFormatChangedListener { private boolean pendingSplice; /** - * Creates a sample queue. + * Creates a sample queue without DRM resource management. + * + * @param allocator An {@link Allocator} from which allocations for sample data can be obtained. + */ + public static SampleQueue createWithoutDrm(Allocator allocator) { + return new SampleQueue( + allocator, + /* playbackLooper= */ null, + /* drmSessionManager= */ null, + /* drmEventDispatcher= */ null); + } + + /** + * Creates a sample queue with DRM resource management. + * + *

    For each sample added to the queue, a {@link DrmSession} will be attached containing the + * keys needed to decrypt it. * * @param allocator An {@link Allocator} from which allocations for sample data can be obtained. * @param playbackLooper The looper associated with the media playback thread. @@ -109,11 +125,23 @@ public interface UpstreamFormatChangedListener { * @param drmEventDispatcher A {@link DrmSessionEventListener.EventDispatcher} to notify of events * related to this SampleQueue. */ - public SampleQueue( + public static SampleQueue createWithDrm( Allocator allocator, Looper playbackLooper, DrmSessionManager drmSessionManager, DrmSessionEventListener.EventDispatcher drmEventDispatcher) { + return new SampleQueue( + allocator, + Assertions.checkNotNull(playbackLooper), + Assertions.checkNotNull(drmSessionManager), + Assertions.checkNotNull(drmEventDispatcher)); + } + + protected SampleQueue( + Allocator allocator, + @Nullable Looper playbackLooper, + @Nullable DrmSessionManager drmSessionManager, + @Nullable DrmSessionEventListener.EventDispatcher drmEventDispatcher) { this.playbackLooper = playbackLooper; this.drmSessionManager = drmSessionManager; this.drmEventDispatcher = drmEventDispatcher; @@ -349,6 +377,20 @@ public synchronized boolean isReady(boolean loadingFinished) { return mayReadSample(relativeReadIndex); } + /** Equivalent to {@link #read}, except it never advances the read position. */ + public final int peek( + FormatHolder formatHolder, + DecoderInputBuffer buffer, + boolean formatRequired, + boolean loadingFinished) { + int result = + peekSampleMetadata(formatHolder, buffer, formatRequired, loadingFinished, extrasHolder); + if (result == C.RESULT_BUFFER_READ && !buffer.isEndOfStream() && !buffer.isFlagsOnly()) { + sampleDataQueue.peekToBuffer(buffer, extrasHolder); + } + return result; + } + /** * Attempts to read from the queue. * @@ -375,9 +417,10 @@ public int read( boolean formatRequired, boolean loadingFinished) { int result = - readSampleMetadata(formatHolder, buffer, formatRequired, loadingFinished, extrasHolder); + peekSampleMetadata(formatHolder, buffer, formatRequired, loadingFinished, extrasHolder); if (result == C.RESULT_BUFFER_READ && !buffer.isEndOfStream() && !buffer.isFlagsOnly()) { sampleDataQueue.readToBuffer(buffer, extrasHolder); + readPosition++; } return result; } @@ -622,7 +665,7 @@ private synchronized void rewind() { } @SuppressWarnings("ReferenceEquality") // See comments in setUpstreamFormat - private synchronized int readSampleMetadata( + private synchronized int peekSampleMetadata( FormatHolder formatHolder, DecoderInputBuffer buffer, boolean formatRequired, @@ -657,14 +700,10 @@ private synchronized int readSampleMetadata( if (buffer.timeUs < startTimeUs) { buffer.addFlag(C.BUFFER_FLAG_DECODE_ONLY); } - if (buffer.isFlagsOnly()) { - return C.RESULT_BUFFER_READ; - } extrasHolder.size = sizes[relativeReadIndex]; extrasHolder.offset = offsets[relativeReadIndex]; extrasHolder.cryptoData = cryptoDatas[relativeReadIndex]; - readPosition++; return C.RESULT_BUFFER_READ; } @@ -842,8 +881,15 @@ private void onFormatResult(Format newFormat, FormatHolder outputFormatHolder) { @Nullable DrmInitData newDrmInitData = newFormat.drmInitData; outputFormatHolder.format = - newFormat.copyWithExoMediaCryptoType(drmSessionManager.getExoMediaCryptoType(newFormat)); + drmSessionManager != null + ? newFormat.copyWithExoMediaCryptoType( + drmSessionManager.getExoMediaCryptoType(newFormat)) + : newFormat; outputFormatHolder.drmSession = currentDrmSession; + if (drmSessionManager == null) { + // This sample queue is not expected to handle DRM. Nothing to do. + return; + } if (!isFirstFormat && Util.areEqual(oldDrmInitData, newDrmInitData)) { // Nothing to do. return; @@ -852,7 +898,8 @@ private void onFormatResult(Format newFormat, FormatHolder outputFormatHolder) { // is being used for both DrmInitData. @Nullable DrmSession previousSession = currentDrmSession; currentDrmSession = - drmSessionManager.acquireSession(playbackLooper, drmEventDispatcher, newFormat); + drmSessionManager.acquireSession( + Assertions.checkNotNull(playbackLooper), drmEventDispatcher, newFormat); outputFormatHolder.drmSession = currentDrmSession; if (previousSession != null) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/SilenceMediaSource.java b/library/core/src/main/java/com/google/android/exoplayer2/source/SilenceMediaSource.java index 26b783f9700..b802717ee20 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/SilenceMediaSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/SilenceMediaSource.java @@ -25,7 +25,7 @@ import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.SeekParameters; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.TransferListener; import com.google.android.exoplayer2.util.Assertions; @@ -132,7 +132,7 @@ protected void prepareSourceInternal(@Nullable TransferListener mediaTransferLis durationUs, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, mediaItem)); } @@ -194,7 +194,7 @@ public TrackGroupArray getTrackGroups() { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/SinglePeriodTimeline.java b/library/core/src/main/java/com/google/android/exoplayer2/source/SinglePeriodTimeline.java index 54230a8b4f9..9c9f2265adc 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/SinglePeriodTimeline.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/SinglePeriodTimeline.java @@ -31,10 +31,7 @@ public final class SinglePeriodTimeline extends Timeline { private static final Object UID = new Object(); private static final MediaItem MEDIA_ITEM = - new MediaItem.Builder() - .setMediaId("com.google.android.exoplayer2.source.SinglePeriodTimeline") - .setUri(Uri.EMPTY) - .build(); + new MediaItem.Builder().setMediaId("SinglePeriodTimeline").setUri(Uri.EMPTY).build(); private final long presentationStartTimeMs; private final long windowStartTimeMs; @@ -45,9 +42,9 @@ public final class SinglePeriodTimeline extends Timeline { private final long windowDefaultStartPositionUs; private final boolean isSeekable; private final boolean isDynamic; - private final boolean isLive; @Nullable private final Object manifest; @Nullable private final MediaItem mediaItem; + @Nullable private final MediaItem.LiveConfiguration liveConfiguration; /** * @deprecated Use {@link #SinglePeriodTimeline(long, boolean, boolean, boolean, Object, @@ -81,7 +78,8 @@ public SinglePeriodTimeline( * @param durationUs The duration of the period, in microseconds. * @param isSeekable Whether seeking is supported within the period. * @param isDynamic Whether the window may change when the timeline is updated. - * @param isLive Whether the window is live. + * @param useLiveConfiguration Whether the window is live and {@link MediaItem#liveConfiguration} + * is used to configure live playback behaviour. * @param manifest The manifest. May be {@code null}. * @param mediaItem A media item used for {@link Window#mediaItem}. */ @@ -89,7 +87,7 @@ public SinglePeriodTimeline( long durationUs, boolean isSeekable, boolean isDynamic, - boolean isLive, + boolean useLiveConfiguration, @Nullable Object manifest, MediaItem mediaItem) { this( @@ -99,7 +97,7 @@ public SinglePeriodTimeline( /* windowDefaultStartPositionUs= */ 0, isSeekable, isDynamic, - isLive, + useLiveConfiguration, manifest, mediaItem); } @@ -148,7 +146,8 @@ public SinglePeriodTimeline( * which to begin playback, in microseconds. * @param isSeekable Whether seeking is supported within the window. * @param isDynamic Whether the window may change when the timeline is updated. - * @param isLive Whether the window is live. + * @param useLiveConfiguration Whether the window is live and {@link MediaItem#liveConfiguration} + * is used to configure live playback behaviour. * @param manifest The manifest. May be (@code null}. * @param mediaItem A media item used for {@link Timeline.Window#mediaItem}. */ @@ -159,7 +158,7 @@ public SinglePeriodTimeline( long windowDefaultStartPositionUs, boolean isSeekable, boolean isDynamic, - boolean isLive, + boolean useLiveConfiguration, @Nullable Object manifest, MediaItem mediaItem) { this( @@ -172,14 +171,14 @@ public SinglePeriodTimeline( windowDefaultStartPositionUs, isSeekable, isDynamic, - isLive, manifest, - mediaItem); + mediaItem, + useLiveConfiguration ? mediaItem.liveConfiguration : null); } /** * @deprecated Use {@link #SinglePeriodTimeline(long, long, long, long, long, long, long, boolean, - * boolean, boolean, Object, MediaItem)} instead. + * boolean, Object, MediaItem, MediaItem.LiveConfiguration)} instead. */ @Deprecated public SinglePeriodTimeline( @@ -205,9 +204,9 @@ public SinglePeriodTimeline( windowDefaultStartPositionUs, isSeekable, isDynamic, - isLive, manifest, - MEDIA_ITEM.buildUpon().setTag(tag).build()); + MEDIA_ITEM.buildUpon().setTag(tag).build(), + isLive ? MEDIA_ITEM.liveConfiguration : null); } /** @@ -229,9 +228,10 @@ public SinglePeriodTimeline( * which to begin playback, in microseconds. * @param isSeekable Whether seeking is supported within the window. * @param isDynamic Whether the window may change when the timeline is updated. - * @param isLive Whether the window is live. * @param manifest The manifest. May be {@code null}. * @param mediaItem A media item used for {@link Timeline.Window#mediaItem}. + * @param liveConfiguration The configuration for live playback behaviour, or {@code null} if the + * window is not live. */ public SinglePeriodTimeline( long presentationStartTimeMs, @@ -243,9 +243,9 @@ public SinglePeriodTimeline( long windowDefaultStartPositionUs, boolean isSeekable, boolean isDynamic, - boolean isLive, @Nullable Object manifest, - MediaItem mediaItem) { + MediaItem mediaItem, + @Nullable MediaItem.LiveConfiguration liveConfiguration) { this.presentationStartTimeMs = presentationStartTimeMs; this.windowStartTimeMs = windowStartTimeMs; this.elapsedRealtimeEpochOffsetMs = elapsedRealtimeEpochOffsetMs; @@ -255,9 +255,9 @@ public SinglePeriodTimeline( this.windowDefaultStartPositionUs = windowDefaultStartPositionUs; this.isSeekable = isSeekable; this.isDynamic = isDynamic; - this.isLive = isLive; this.manifest = manifest; this.mediaItem = checkNotNull(mediaItem); + this.liveConfiguration = liveConfiguration; } @Override @@ -291,7 +291,7 @@ public Window getWindow(int windowIndex, Window window, long defaultPositionProj elapsedRealtimeEpochOffsetMs, isSeekable, isDynamic, - isLive, + liveConfiguration, windowDefaultStartPositionUs, windowDurationUs, /* firstPeriodIndex= */ 0, diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/SingleSampleMediaPeriod.java b/library/core/src/main/java/com/google/android/exoplayer2/source/SingleSampleMediaPeriod.java index 352785d37d9..9e5d8aae546 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/SingleSampleMediaPeriod.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/SingleSampleMediaPeriod.java @@ -22,7 +22,7 @@ import com.google.android.exoplayer2.SeekParameters; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; import com.google.android.exoplayer2.source.MediaSourceEventListener.EventDispatcher; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DataSpec; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy; @@ -33,6 +33,7 @@ import com.google.android.exoplayer2.upstream.StatsDataSource; import com.google.android.exoplayer2.upstream.TransferListener; import com.google.android.exoplayer2.util.Assertions; +import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.Util; import java.io.IOException; @@ -41,15 +42,13 @@ import org.checkerframework.checker.nullness.compatqual.NullableType; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; -/** - * A {@link MediaPeriod} with a single sample. - */ -/* package */ final class SingleSampleMediaPeriod implements MediaPeriod, - Loader.Callback { +/** A {@link MediaPeriod} with a single sample. */ +/* package */ final class SingleSampleMediaPeriod + implements MediaPeriod, Loader.Callback { + + private static final String TAG = "SingleSampleMediaPeriod"; - /** - * The initial size of the allocation used to hold the sample data. - */ + /** The initial size of the allocation used to hold the sample data. */ private static final int INITIAL_SAMPLE_SIZE = 1024; private final DataSpec dataSpec; @@ -113,7 +112,7 @@ public TrackGroupArray getTrackGroups() { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, @@ -294,6 +293,7 @@ public LoadErrorAction onLoadError( LoadErrorAction action; if (treatLoadErrorsAsEndOfStream && errorCanBePropagated) { + Log.w(TAG, "Loading failed, treating as end-of-stream.", error); loadingFinished = true; action = Loader.DONT_RETRY; } else { @@ -348,8 +348,8 @@ public void maybeThrowError() throws IOException { } @Override - public int readData(FormatHolder formatHolder, DecoderInputBuffer buffer, - boolean requireFormat) { + public int readData( + FormatHolder formatHolder, DecoderInputBuffer buffer, boolean requireFormat) { maybeNotifyDownstreamFormat(); if (streamState == STREAM_STATE_END_OF_STREAM) { buffer.addFlag(C.BUFFER_FLAG_END_OF_STREAM); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/SingleSampleMediaSource.java b/library/core/src/main/java/com/google/android/exoplayer2/source/SingleSampleMediaSource.java index 6cb8a451b3a..3d557cf17d2 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/SingleSampleMediaSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/SingleSampleMediaSource.java @@ -19,9 +19,7 @@ import static com.google.android.exoplayer2.util.Util.castNonNull; import android.net.Uri; -import android.os.Handler; import androidx.annotation.Nullable; -import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.Timeline; @@ -31,7 +29,6 @@ import com.google.android.exoplayer2.upstream.DefaultLoadErrorHandlingPolicy; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy; import com.google.android.exoplayer2.upstream.TransferListener; -import java.io.IOException; import java.util.Collections; /** @@ -39,24 +36,6 @@ */ public final class SingleSampleMediaSource extends BaseMediaSource { - /** - * Listener of {@link SingleSampleMediaSource} events. - * - * @deprecated Use {@link MediaSourceEventListener}. - */ - @Deprecated - public interface EventListener { - - /** - * Called when an error occurs loading media data. - * - * @param sourceId The id of the reporting {@link SingleSampleMediaSource}. - * @param e The cause of the failure. - */ - void onLoadError(int sourceId, IOException e); - - } - /** Factory for {@link SingleSampleMediaSource}. */ public static final class Factory { @@ -76,6 +55,7 @@ public static final class Factory { public Factory(DataSource.Factory dataSourceFactory) { this.dataSourceFactory = checkNotNull(dataSourceFactory); loadErrorHandlingPolicy = new DefaultLoadErrorHandlingPolicy(); + treatLoadErrorsAsEndOfStream = true; } /** @@ -102,29 +82,10 @@ public Factory setTrackId(@Nullable String trackId) { return this; } - /** - * Sets the minimum number of times to retry if a loading error occurs. See {@link - * #setLoadErrorHandlingPolicy} for the default value. - * - *

    Calling this method is equivalent to calling {@link #setLoadErrorHandlingPolicy} with - * {@link DefaultLoadErrorHandlingPolicy#DefaultLoadErrorHandlingPolicy(int) - * DefaultLoadErrorHandlingPolicy(minLoadableRetryCount)} - * - * @param minLoadableRetryCount The minimum number of times to retry if a loading error occurs. - * @return This factory, for convenience. - * @deprecated Use {@link #setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)} instead. - */ - @Deprecated - public Factory setMinLoadableRetryCount(int minLoadableRetryCount) { - return setLoadErrorHandlingPolicy(new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount)); - } - /** * Sets the {@link LoadErrorHandlingPolicy}. The default value is created by calling {@link * DefaultLoadErrorHandlingPolicy#DefaultLoadErrorHandlingPolicy()}. * - *

    Calling this method overrides any calls to {@link #setMinLoadableRetryCount(int)}. - * * @param loadErrorHandlingPolicy A {@link LoadErrorHandlingPolicy}. * @return This factory, for convenience. */ @@ -139,7 +100,7 @@ public Factory setLoadErrorHandlingPolicy( /** * Sets whether load errors will be treated as end-of-stream signal (load errors will not be - * propagated). The default value is false. + * propagated). The default value is true. * * @param treatLoadErrorsAsEndOfStream If true, load errors will not be propagated by sample * streams, treating them as ended instead. If false, load errors will be propagated @@ -195,67 +156,6 @@ uri, checkNotNull(format.sampleMimeType), format.language, format.selectionFlags @Nullable private TransferListener transferListener; - /** @deprecated Use {@link Factory} instead. */ - @Deprecated - @SuppressWarnings("deprecation") - public SingleSampleMediaSource( - Uri uri, DataSource.Factory dataSourceFactory, Format format, long durationUs) { - this( - uri, - dataSourceFactory, - format, - durationUs, - DefaultLoadErrorHandlingPolicy.DEFAULT_MIN_LOADABLE_RETRY_COUNT); - } - - /** @deprecated Use {@link Factory} instead. */ - @SuppressWarnings("deprecation") - @Deprecated - public SingleSampleMediaSource( - Uri uri, - DataSource.Factory dataSourceFactory, - Format format, - long durationUs, - int minLoadableRetryCount) { - this( - uri, - dataSourceFactory, - format, - durationUs, - minLoadableRetryCount, - /* eventHandler= */ null, - /* eventListener= */ null, - /* ignored */ C.INDEX_UNSET, - /* treatLoadErrorsAsEndOfStream= */ false); - } - - /** @deprecated Use {@link Factory} instead. */ - @SuppressWarnings("deprecation") - @Deprecated - public SingleSampleMediaSource( - Uri uri, - DataSource.Factory dataSourceFactory, - Format format, - long durationUs, - int minLoadableRetryCount, - @Nullable Handler eventHandler, - @Nullable EventListener eventListener, - int eventSourceId, - boolean treatLoadErrorsAsEndOfStream) { - this( - /* trackId= */ null, - new MediaItem.Subtitle( - uri, checkNotNull(format.sampleMimeType), format.language, format.selectionFlags), - dataSourceFactory, - durationUs, - new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount), - treatLoadErrorsAsEndOfStream, - /* tag= */ null); - if (eventHandler != null && eventListener != null) { - addEventListener(eventHandler, new EventListenerWrapper(eventListener, eventSourceId)); - } - } - private SingleSampleMediaSource( @Nullable String trackId, MediaItem.Subtitle subtitle, @@ -291,7 +191,7 @@ private SingleSampleMediaSource( durationUs, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, mediaItem); } @@ -347,32 +247,4 @@ public void releasePeriod(MediaPeriod mediaPeriod) { protected void releaseSourceInternal() { // Do nothing. } - - /** - * Wraps a deprecated {@link EventListener}, invoking its callback from the equivalent callback in - * {@link MediaSourceEventListener}. - */ - @Deprecated - @SuppressWarnings("deprecation") - private static final class EventListenerWrapper implements MediaSourceEventListener { - - private final EventListener eventListener; - private final int eventSourceId; - - public EventListenerWrapper(EventListener eventListener, int eventSourceId) { - this.eventListener = checkNotNull(eventListener); - this.eventSourceId = eventSourceId; - } - - @Override - public void onLoadError( - int windowIndex, - @Nullable MediaPeriodId mediaPeriodId, - LoadEventInfo loadEventInfo, - MediaLoadData mediaLoadData, - IOException error, - boolean wasCanceled) { - eventListener.onLoadError(eventSourceId, error); - } - } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdsLoader.java b/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdsLoader.java index fda5e15215d..a6e450f8e9f 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdsLoader.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdsLoader.java @@ -38,16 +38,17 @@ * with a new copy of the current {@link AdPlaybackState} whenever further information about ads * becomes known (for example, when an ad media URI is available, or an ad has played to the end). * - *

    {@link #start(EventListener, AdViewProvider)} will be called when the ads media source first - * initializes, at which point the loader can request ads. If the player enters the background, - * {@link #stop()} will be called. Loaders should maintain any ad playback state in preparation for - * a later call to {@link #start(EventListener, AdViewProvider)}. If an ad is playing when the - * player is detached, update the ad playback state with the current playback position using {@link - * AdPlaybackState#withAdResumePositionUs(long)}. + *

    {@link #start(AdsMediaSource, DataSpec, Object, AdViewProvider, EventListener)} will be called + * when an ads media source first initializes, at which point the loader can request ads. If the + * player enters the background, {@link #stop(AdsMediaSource, EventListener)} will be called. + * Loaders should maintain any ad playback state in preparation for a later call to {@link + * #start(AdsMediaSource, DataSpec, Object, AdViewProvider, EventListener)}. If an ad is playing + * when the player is detached, update the ad playback state with the current playback position + * using {@link AdPlaybackState#withAdResumePositionUs(long)}. * *

    If {@link EventListener#onAdPlaybackState(AdPlaybackState)} has been called, the - * implementation of {@link #start(EventListener, AdViewProvider)} should invoke the same listener - * to provide the existing playback state to the new player. + * implementation of {@link #start(AdsMediaSource, DataSpec, Object, AdViewProvider, EventListener)} + * should invoke the same listener to provide the existing playback state to the new player. */ public interface AdsLoader { @@ -190,53 +191,59 @@ public OverlayInfo(View view, @Purpose int purpose, @Nullable String detailedRea /** * Sets the supported content types for ad media. Must be called before the first call to {@link - * #start(EventListener, AdViewProvider)}. Subsequent calls may be ignored. Called on the main - * thread by {@link AdsMediaSource}. + * #start(AdsMediaSource, DataSpec, Object, AdViewProvider, EventListener)}. Subsequent calls may + * be ignored. Called on the main thread by {@link AdsMediaSource}. * * @param contentTypes The supported content types for ad media. Each element must be one of * {@link C#TYPE_DASH}, {@link C#TYPE_HLS}, {@link C#TYPE_SS} and {@link C#TYPE_OTHER}. */ void setSupportedContentTypes(@C.ContentType int... contentTypes); - /** - * Sets the data spec of the ad tag to load. - * - * @param adTagDataSpec The data spec of the ad tag to load. See the implementation's - * documentation for information about compatible ad tag formats. - */ - void setAdTagDataSpec(DataSpec adTagDataSpec); - /** * Starts using the ads loader for playback. Called on the main thread by {@link AdsMediaSource}. * - * @param eventListener Listener for ads loader events. + * @param adsMediaSource The ads media source requesting to start loading ads. + * @param adTagDataSpec A data spec for the ad tag to load. + * @param adsId An opaque identifier for the ad playback state across start/stop calls. * @param adViewProvider Provider of views for the ad UI. + * @param eventListener Listener for ads loader events. */ - void start(EventListener eventListener, AdViewProvider adViewProvider); + void start( + AdsMediaSource adsMediaSource, + DataSpec adTagDataSpec, + Object adsId, + AdViewProvider adViewProvider, + EventListener eventListener); /** * Stops using the ads loader for playback and deregisters the event listener. Called on the main * thread by {@link AdsMediaSource}. + * + * @param adsMediaSource The ads media source requesting to stop loading/playing ads. + * @param eventListener The ads media source's listener for ads loader events. */ - void stop(); + void stop(AdsMediaSource adsMediaSource, EventListener eventListener); /** * Notifies the ads loader that preparation of an ad media period is complete. Called on the main * thread by {@link AdsMediaSource}. * + * @param adsMediaSource The ads media source for which preparation of ad media completed. * @param adGroupIndex The index of the ad group. * @param adIndexInAdGroup The index of the ad in the ad group. */ - void handlePrepareComplete(int adGroupIndex, int adIndexInAdGroup); + void handlePrepareComplete(AdsMediaSource adsMediaSource, int adGroupIndex, int adIndexInAdGroup); /** * Notifies the ads loader that the player was not able to prepare media for a given ad. * Implementations should update the ad playback state as the specified ad has failed to load. * Called on the main thread by {@link AdsMediaSource}. * + * @param adsMediaSource The ads media source for which preparation of ad media failed. * @param adGroupIndex The index of the ad group. * @param adIndexInAdGroup The index of the ad in the ad group. * @param exception The preparation error. */ - void handlePrepareError(int adGroupIndex, int adIndexInAdGroup, IOException exception); + void handlePrepareError( + AdsMediaSource adsMediaSource, int adGroupIndex, int adIndexInAdGroup, IOException exception); } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdsMediaSource.java b/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdsMediaSource.java index 10e9d8e3bed..4f2617e8687 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdsMediaSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/ads/AdsMediaSource.java @@ -35,9 +35,7 @@ import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.MediaSourceEventListener; import com.google.android.exoplayer2.source.MediaSourceFactory; -import com.google.android.exoplayer2.source.ProgressiveMediaSource; import com.google.android.exoplayer2.upstream.Allocator; -import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DataSpec; import com.google.android.exoplayer2.upstream.TransferListener; import com.google.android.exoplayer2.util.Assertions; @@ -130,7 +128,8 @@ public RuntimeException getRuntimeExceptionForUnexpected() { private final MediaSourceFactory adMediaSourceFactory; private final AdsLoader adsLoader; private final AdsLoader.AdViewProvider adViewProvider; - @Nullable private final DataSpec adTagDataSpec; + private final DataSpec adTagDataSpec; + private final Object adsId; private final Handler mainHandler; private final Timeline.Period period; @@ -140,62 +139,16 @@ public RuntimeException getRuntimeExceptionForUnexpected() { @Nullable private AdPlaybackState adPlaybackState; private @NullableType AdMediaSourceHolder[][] adMediaSourceHolders; - /** - * Constructs a new source that inserts ads linearly with the content specified by {@code - * contentMediaSource}. Ad media is loaded using {@link ProgressiveMediaSource}. - * - * @param contentMediaSource The {@link MediaSource} providing the content to play. - * @param dataSourceFactory Factory for data sources used to load ad media. - * @param adsLoader The loader for ads. - * @param adViewProvider Provider of views for the ad UI. - * @deprecated Use {@link AdsMediaSource#AdsMediaSource(MediaSource, DataSpec, MediaSourceFactory, - * AdsLoader, AdsLoader.AdViewProvider)} instead. - */ - @Deprecated - public AdsMediaSource( - MediaSource contentMediaSource, - DataSource.Factory dataSourceFactory, - AdsLoader adsLoader, - AdsLoader.AdViewProvider adViewProvider) { - this( - contentMediaSource, - new ProgressiveMediaSource.Factory(dataSourceFactory), - adsLoader, - adViewProvider, - /* adTagDataSpec= */ null); - } - - /** - * Constructs a new source that inserts ads linearly with the content specified by {@code - * contentMediaSource}. - * - * @param contentMediaSource The {@link MediaSource} providing the content to play. - * @param adMediaSourceFactory Factory for media sources used to load ad media. - * @param adsLoader The loader for ads. - * @param adViewProvider Provider of views for the ad UI. - * @deprecated Use {@link AdsMediaSource#AdsMediaSource(MediaSource, DataSpec, MediaSourceFactory, - * AdsLoader, AdsLoader.AdViewProvider)} instead. - */ - @Deprecated - public AdsMediaSource( - MediaSource contentMediaSource, - MediaSourceFactory adMediaSourceFactory, - AdsLoader adsLoader, - AdsLoader.AdViewProvider adViewProvider) { - this( - contentMediaSource, - adMediaSourceFactory, - adsLoader, - adViewProvider, - /* adTagDataSpec= */ null); - } - /** * Constructs a new source that inserts ads linearly with the content specified by {@code * contentMediaSource}. * * @param contentMediaSource The {@link MediaSource} providing the content to play. * @param adTagDataSpec The data specification of the ad tag to load. + * @param adsId An opaque identifier for ad playback state associated with this instance. Ad + * loading and playback state is shared among all playlist items that have the same ads id (by + * {@link Object#equals(Object) equality}), so it is important to pass the same identifiers + * when constructing playlist items each time the player returns to the foreground. * @param adMediaSourceFactory Factory for media sources used to load ad media. * @param adsLoader The loader for ads. * @param adViewProvider Provider of views for the ad UI. @@ -203,23 +156,16 @@ public AdsMediaSource( public AdsMediaSource( MediaSource contentMediaSource, DataSpec adTagDataSpec, + Object adsId, MediaSourceFactory adMediaSourceFactory, AdsLoader adsLoader, AdsLoader.AdViewProvider adViewProvider) { - this(contentMediaSource, adMediaSourceFactory, adsLoader, adViewProvider, adTagDataSpec); - } - - private AdsMediaSource( - MediaSource contentMediaSource, - MediaSourceFactory adMediaSourceFactory, - AdsLoader adsLoader, - AdsLoader.AdViewProvider adViewProvider, - @Nullable DataSpec adTagDataSpec) { this.contentMediaSource = contentMediaSource; this.adMediaSourceFactory = adMediaSourceFactory; this.adsLoader = adsLoader; this.adViewProvider = adViewProvider; this.adTagDataSpec = adTagDataSpec; + this.adsId = adsId; mainHandler = new Handler(Looper.getMainLooper()); period = new Timeline.Period(); adMediaSourceHolders = new AdMediaSourceHolder[0][]; @@ -249,12 +195,13 @@ protected void prepareSourceInternal(@Nullable TransferListener mediaTransferLis this.componentListener = componentListener; prepareChildSource(CHILD_SOURCE_MEDIA_PERIOD_ID, contentMediaSource); mainHandler.post( - () -> { - if (adTagDataSpec != null) { - adsLoader.setAdTagDataSpec(adTagDataSpec); - } - adsLoader.start(componentListener, adViewProvider); - }); + () -> + adsLoader.start( + /* adsMediaSource= */ this, + adTagDataSpec, + adsId, + adViewProvider, + componentListener)); } @Override @@ -305,12 +252,13 @@ public void releasePeriod(MediaPeriod mediaPeriod) { @Override protected void releaseSourceInternal() { super.releaseSourceInternal(); - checkNotNull(componentListener).release(); - componentListener = null; + ComponentListener componentListener = checkNotNull(this.componentListener); + this.componentListener = null; + componentListener.stop(); contentTimeline = null; adPlaybackState = null; adMediaSourceHolders = new AdMediaSourceHolder[0][]; - mainHandler.post(adsLoader::stop); + mainHandler.post(() -> adsLoader.stop(/* adsMediaSource= */ this, componentListener)); } @Override @@ -408,7 +356,7 @@ private final class ComponentListener implements AdsLoader.EventListener { private final Handler playerHandler; - private volatile boolean released; + private volatile boolean stopped; /** * Creates new listener which forwards ad playback states on the creating thread and all other @@ -418,20 +366,20 @@ public ComponentListener() { playerHandler = Util.createHandlerForCurrentLooper(); } - /** Releases the component listener. */ - public void release() { - released = true; + /** Stops event delivery from this instance. */ + public void stop() { + stopped = true; playerHandler.removeCallbacksAndMessages(null); } @Override public void onAdPlaybackState(final AdPlaybackState adPlaybackState) { - if (released) { + if (stopped) { return; } playerHandler.post( () -> { - if (released) { + if (stopped) { return; } AdsMediaSource.this.onAdPlaybackState(adPlaybackState); @@ -440,7 +388,7 @@ public void onAdPlaybackState(final AdPlaybackState adPlaybackState) { @Override public void onAdLoadError(final AdLoadException error, DataSpec dataSpec) { - if (released) { + if (stopped) { return; } createEventDispatcher(/* mediaPeriodId= */ null) @@ -468,7 +416,9 @@ public void onPrepareComplete(MediaPeriodId mediaPeriodId) { mainHandler.post( () -> adsLoader.handlePrepareComplete( - mediaPeriodId.adGroupIndex, mediaPeriodId.adIndexInAdGroup)); + /* adsMediaSource= */ AdsMediaSource.this, + mediaPeriodId.adGroupIndex, + mediaPeriodId.adIndexInAdGroup)); } @Override @@ -485,7 +435,10 @@ public void onPrepareError(MediaPeriodId mediaPeriodId, IOException exception) { mainHandler.post( () -> adsLoader.handlePrepareError( - mediaPeriodId.adGroupIndex, mediaPeriodId.adIndexInAdGroup, exception)); + /* adsMediaSource= */ AdsMediaSource.this, + mediaPeriodId.adGroupIndex, + mediaPeriodId.adIndexInAdGroup, + exception)); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/chunk/ChunkSampleStream.java b/library/core/src/main/java/com/google/android/exoplayer2/source/chunk/ChunkSampleStream.java index 1a5371c83f8..88a8a0d1eae 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/source/chunk/ChunkSampleStream.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/chunk/ChunkSampleStream.java @@ -145,7 +145,7 @@ public ChunkSampleStream( SampleQueue[] sampleQueues = new SampleQueue[1 + embeddedTrackCount]; primarySampleQueue = - new SampleQueue( + SampleQueue.createWithDrm( allocator, /* playbackLooper= */ checkNotNull(Looper.myLooper()), drmSessionManager, @@ -154,12 +154,7 @@ public ChunkSampleStream( sampleQueues[0] = primarySampleQueue; for (int i = 0; i < embeddedTrackCount; i++) { - SampleQueue sampleQueue = - new SampleQueue( - allocator, - /* playbackLooper= */ checkNotNull(Looper.myLooper()), - DrmSessionManager.getDummyDrmSessionManager(), - drmEventDispatcher); + SampleQueue sampleQueue = SampleQueue.createWithoutDrm(allocator); embeddedSampleQueues[i] = sampleQueue; sampleQueues[i + 1] = sampleQueue; trackTypes[i + 1] = this.embeddedTrackTypes[i]; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/chunk/MediaParserChunkExtractor.java b/library/core/src/main/java/com/google/android/exoplayer2/source/chunk/MediaParserChunkExtractor.java new file mode 100644 index 00000000000..7c440b46d71 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/chunk/MediaParserChunkExtractor.java @@ -0,0 +1,169 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.chunk; + +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_EAGERLY_EXPOSE_TRACK_TYPE; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_EXPOSE_CAPTION_FORMATS; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_EXPOSE_CHUNK_INDEX_AS_MEDIA_FORMAT; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_EXPOSE_DUMMY_SEEK_MAP; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_INCLUDE_SUPPLEMENTAL_DATA; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_IN_BAND_CRYPTO_INFO; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_OVERRIDE_IN_BAND_CAPTION_DECLARATIONS; + +import android.annotation.SuppressLint; +import android.media.MediaFormat; +import android.media.MediaParser; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.extractor.ChunkIndex; +import com.google.android.exoplayer2.extractor.DummyTrackOutput; +import com.google.android.exoplayer2.extractor.ExtractorInput; +import com.google.android.exoplayer2.extractor.ExtractorOutput; +import com.google.android.exoplayer2.extractor.SeekMap; +import com.google.android.exoplayer2.extractor.TrackOutput; +import com.google.android.exoplayer2.source.mediaparser.InputReaderAdapterV30; +import com.google.android.exoplayer2.source.mediaparser.MediaParserUtil; +import com.google.android.exoplayer2.source.mediaparser.OutputConsumerAdapterV30; +import com.google.android.exoplayer2.util.Assertions; +import com.google.android.exoplayer2.util.MimeTypes; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; + +/** {@link ChunkExtractor} implemented on top of the platform's {@link MediaParser}. */ +@RequiresApi(30) +public final class MediaParserChunkExtractor implements ChunkExtractor { + + private final OutputConsumerAdapterV30 outputConsumerAdapter; + private final InputReaderAdapterV30 inputReaderAdapter; + private final MediaParser mediaParser; + private final TrackOutputProviderAdapter trackOutputProviderAdapter; + private final DummyTrackOutput dummyTrackOutput; + private long pendingSeekUs; + @Nullable private TrackOutputProvider trackOutputProvider; + @Nullable private Format[] sampleFormats; + + /** + * Creates a new instance. + * + * @param primaryTrackType The type of the primary track, or {@link C#TRACK_TYPE_NONE} if there is + * no primary track. Must be one of the {@link C C.TRACK_TYPE_*} constants. + * @param manifestFormat The chunks {@link Format} as obtained from the manifest. + * @param closedCaptionFormats A list containing the {@link Format Formats} of the closed-caption + * tracks in the chunks. + */ + @SuppressLint("WrongConstant") + public MediaParserChunkExtractor( + int primaryTrackType, Format manifestFormat, List closedCaptionFormats) { + outputConsumerAdapter = + new OutputConsumerAdapterV30( + manifestFormat, primaryTrackType, /* expectDummySeekMap= */ true); + inputReaderAdapter = new InputReaderAdapterV30(); + String mimeType = Assertions.checkNotNull(manifestFormat.containerMimeType); + String parserName = + MimeTypes.isMatroska(mimeType) + ? MediaParser.PARSER_NAME_MATROSKA + : MediaParser.PARSER_NAME_FMP4; + outputConsumerAdapter.setSelectedParserName(parserName); + mediaParser = MediaParser.createByName(parserName, outputConsumerAdapter); + mediaParser.setParameter(MediaParser.PARAMETER_MATROSKA_DISABLE_CUES_SEEKING, true); + mediaParser.setParameter(PARAMETER_IN_BAND_CRYPTO_INFO, true); + mediaParser.setParameter(PARAMETER_INCLUDE_SUPPLEMENTAL_DATA, true); + mediaParser.setParameter(PARAMETER_EAGERLY_EXPOSE_TRACK_TYPE, true); + mediaParser.setParameter(PARAMETER_EXPOSE_DUMMY_SEEK_MAP, true); + mediaParser.setParameter(PARAMETER_EXPOSE_CHUNK_INDEX_AS_MEDIA_FORMAT, true); + mediaParser.setParameter(PARAMETER_OVERRIDE_IN_BAND_CAPTION_DECLARATIONS, true); + ArrayList closedCaptionMediaFormats = new ArrayList<>(); + for (int i = 0; i < closedCaptionFormats.size(); i++) { + closedCaptionMediaFormats.add( + MediaParserUtil.toCaptionsMediaFormat(closedCaptionFormats.get(i))); + } + mediaParser.setParameter(PARAMETER_EXPOSE_CAPTION_FORMATS, closedCaptionMediaFormats); + outputConsumerAdapter.setMuxedCaptionFormats(closedCaptionFormats); + trackOutputProviderAdapter = new TrackOutputProviderAdapter(); + dummyTrackOutput = new DummyTrackOutput(); + pendingSeekUs = C.TIME_UNSET; + } + + // ChunkExtractor implementation. + + @Override + public void init( + @Nullable TrackOutputProvider trackOutputProvider, long startTimeUs, long endTimeUs) { + this.trackOutputProvider = trackOutputProvider; + outputConsumerAdapter.setSampleTimestampUpperLimitFilterUs(endTimeUs); + outputConsumerAdapter.setExtractorOutput(trackOutputProviderAdapter); + pendingSeekUs = startTimeUs; + } + + @Override + public void release() { + mediaParser.release(); + } + + @Override + public boolean read(ExtractorInput extractorInput) throws IOException { + maybeExecutePendingSeek(); + inputReaderAdapter.setDataReader(extractorInput, extractorInput.getLength()); + return mediaParser.advance(inputReaderAdapter); + } + + @Nullable + @Override + public ChunkIndex getChunkIndex() { + return outputConsumerAdapter.getChunkIndex(); + } + + @Nullable + @Override + public Format[] getSampleFormats() { + return sampleFormats; + } + + // Internal methods. + + private void maybeExecutePendingSeek() { + @Nullable MediaParser.SeekMap dummySeekMap = outputConsumerAdapter.getDummySeekMap(); + if (pendingSeekUs != C.TIME_UNSET && dummySeekMap != null) { + mediaParser.seek(dummySeekMap.getSeekPoints(pendingSeekUs).first); + pendingSeekUs = C.TIME_UNSET; + } + } + + // Internal classes. + + private class TrackOutputProviderAdapter implements ExtractorOutput { + + @Override + public TrackOutput track(int id, int type) { + return trackOutputProvider != null ? trackOutputProvider.track(id, type) : dummyTrackOutput; + } + + @Override + public void endTracks() { + // Imitate BundledChunkExtractor behavior, which captures a sample format snapshot when + // endTracks is called. + sampleFormats = outputConsumerAdapter.getSampleFormats(); + } + + @Override + public void seekMap(SeekMap seekMap) { + // Do nothing. + } + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/InputReaderAdapterV30.java b/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/InputReaderAdapterV30.java new file mode 100644 index 00000000000..3a55645e960 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/InputReaderAdapterV30.java @@ -0,0 +1,87 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.mediaparser; + +import android.annotation.SuppressLint; +import android.media.MediaParser; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.upstream.DataReader; +import com.google.android.exoplayer2.util.Util; +import java.io.IOException; + +/** {@link MediaParser.SeekableInputReader} implementation wrapping a {@link DataReader}. */ +@RequiresApi(30) +@SuppressLint("Override") // TODO: Remove once the SDK becomes stable. +public final class InputReaderAdapterV30 implements MediaParser.SeekableInputReader { + + @Nullable private DataReader dataReader; + private long resourceLength; + private long currentPosition; + private long lastSeekPosition; + + /** + * Sets the wrapped {@link DataReader}. + * + * @param dataReader The {@link DataReader} to wrap. + * @param length The length of the resource from which {@code dataReader} reads. + */ + public void setDataReader(DataReader dataReader, long length) { + this.dataReader = dataReader; + resourceLength = length; + lastSeekPosition = C.POSITION_UNSET; + } + + /** Sets the absolute position in the resource from which the wrapped {@link DataReader} reads. */ + public void setCurrentPosition(long position) { + currentPosition = position; + } + + /** + * Returns the last value passed to {@link #seekToPosition(long)} and sets the stored value to + * {@link C#POSITION_UNSET}. + */ + public long getAndResetSeekPosition() { + long lastSeekPosition = this.lastSeekPosition; + this.lastSeekPosition = C.POSITION_UNSET; + return lastSeekPosition; + } + + // SeekableInputReader implementation. + + @Override + public void seekToPosition(long position) { + lastSeekPosition = position; + } + + @Override + public int read(byte[] bytes, int offset, int readLength) throws IOException { + int bytesRead = Util.castNonNull(dataReader).read(bytes, offset, readLength); + currentPosition += bytesRead; + return bytesRead; + } + + @Override + public long getPosition() { + return currentPosition; + } + + @Override + public long getLength() { + return resourceLength; + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/MediaParserUtil.java b/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/MediaParserUtil.java new file mode 100644 index 00000000000..db37535a15e --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/MediaParserUtil.java @@ -0,0 +1,60 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.mediaparser; + +import android.media.MediaFormat; +import android.media.MediaParser; +import com.google.android.exoplayer2.Format; + +/** + * Miscellaneous constants and utility methods related to the {@link MediaParser} integration. + * + *

    For documentation on constants, please see the {@link MediaParser} documentation. + */ +public final class MediaParserUtil { + + public static final String PARAMETER_IN_BAND_CRYPTO_INFO = + "android.media.mediaparser.inBandCryptoInfo"; + public static final String PARAMETER_INCLUDE_SUPPLEMENTAL_DATA = + "android.media.mediaparser.includeSupplementalData"; + public static final String PARAMETER_EAGERLY_EXPOSE_TRACK_TYPE = + "android.media.mediaparser.eagerlyExposeTrackType"; + public static final String PARAMETER_EXPOSE_DUMMY_SEEK_MAP = + "android.media.mediaparser.exposeDummySeekMap"; + public static final String PARAMETER_EXPOSE_CHUNK_INDEX_AS_MEDIA_FORMAT = + "android.media.mediaParser.exposeChunkIndexAsMediaFormat"; + public static final String PARAMETER_OVERRIDE_IN_BAND_CAPTION_DECLARATIONS = + "android.media.mediaParser.overrideInBandCaptionDeclarations"; + public static final String PARAMETER_EXPOSE_CAPTION_FORMATS = + "android.media.mediaParser.exposeCaptionFormats"; + public static final String PARAMETER_IGNORE_TIMESTAMP_OFFSET = + "android.media.mediaparser.ignoreTimestampOffset"; + + private MediaParserUtil() {} + + /** + * Returns a {@link MediaFormat} with equivalent {@link MediaFormat#KEY_MIME} and {@link + * MediaFormat#KEY_CAPTION_SERVICE_NUMBER} to the given {@link Format}. + */ + public static MediaFormat toCaptionsMediaFormat(Format format) { + MediaFormat mediaFormat = new MediaFormat(); + mediaFormat.setString(MediaFormat.KEY_MIME, format.sampleMimeType); + if (format.accessibilityChannel != Format.NO_VALUE) { + mediaFormat.setInteger(MediaFormat.KEY_CAPTION_SERVICE_NUMBER, format.accessibilityChannel); + } + return mediaFormat; + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/OutputConsumerAdapterV30.java b/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/OutputConsumerAdapterV30.java new file mode 100644 index 00000000000..f3bed012ec2 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/OutputConsumerAdapterV30.java @@ -0,0 +1,691 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.mediaparser; + +import static android.media.MediaParser.PARSER_NAME_AC3; +import static android.media.MediaParser.PARSER_NAME_AC4; +import static android.media.MediaParser.PARSER_NAME_ADTS; +import static android.media.MediaParser.PARSER_NAME_AMR; +import static android.media.MediaParser.PARSER_NAME_FLAC; +import static android.media.MediaParser.PARSER_NAME_FLV; +import static android.media.MediaParser.PARSER_NAME_FMP4; +import static android.media.MediaParser.PARSER_NAME_MATROSKA; +import static android.media.MediaParser.PARSER_NAME_MP3; +import static android.media.MediaParser.PARSER_NAME_MP4; +import static android.media.MediaParser.PARSER_NAME_OGG; +import static android.media.MediaParser.PARSER_NAME_PS; +import static android.media.MediaParser.PARSER_NAME_TS; +import static android.media.MediaParser.PARSER_NAME_WAV; + +import android.annotation.SuppressLint; +import android.media.DrmInitData.SchemeInitData; +import android.media.MediaCodec; +import android.media.MediaCodec.CryptoInfo; +import android.media.MediaFormat; +import android.media.MediaParser; +import android.media.MediaParser.TrackData; +import android.util.Pair; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.C.SelectionFlags; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.drm.DrmInitData; +import com.google.android.exoplayer2.drm.DrmInitData.SchemeData; +import com.google.android.exoplayer2.extractor.ChunkIndex; +import com.google.android.exoplayer2.extractor.DummyExtractorOutput; +import com.google.android.exoplayer2.extractor.ExtractorOutput; +import com.google.android.exoplayer2.extractor.SeekMap; +import com.google.android.exoplayer2.extractor.SeekPoint; +import com.google.android.exoplayer2.extractor.TrackOutput; +import com.google.android.exoplayer2.extractor.TrackOutput.CryptoData; +import com.google.android.exoplayer2.upstream.DataReader; +import com.google.android.exoplayer2.util.Assertions; +import com.google.android.exoplayer2.util.Log; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.android.exoplayer2.util.TimestampAdjuster; +import com.google.android.exoplayer2.util.Util; +import com.google.android.exoplayer2.video.ColorInfo; +import com.google.common.collect.ImmutableList; +import java.io.IOException; +import java.nio.ByteBuffer; +import java.nio.IntBuffer; +import java.nio.LongBuffer; +import java.util.ArrayList; +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import org.checkerframework.checker.nullness.compatqual.NullableType; + +/** + * {@link MediaParser.OutputConsumer} implementation that redirects output to an {@link + * ExtractorOutput}. + */ +@RequiresApi(30) +@SuppressLint("Override") // TODO: Remove once the SDK becomes stable. +public final class OutputConsumerAdapterV30 implements MediaParser.OutputConsumer { + + private static final String TAG = "OutputConsumerAdapterV30"; + + private static final Pair SEEK_POINT_PAIR_START = + Pair.create(MediaParser.SeekPoint.START, MediaParser.SeekPoint.START); + private static final String MEDIA_FORMAT_KEY_TRACK_TYPE = "track-type-string"; + private static final String MEDIA_FORMAT_KEY_CHUNK_INDEX_SIZES = "chunk-index-int-sizes"; + private static final String MEDIA_FORMAT_KEY_CHUNK_INDEX_OFFSETS = "chunk-index-long-offsets"; + private static final String MEDIA_FORMAT_KEY_CHUNK_INDEX_DURATIONS = + "chunk-index-long-us-durations"; + private static final String MEDIA_FORMAT_KEY_CHUNK_INDEX_TIMES = "chunk-index-long-us-times"; + private static final Pattern REGEX_CRYPTO_INFO_PATTERN = + Pattern.compile("pattern \\(encrypt: (\\d+), skip: (\\d+)\\)"); + + private final ArrayList<@NullableType TrackOutput> trackOutputs; + private final ArrayList<@NullableType Format> trackFormats; + private final ArrayList<@NullableType CryptoInfo> lastReceivedCryptoInfos; + private final ArrayList<@NullableType CryptoData> lastOutputCryptoDatas; + private final DataReaderAdapter scratchDataReaderAdapter; + private final boolean expectDummySeekMap; + private final int primaryTrackType; + @Nullable private final Format primaryTrackManifestFormat; + + private ExtractorOutput extractorOutput; + @Nullable private MediaParser.SeekMap dummySeekMap; + @Nullable private MediaParser.SeekMap lastSeekMap; + @Nullable private String containerMimeType; + @Nullable private ChunkIndex lastChunkIndex; + @Nullable private TimestampAdjuster timestampAdjuster; + private List muxedCaptionFormats; + private int primaryTrackIndex; + private long sampleTimestampUpperLimitFilterUs; + private boolean tracksFoundCalled; + private boolean tracksEnded; + private boolean seekingDisabled; + + /** + * Equivalent to {@link #OutputConsumerAdapterV30(Format, int, boolean) + * OutputConsumerAdapterV30(primaryTrackManifestFormat= null, primaryTrackType= C.TRACK_TYPE_NONE, + * expectDummySeekMap= false)} + */ + public OutputConsumerAdapterV30() { + this( + /* primaryTrackManifestFormat= */ null, + /* primaryTrackType= */ C.TRACK_TYPE_NONE, + /* expectDummySeekMap= */ false); + } + + /** + * Creates a new instance. + * + * @param primaryTrackManifestFormat The manifest-obtained format of the primary track, or null if + * not applicable. + * @param primaryTrackType The type of the primary track, or {@link C#TRACK_TYPE_NONE} if there is + * no primary track. Must be one of the {@link C C.TRACK_TYPE_*} constants. + * @param expectDummySeekMap Whether the output consumer should expect an initial dummy seek map + * which should be exposed through {@link #getDummySeekMap()}. + */ + public OutputConsumerAdapterV30( + @Nullable Format primaryTrackManifestFormat, + int primaryTrackType, + boolean expectDummySeekMap) { + this.expectDummySeekMap = expectDummySeekMap; + this.primaryTrackManifestFormat = primaryTrackManifestFormat; + this.primaryTrackType = primaryTrackType; + trackOutputs = new ArrayList<>(); + trackFormats = new ArrayList<>(); + lastReceivedCryptoInfos = new ArrayList<>(); + lastOutputCryptoDatas = new ArrayList<>(); + scratchDataReaderAdapter = new DataReaderAdapter(); + extractorOutput = new DummyExtractorOutput(); + sampleTimestampUpperLimitFilterUs = C.TIME_UNSET; + muxedCaptionFormats = ImmutableList.of(); + } + + /** + * Sets an upper limit for sample timestamp filtering. + * + *

    When set, samples with timestamps greater than {@code sampleTimestampUpperLimitFilterUs} + * will be discarded. + * + * @param sampleTimestampUpperLimitFilterUs The maximum allowed sample timestamp, or {@link + * C#TIME_UNSET} to remove filtering. + */ + public void setSampleTimestampUpperLimitFilterUs(long sampleTimestampUpperLimitFilterUs) { + this.sampleTimestampUpperLimitFilterUs = sampleTimestampUpperLimitFilterUs; + } + + /** Sets a {@link TimestampAdjuster} for adjusting the timestamps of the output samples. */ + public void setTimestampAdjuster(TimestampAdjuster timestampAdjuster) { + this.timestampAdjuster = timestampAdjuster; + } + + /** + * Sets the {@link ExtractorOutput} to which {@link MediaParser MediaParser's} output is directed. + */ + public void setExtractorOutput(ExtractorOutput extractorOutput) { + this.extractorOutput = extractorOutput; + } + + /** Sets {@link Format} information associated to the caption tracks multiplexed in the media. */ + public void setMuxedCaptionFormats(List muxedCaptionFormats) { + this.muxedCaptionFormats = muxedCaptionFormats; + } + + /** Overrides future received {@link SeekMap SeekMaps} with non-seekable instances. */ + public void disableSeeking() { + seekingDisabled = true; + } + + /** + * Returns a dummy {@link MediaParser.SeekMap}, or null if not available. + * + *

    the dummy {@link MediaParser.SeekMap} returns a single {@link MediaParser.SeekPoint} whose + * {@link MediaParser.SeekPoint#timeMicros} matches the requested timestamp, and {@link + * MediaParser.SeekPoint#position} is 0. + */ + @Nullable + public MediaParser.SeekMap getDummySeekMap() { + return dummySeekMap; + } + + /** Returns the most recently output {@link ChunkIndex}, or null if none has been output. */ + @Nullable + public ChunkIndex getChunkIndex() { + return lastChunkIndex; + } + + /** + * Returns the {@link MediaParser.SeekPoint} instances corresponding to the given timestamp. + * + * @param seekTimeUs The timestamp in microseconds to retrieve {@link MediaParser.SeekPoint} + * instances for. + * @return The {@link MediaParser.SeekPoint} instances corresponding to the given timestamp. + */ + public Pair getSeekPoints(long seekTimeUs) { + return lastSeekMap != null ? lastSeekMap.getSeekPoints(seekTimeUs) : SEEK_POINT_PAIR_START; + } + + /** + * Defines the container mime type to propagate through {@link TrackOutput#format}. + * + * @param parserName The name of the selected parser. + */ + public void setSelectedParserName(String parserName) { + containerMimeType = getMimeType(parserName); + } + + /** + * Returns the last output format for each track, or null if not all the tracks have been + * identified. + */ + @Nullable + public Format[] getSampleFormats() { + if (!tracksFoundCalled) { + return null; + } + Format[] sampleFormats = new Format[trackFormats.size()]; + for (int i = 0; i < trackFormats.size(); i++) { + sampleFormats[i] = Assertions.checkNotNull(trackFormats.get(i)); + } + return sampleFormats; + } + + // MediaParser.OutputConsumer implementation. + + @Override + public void onTrackCountFound(int numberOfTracks) { + tracksFoundCalled = true; + maybeEndTracks(); + } + + @Override + public void onSeekMapFound(MediaParser.SeekMap seekMap) { + if (expectDummySeekMap && dummySeekMap == null) { + // This is a dummy seek map. + dummySeekMap = seekMap; + } else { + lastSeekMap = seekMap; + long durationUs = seekMap.getDurationMicros(); + extractorOutput.seekMap( + seekingDisabled + ? new SeekMap.Unseekable( + durationUs != MediaParser.SeekMap.UNKNOWN_DURATION ? durationUs : C.TIME_UNSET) + : new SeekMapAdapter(seekMap)); + } + } + + @Override + public void onTrackDataFound(int trackIndex, TrackData trackData) { + if (maybeObtainChunkIndex(trackData.mediaFormat)) { + // The MediaFormat contains a chunk index. It does not contain anything else. + return; + } + + ensureSpaceForTrackIndex(trackIndex); + @Nullable TrackOutput trackOutput = trackOutputs.get(trackIndex); + if (trackOutput == null) { + @Nullable + String trackTypeString = trackData.mediaFormat.getString(MEDIA_FORMAT_KEY_TRACK_TYPE); + int trackType = + toTrackTypeConstant( + trackTypeString != null + ? trackTypeString + : trackData.mediaFormat.getString(MediaFormat.KEY_MIME)); + if (trackType == primaryTrackType) { + primaryTrackIndex = trackIndex; + } + trackOutput = extractorOutput.track(trackIndex, trackType); + trackOutputs.set(trackIndex, trackOutput); + if (trackTypeString != null) { + // The MediaFormat includes the track type string, so it cannot include any other keys, as + // per the android.media.mediaparser.eagerlyExposeTrackType parameter documentation. + return; + } + } + Format format = toExoPlayerFormat(trackData); + trackOutput.format( + primaryTrackManifestFormat != null && trackIndex == primaryTrackIndex + ? format.withManifestFormatInfo(primaryTrackManifestFormat) + : format); + trackFormats.set(trackIndex, format); + maybeEndTracks(); + } + + @Override + public void onSampleDataFound(int trackIndex, MediaParser.InputReader sampleData) + throws IOException { + ensureSpaceForTrackIndex(trackIndex); + scratchDataReaderAdapter.input = sampleData; + TrackOutput trackOutput = trackOutputs.get(trackIndex); + if (trackOutput == null) { + trackOutput = extractorOutput.track(trackIndex, C.TRACK_TYPE_UNKNOWN); + trackOutputs.set(trackIndex, trackOutput); + } + trackOutput.sampleData( + scratchDataReaderAdapter, (int) sampleData.getLength(), /* allowEndOfInput= */ true); + } + + @Override + public void onSampleCompleted( + int trackIndex, + long timeUs, + int flags, + int size, + int offset, + @Nullable MediaCodec.CryptoInfo cryptoInfo) { + if (sampleTimestampUpperLimitFilterUs != C.TIME_UNSET + && timeUs >= sampleTimestampUpperLimitFilterUs) { + // Ignore this sample. + return; + } else if (timestampAdjuster != null) { + timeUs = timestampAdjuster.adjustSampleTimestamp(timeUs); + } + Assertions.checkNotNull(trackOutputs.get(trackIndex)) + .sampleMetadata(timeUs, flags, size, offset, toExoPlayerCryptoData(trackIndex, cryptoInfo)); + } + + // Private methods. + + private boolean maybeObtainChunkIndex(MediaFormat mediaFormat) { + @Nullable + ByteBuffer chunkIndexSizesByteBuffer = + mediaFormat.getByteBuffer(MEDIA_FORMAT_KEY_CHUNK_INDEX_SIZES); + if (chunkIndexSizesByteBuffer == null) { + return false; + } + IntBuffer chunkIndexSizes = chunkIndexSizesByteBuffer.asIntBuffer(); + LongBuffer chunkIndexOffsets = + Assertions.checkNotNull(mediaFormat.getByteBuffer(MEDIA_FORMAT_KEY_CHUNK_INDEX_OFFSETS)) + .asLongBuffer(); + LongBuffer chunkIndexDurationsUs = + Assertions.checkNotNull(mediaFormat.getByteBuffer(MEDIA_FORMAT_KEY_CHUNK_INDEX_DURATIONS)) + .asLongBuffer(); + LongBuffer chunkIndexTimesUs = + Assertions.checkNotNull(mediaFormat.getByteBuffer(MEDIA_FORMAT_KEY_CHUNK_INDEX_TIMES)) + .asLongBuffer(); + int[] sizes = new int[chunkIndexSizes.remaining()]; + long[] offsets = new long[chunkIndexOffsets.remaining()]; + long[] durationsUs = new long[chunkIndexDurationsUs.remaining()]; + long[] timesUs = new long[chunkIndexTimesUs.remaining()]; + chunkIndexSizes.get(sizes); + chunkIndexOffsets.get(offsets); + chunkIndexDurationsUs.get(durationsUs); + chunkIndexTimesUs.get(timesUs); + lastChunkIndex = new ChunkIndex(sizes, offsets, durationsUs, timesUs); + extractorOutput.seekMap(lastChunkIndex); + return true; + } + + private void ensureSpaceForTrackIndex(int trackIndex) { + for (int i = trackOutputs.size(); i <= trackIndex; i++) { + trackOutputs.add(null); + trackFormats.add(null); + lastReceivedCryptoInfos.add(null); + lastOutputCryptoDatas.add(null); + } + } + + @Nullable + private CryptoData toExoPlayerCryptoData(int trackIndex, @Nullable CryptoInfo cryptoInfo) { + if (cryptoInfo == null) { + return null; + } + + @Nullable CryptoInfo lastReceivedCryptoInfo = lastReceivedCryptoInfos.get(trackIndex); + CryptoData cryptoDataToOutput; + // MediaParser keeps identity and value equality aligned for efficient comparison. + if (lastReceivedCryptoInfo == cryptoInfo) { + // They match, we can reuse the last one we created. + cryptoDataToOutput = Assertions.checkNotNull(lastOutputCryptoDatas.get(trackIndex)); + } else { + // They don't match, we create a new CryptoData. + + // TODO: Access pattern encryption info directly once the Android SDK makes it visible. + // See [Internal ref: b/154248283]. + int encryptedBlocks; + int clearBlocks; + try { + Matcher matcher = REGEX_CRYPTO_INFO_PATTERN.matcher(cryptoInfo.toString()); + matcher.find(); + encryptedBlocks = Integer.parseInt(Util.castNonNull(matcher.group(1))); + clearBlocks = Integer.parseInt(Util.castNonNull(matcher.group(2))); + } catch (RuntimeException e) { + // Should never happen. + Log.e(TAG, "Unexpected error while parsing CryptoInfo: " + cryptoInfo, e); + // Assume no-pattern encryption. + encryptedBlocks = 0; + clearBlocks = 0; + } + cryptoDataToOutput = + new CryptoData(cryptoInfo.mode, cryptoInfo.key, encryptedBlocks, clearBlocks); + lastReceivedCryptoInfos.set(trackIndex, cryptoInfo); + lastOutputCryptoDatas.set(trackIndex, cryptoDataToOutput); + } + return cryptoDataToOutput; + } + + private void maybeEndTracks() { + if (!tracksFoundCalled || tracksEnded) { + return; + } + int size = trackOutputs.size(); + for (int i = 0; i < size; i++) { + if (trackOutputs.get(i) == null) { + return; + } + } + extractorOutput.endTracks(); + tracksEnded = true; + } + + private static int toTrackTypeConstant(@Nullable String string) { + if (string == null) { + return C.TRACK_TYPE_UNKNOWN; + } + switch (string) { + case "audio": + return C.TRACK_TYPE_AUDIO; + case "video": + return C.TRACK_TYPE_VIDEO; + case "text": + return C.TRACK_TYPE_TEXT; + case "metadata": + return C.TRACK_TYPE_METADATA; + case "unknown": + return C.TRACK_TYPE_UNKNOWN; + default: + // Must be a MIME type. + return MimeTypes.getTrackType(string); + } + } + + private Format toExoPlayerFormat(TrackData trackData) { + // TODO: Consider adding support for the following: + // format.id + // format.stereoMode + // format.projectionData + MediaFormat mediaFormat = trackData.mediaFormat; + @Nullable String mediaFormatMimeType = mediaFormat.getString(MediaFormat.KEY_MIME); + int mediaFormatAccessibilityChannel = + mediaFormat.getInteger( + MediaFormat.KEY_CAPTION_SERVICE_NUMBER, /* defaultValue= */ Format.NO_VALUE); + Format.Builder formatBuilder = + new Format.Builder() + .setDrmInitData( + toExoPlayerDrmInitData( + mediaFormat.getString("crypto-mode-fourcc"), trackData.drmInitData)) + .setContainerMimeType(containerMimeType) + .setPeakBitrate( + mediaFormat.getInteger( + MediaFormat.KEY_BIT_RATE, /* defaultValue= */ Format.NO_VALUE)) + .setChannelCount( + mediaFormat.getInteger( + MediaFormat.KEY_CHANNEL_COUNT, /* defaultValue= */ Format.NO_VALUE)) + .setColorInfo(getColorInfo(mediaFormat)) + .setSampleMimeType(mediaFormatMimeType) + .setCodecs(mediaFormat.getString(MediaFormat.KEY_CODECS_STRING)) + .setFrameRate( + mediaFormat.getFloat( + MediaFormat.KEY_FRAME_RATE, /* defaultValue= */ Format.NO_VALUE)) + .setWidth( + mediaFormat.getInteger(MediaFormat.KEY_WIDTH, /* defaultValue= */ Format.NO_VALUE)) + .setHeight( + mediaFormat.getInteger(MediaFormat.KEY_HEIGHT, /* defaultValue= */ Format.NO_VALUE)) + .setInitializationData(getInitializationData(mediaFormat)) + .setLanguage(mediaFormat.getString(MediaFormat.KEY_LANGUAGE)) + .setMaxInputSize( + mediaFormat.getInteger( + MediaFormat.KEY_MAX_INPUT_SIZE, /* defaultValue= */ Format.NO_VALUE)) + .setPcmEncoding( + mediaFormat.getInteger("exo-pcm-encoding", /* defaultValue= */ Format.NO_VALUE)) + .setRotationDegrees( + mediaFormat.getInteger(MediaFormat.KEY_ROTATION, /* defaultValue= */ 0)) + .setSampleRate( + mediaFormat.getInteger( + MediaFormat.KEY_SAMPLE_RATE, /* defaultValue= */ Format.NO_VALUE)) + .setSelectionFlags(getSelectionFlags(mediaFormat)) + .setEncoderDelay( + mediaFormat.getInteger(MediaFormat.KEY_ENCODER_DELAY, /* defaultValue= */ 0)) + .setEncoderPadding( + mediaFormat.getInteger(MediaFormat.KEY_ENCODER_PADDING, /* defaultValue= */ 0)) + .setPixelWidthHeightRatio( + mediaFormat.getFloat("pixel-width-height-ratio-float", /* defaultValue= */ 1f)) + .setSubsampleOffsetUs( + mediaFormat.getLong( + "subsample-offset-us-long", /* defaultValue= */ Format.OFFSET_SAMPLE_RELATIVE)) + .setAccessibilityChannel(mediaFormatAccessibilityChannel); + for (int i = 0; i < muxedCaptionFormats.size(); i++) { + Format muxedCaptionFormat = muxedCaptionFormats.get(i); + if (Util.areEqual(muxedCaptionFormat.sampleMimeType, mediaFormatMimeType) + && muxedCaptionFormat.accessibilityChannel == mediaFormatAccessibilityChannel) { + // The track's format matches this muxedCaptionFormat, so we apply the manifest format + // information to the track. + formatBuilder + .setLanguage(muxedCaptionFormat.language) + .setRoleFlags(muxedCaptionFormat.roleFlags) + .setSelectionFlags(muxedCaptionFormat.selectionFlags) + .setLabel(muxedCaptionFormat.label) + .setMetadata(muxedCaptionFormat.metadata); + break; + } + } + return formatBuilder.build(); + } + + @Nullable + private static DrmInitData toExoPlayerDrmInitData( + @Nullable String schemeType, @Nullable android.media.DrmInitData drmInitData) { + if (drmInitData == null) { + return null; + } + SchemeData[] schemeDatas = new SchemeData[drmInitData.getSchemeInitDataCount()]; + for (int i = 0; i < schemeDatas.length; i++) { + SchemeInitData schemeInitData = drmInitData.getSchemeInitDataAt(i); + schemeDatas[i] = + new SchemeData(schemeInitData.uuid, schemeInitData.mimeType, schemeInitData.data); + } + return new DrmInitData(schemeType, schemeDatas); + } + + @SelectionFlags + private static int getSelectionFlags(MediaFormat mediaFormat) { + int selectionFlags = 0; + selectionFlags |= + getFlag( + mediaFormat, + /* key= */ MediaFormat.KEY_IS_AUTOSELECT, + /* returnValueIfPresent= */ C.SELECTION_FLAG_AUTOSELECT); + selectionFlags |= + getFlag( + mediaFormat, + /* key= */ MediaFormat.KEY_IS_DEFAULT, + /* returnValueIfPresent= */ C.SELECTION_FLAG_DEFAULT); + selectionFlags |= + getFlag( + mediaFormat, + /* key= */ MediaFormat.KEY_IS_FORCED_SUBTITLE, + /* returnValueIfPresent= */ C.SELECTION_FLAG_FORCED); + return selectionFlags; + } + + private static int getFlag(MediaFormat mediaFormat, String key, int returnValueIfPresent) { + return mediaFormat.getInteger(key, /* defaultValue= */ 0) != 0 ? returnValueIfPresent : 0; + } + + private static List getInitializationData(MediaFormat mediaFormat) { + ArrayList initData = new ArrayList<>(); + int i = 0; + while (true) { + @Nullable ByteBuffer byteBuffer = mediaFormat.getByteBuffer("csd-" + i++); + if (byteBuffer == null) { + break; + } + initData.add(getArray(byteBuffer)); + } + return initData; + } + + @Nullable + private static ColorInfo getColorInfo(MediaFormat mediaFormat) { + @Nullable + ByteBuffer hdrStaticInfoByteBuffer = mediaFormat.getByteBuffer(MediaFormat.KEY_HDR_STATIC_INFO); + @Nullable + byte[] hdrStaticInfo = + hdrStaticInfoByteBuffer != null ? getArray(hdrStaticInfoByteBuffer) : null; + int colorTransfer = + mediaFormat.getInteger(MediaFormat.KEY_COLOR_TRANSFER, /* defaultValue= */ Format.NO_VALUE); + int colorRange = + mediaFormat.getInteger(MediaFormat.KEY_COLOR_RANGE, /* defaultValue= */ Format.NO_VALUE); + int colorStandard = + mediaFormat.getInteger(MediaFormat.KEY_COLOR_STANDARD, /* defaultValue= */ Format.NO_VALUE); + + if (hdrStaticInfo != null + || colorTransfer != Format.NO_VALUE + || colorRange != Format.NO_VALUE + || colorStandard != Format.NO_VALUE) { + return new ColorInfo(colorStandard, colorRange, colorTransfer, hdrStaticInfo); + } + return null; + } + + private static byte[] getArray(ByteBuffer byteBuffer) { + byte[] array = new byte[byteBuffer.remaining()]; + byteBuffer.get(array); + return array; + } + + private static String getMimeType(String parserName) { + switch (parserName) { + case PARSER_NAME_MATROSKA: + return MimeTypes.VIDEO_WEBM; + case PARSER_NAME_FMP4: + case PARSER_NAME_MP4: + return MimeTypes.VIDEO_MP4; + case PARSER_NAME_MP3: + return MimeTypes.AUDIO_MPEG; + case PARSER_NAME_ADTS: + return MimeTypes.AUDIO_AAC; + case PARSER_NAME_AC3: + return MimeTypes.AUDIO_AC3; + case PARSER_NAME_TS: + return MimeTypes.VIDEO_MP2T; + case PARSER_NAME_FLV: + return MimeTypes.VIDEO_FLV; + case PARSER_NAME_OGG: + return MimeTypes.AUDIO_OGG; + case PARSER_NAME_PS: + return MimeTypes.VIDEO_PS; + case PARSER_NAME_WAV: + return MimeTypes.AUDIO_RAW; + case PARSER_NAME_AMR: + return MimeTypes.AUDIO_AMR; + case PARSER_NAME_AC4: + return MimeTypes.AUDIO_AC4; + case PARSER_NAME_FLAC: + return MimeTypes.AUDIO_FLAC; + default: + throw new IllegalArgumentException("Illegal parser name: " + parserName); + } + } + + private static final class SeekMapAdapter implements SeekMap { + + private final MediaParser.SeekMap adaptedSeekMap; + + public SeekMapAdapter(MediaParser.SeekMap adaptedSeekMap) { + this.adaptedSeekMap = adaptedSeekMap; + } + + @Override + public boolean isSeekable() { + return adaptedSeekMap.isSeekable(); + } + + @Override + public long getDurationUs() { + long durationMicros = adaptedSeekMap.getDurationMicros(); + return durationMicros != MediaParser.SeekMap.UNKNOWN_DURATION ? durationMicros : C.TIME_UNSET; + } + + @Override + @SuppressWarnings("ReferenceEquality") + public SeekPoints getSeekPoints(long timeUs) { + Pair seekPoints = + adaptedSeekMap.getSeekPoints(timeUs); + SeekPoints exoPlayerSeekPoints; + if (seekPoints.first == seekPoints.second) { + exoPlayerSeekPoints = new SeekPoints(asExoPlayerSeekPoint(seekPoints.first)); + } else { + exoPlayerSeekPoints = + new SeekPoints( + asExoPlayerSeekPoint(seekPoints.first), asExoPlayerSeekPoint(seekPoints.second)); + } + return exoPlayerSeekPoints; + } + + private static SeekPoint asExoPlayerSeekPoint(MediaParser.SeekPoint seekPoint) { + return new SeekPoint(seekPoint.timeMicros, seekPoint.position); + } + } + + private static final class DataReaderAdapter implements DataReader { + + @Nullable public MediaParser.InputReader input; + + @Override + public int read(byte[] target, int offset, int length) throws IOException { + return Util.castNonNull(input).read(target, offset, length); + } + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/analytics/DefaultAnalyticsListener.java b/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/package-info.java similarity index 62% rename from library/core/src/main/java/com/google/android/exoplayer2/analytics/DefaultAnalyticsListener.java rename to library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/package-info.java index d487a8aa99b..3eedf0c7a4e 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/analytics/DefaultAnalyticsListener.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/source/mediaparser/package-info.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2018 The Android Open Source Project + * Copyright (C) 2020 The Android Open Source Project * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -13,11 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.google.android.exoplayer2.analytics; +@NonNullApi +package com.google.android.exoplayer2.source.mediaparser; -/** - * @deprecated Use {@link AnalyticsListener} directly for selective overrides as all methods are - * implemented as no-op default methods. - */ -@Deprecated -public abstract class DefaultAnalyticsListener implements AnalyticsListener {} +import com.google.android.exoplayer2.util.NonNullApi; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/TextRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/text/TextRenderer.java index 76c13600457..e1c9e7cebe4 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/text/TextRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/text/TextRenderer.java @@ -133,11 +133,11 @@ public String getName() { public int supportsFormat(Format format) { if (decoderFactory.supportsFormat(format)) { return RendererCapabilities.create( - format.exoMediaCryptoType == null ? FORMAT_HANDLED : FORMAT_UNSUPPORTED_DRM); + format.exoMediaCryptoType == null ? C.FORMAT_HANDLED : C.FORMAT_UNSUPPORTED_DRM); } else if (MimeTypes.isText(format.sampleMimeType)) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_SUBTYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_SUBTYPE); } else { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/cea/Cea708Decoder.java b/library/core/src/main/java/com/google/android/exoplayer2/text/cea/Cea708Decoder.java index 56dd4ebef27..dd25d74e105 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/text/cea/Cea708Decoder.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/text/cea/Cea708Decoder.java @@ -145,6 +145,7 @@ public final class Cea708Decoder extends CeaDecoder { private final ParsableByteArray ccData; private final ParsableBitArray serviceBlockPacket; + private int previousSequenceNumber; // TODO: Use isWideAspectRatio in decoding. @SuppressWarnings({"unused", "FieldCanBeLocal"}) private final boolean isWideAspectRatio; @@ -162,6 +163,7 @@ public final class Cea708Decoder extends CeaDecoder { public Cea708Decoder(int accessibilityChannel, @Nullable List initializationData) { ccData = new ParsableByteArray(); serviceBlockPacket = new ParsableBitArray(); + previousSequenceNumber = C.INDEX_UNSET; selectedServiceNumber = accessibilityChannel == Format.NO_VALUE ? 1 : accessibilityChannel; isWideAspectRatio = initializationData != null @@ -231,6 +233,18 @@ protected void decode(SubtitleInputBuffer inputBuffer) { finalizeCurrentPacket(); int sequenceNumber = (ccData1 & 0xC0) >> 6; // first 2 bits + if (previousSequenceNumber != C.INDEX_UNSET + && sequenceNumber != (previousSequenceNumber + 1) % 4) { + resetCueBuilders(); + Log.w( + TAG, + "Sequence number discontinuity. previous=" + + previousSequenceNumber + + " current=" + + sequenceNumber); + } + previousSequenceNumber = sequenceNumber; + int packetSize = ccData1 & 0x3F; // last 6 bits if (packetSize == 0) { packetSize = 64; @@ -270,10 +284,18 @@ private void finalizeCurrentPacket() { @RequiresNonNull("currentDtvCcPacket") private void processCurrentPacket() { if (currentDtvCcPacket.currentIndex != (currentDtvCcPacket.packetSize * 2 - 1)) { - Log.w(TAG, "DtvCcPacket ended prematurely; size is " + (currentDtvCcPacket.packetSize * 2 - 1) - + ", but current index is " + currentDtvCcPacket.currentIndex + " (sequence number " - + currentDtvCcPacket.sequenceNumber + "); ignoring packet"); - return; + Log.d( + TAG, + "DtvCcPacket ended prematurely; size is " + + (currentDtvCcPacket.packetSize * 2 - 1) + + ", but current index is " + + currentDtvCcPacket.currentIndex + + " (sequence number " + + currentDtvCcPacket.sequenceNumber + + ");"); + // We've received cc_type=0x03 (packet start) before receiving packetSize byte pairs of data. + // This might indicate a byte pair has been lost, but we'll still attempt to process the data + // we have received. } serviceBlockPacket.reset(currentDtvCcPacket.packetData, currentDtvCcPacket.currentIndex); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/pgs/PgsDecoder.java b/library/core/src/main/java/com/google/android/exoplayer2/text/pgs/PgsDecoder.java index 163cc6b12bf..47dd6e2fdf5 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/text/pgs/PgsDecoder.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/text/pgs/PgsDecoder.java @@ -134,7 +134,7 @@ public CueBuilder() { private void parsePaletteSection(ParsableByteArray buffer, int sectionLength) { if ((sectionLength % 5) != 2) { - // Section must be two bytes followed by a whole number of (index, y, cb, cr, a) entries. + // Section must be two bytes then a whole number of (index, Y, Cr, Cb, alpha) entries. return; } buffer.skipBytes(2); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/ssa/SsaDecoder.java b/library/core/src/main/java/com/google/android/exoplayer2/text/ssa/SsaDecoder.java index f44db4924f8..b8e047dbcb9 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/text/ssa/SsaDecoder.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/text/ssa/SsaDecoder.java @@ -19,6 +19,8 @@ import static com.google.android.exoplayer2.util.Util.castNonNull; import android.text.Layout; +import android.text.SpannableString; +import android.text.style.ForegroundColorSpan; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.text.Cue; @@ -301,7 +303,18 @@ private static Cue createCue( SsaStyle.Overrides styleOverrides, float screenWidth, float screenHeight) { - Cue.Builder cue = new Cue.Builder().setText(text); + SpannableString spannableText = new SpannableString(text); + Cue.Builder cue = new Cue.Builder().setText(spannableText); + + if (style != null) { + if (style.primaryColor != null) { + spannableText.setSpan( + new ForegroundColorSpan(style.primaryColor), + /* start= */ 0, + /* end= */ spannableText.length(), + SpannableString.SPAN_EXCLUSIVE_EXCLUSIVE); + } + } @SsaStyle.SsaAlignment int alignment; if (styleOverrides.alignment != SsaStyle.SSA_ALIGNMENT_UNKNOWN) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/ssa/SsaStyle.java b/library/core/src/main/java/com/google/android/exoplayer2/text/ssa/SsaStyle.java index 0cba3390346..bd378cccec7 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/text/ssa/SsaStyle.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/text/ssa/SsaStyle.java @@ -17,16 +17,20 @@ package com.google.android.exoplayer2.text.ssa; import static com.google.android.exoplayer2.text.ssa.SsaDecoder.STYLE_LINE_PREFIX; +import static com.google.android.exoplayer2.util.Assertions.checkArgument; import static java.lang.annotation.RetentionPolicy.SOURCE; +import android.graphics.Color; import android.graphics.PointF; import android.text.TextUtils; +import androidx.annotation.ColorInt; import androidx.annotation.IntDef; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.Util; +import com.google.common.primitives.Ints; import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.util.regex.Matcher; @@ -85,15 +89,18 @@ public final String name; @SsaAlignment public final int alignment; + @Nullable @ColorInt public final Integer primaryColor; - private SsaStyle(String name, @SsaAlignment int alignment) { + private SsaStyle( + String name, @SsaAlignment int alignment, @Nullable @ColorInt Integer primaryColor) { this.name = name; this.alignment = alignment; + this.primaryColor = primaryColor; } @Nullable public static SsaStyle fromStyleLine(String styleLine, Format format) { - Assertions.checkArgument(styleLine.startsWith(STYLE_LINE_PREFIX)); + checkArgument(styleLine.startsWith(STYLE_LINE_PREFIX)); String[] styleValues = TextUtils.split(styleLine.substring(STYLE_LINE_PREFIX.length()), ","); if (styleValues.length != format.length) { Log.w( @@ -105,7 +112,9 @@ public static SsaStyle fromStyleLine(String styleLine, Format format) { } try { return new SsaStyle( - styleValues[format.nameIndex].trim(), parseAlignment(styleValues[format.alignmentIndex])); + styleValues[format.nameIndex].trim(), + parseAlignment(styleValues[format.alignmentIndex].trim()), + parseColor(styleValues[format.primaryColorIndex].trim())); } catch (RuntimeException e) { Log.w(TAG, "Skipping malformed 'Style:' line: '" + styleLine + "'", e); return null; @@ -144,6 +153,44 @@ private static boolean isValidAlignment(@SsaAlignment int alignment) { } } + /** + * Parses a SSA V4+ color expression. + * + *

    A SSA V4+ color can be represented in hex {@code ("&HAABBGGRR")} or in 64-bit decimal format + * (byte order AABBGGRR). In both cases the alpha channel's value needs to be inverted because in + * SSA the 0xFF alpha value means transparent and 0x00 means opaque which is the opposite from the + * Android {@link ColorInt} representation. + * + * @param ssaColorExpression A SSA V4+ color expression. + * @return The parsed color value, or null if parsing failed. + */ + @Nullable + @ColorInt + public static Integer parseColor(String ssaColorExpression) { + // We use a long because the value is an unsigned 32-bit number, so can be larger than + // Integer.MAX_VALUE. + long abgr; + try { + abgr = + ssaColorExpression.startsWith("&H") + // Parse color from hex format (&HAABBGGRR). + ? Long.parseLong(ssaColorExpression.substring(2), /* radix= */ 16) + // Parse color from decimal format (bytes order AABBGGRR). + : Long.parseLong(ssaColorExpression); + // Ensure only the bottom 4 bytes of abgr are set. + checkArgument(abgr <= 0xFFFFFFFFL); + } catch (IllegalArgumentException e) { + Log.w(TAG, "Failed to parse color expression: '" + ssaColorExpression + "'", e); + return null; + } + // Convert ABGR to ARGB. + int a = Ints.checkedCast(((abgr >> 24) & 0xFF) ^ 0xFF); // Flip alpha. + int b = Ints.checkedCast((abgr >> 16) & 0xFF); + int g = Ints.checkedCast((abgr >> 8) & 0xFF); + int r = Ints.checkedCast(abgr & 0xFF); + return Color.argb(a, r, g, b); + } + /** * Represents a {@code Format:} line from the {@code [V4+ Styles]} section * @@ -154,11 +201,13 @@ private static boolean isValidAlignment(@SsaAlignment int alignment) { public final int nameIndex; public final int alignmentIndex; + public final int primaryColorIndex; public final int length; - private Format(int nameIndex, int alignmentIndex, int length) { + private Format(int nameIndex, int alignmentIndex, int primaryColorIndex, int length) { this.nameIndex = nameIndex; this.alignmentIndex = alignmentIndex; + this.primaryColorIndex = primaryColorIndex; this.length = length; } @@ -171,6 +220,7 @@ private Format(int nameIndex, int alignmentIndex, int length) { public static Format fromFormatLine(String styleFormatLine) { int nameIndex = C.INDEX_UNSET; int alignmentIndex = C.INDEX_UNSET; + int primaryColorIndex = C.INDEX_UNSET; String[] keys = TextUtils.split(styleFormatLine.substring(SsaDecoder.FORMAT_LINE_PREFIX.length()), ","); for (int i = 0; i < keys.length; i++) { @@ -181,9 +231,14 @@ public static Format fromFormatLine(String styleFormatLine) { case "alignment": alignmentIndex = i; break; + case "primarycolour": + primaryColorIndex = i; + break; } } - return nameIndex != C.INDEX_UNSET ? new Format(nameIndex, alignmentIndex, keys.length) : null; + return nameIndex != C.INDEX_UNSET + ? new Format(nameIndex, alignmentIndex, primaryColorIndex, keys.length) + : null; } } @@ -237,8 +292,7 @@ public static Overrides parseFromDialogue(String text) { // Ignore invalid \pos() or \move() function. } try { - @SsaAlignment - int parsedAlignment = parseAlignmentOverride(braceContents); + @SsaAlignment int parsedAlignment = parseAlignmentOverride(braceContents); if (parsedAlignment != SSA_ALIGNMENT_UNKNOWN) { alignment = parsedAlignment; } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/text/tx3g/Tx3gDecoder.java b/library/core/src/main/java/com/google/android/exoplayer2/text/tx3g/Tx3gDecoder.java index 907607f859f..ad1abdc7bc7 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/text/tx3g/Tx3gDecoder.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/text/tx3g/Tx3gDecoder.java @@ -72,12 +72,12 @@ public final class Tx3gDecoder extends SimpleSubtitleDecoder { private final ParsableByteArray parsableByteArray; - private boolean customVerticalPlacement; - private int defaultFontFace; - private int defaultColorRgba; - private String defaultFontFamily; - private float defaultVerticalPlacement; - private int calculatedVideoTrackHeight; + private final boolean customVerticalPlacement; + private final int defaultFontFace; + private final int defaultColorRgba; + private final String defaultFontFamily; + private final float defaultVerticalPlacement; + private final int calculatedVideoTrackHeight; /** * Sets up a new {@link Tx3gDecoder} with default values. @@ -88,7 +88,7 @@ public Tx3gDecoder(List initializationData) { super("Tx3gDecoder"); parsableByteArray = new ParsableByteArray(); - if (initializationData != null && initializationData.size() == 1 + if (initializationData.size() == 1 && (initializationData.get(0).length == 48 || initializationData.get(0).length == 53)) { byte[] initializationBytes = initializationData.get(0); defaultFontFace = initializationBytes[24]; @@ -105,8 +105,9 @@ public Tx3gDecoder(List initializationData) { if (customVerticalPlacement) { int requestedVerticalPlacement = ((initializationBytes[10] & 0xFF) << 8) | (initializationBytes[11] & 0xFF); - defaultVerticalPlacement = (float) requestedVerticalPlacement / calculatedVideoTrackHeight; - defaultVerticalPlacement = Util.constrainValue(defaultVerticalPlacement, 0.0f, 0.95f); + defaultVerticalPlacement = + Util.constrainValue( + (float) requestedVerticalPlacement / calculatedVideoTrackHeight, 0.0f, 0.95f); } else { defaultVerticalPlacement = DEFAULT_VERTICAL_PLACEMENT; } @@ -116,6 +117,7 @@ public Tx3gDecoder(List initializationData) { defaultFontFamily = DEFAULT_FONT_FAMILY; customVerticalPlacement = false; defaultVerticalPlacement = DEFAULT_VERTICAL_PLACEMENT; + calculatedVideoTrackHeight = C.LENGTH_UNSET; } } @@ -133,8 +135,7 @@ protected Subtitle decode(byte[] bytes, int length, boolean reset) SPAN_PRIORITY_LOW); attachColor(cueText, defaultColorRgba, DEFAULT_COLOR, 0, cueText.length(), SPAN_PRIORITY_LOW); - attachFontFamily(cueText, defaultFontFamily, DEFAULT_FONT_FAMILY, 0, cueText.length(), - SPAN_PRIORITY_LOW); + attachFontFamily(cueText, defaultFontFamily, 0, cueText.length()); float verticalPlacement = defaultVerticalPlacement; // Find and attach additional styles. while (parsableByteArray.bytesLeft() >= SIZE_ATOM_HEADER) { @@ -237,11 +238,14 @@ private static void attachColor(SpannableStringBuilder cueText, int colorRgba, } @SuppressWarnings("ReferenceEquality") - private static void attachFontFamily(SpannableStringBuilder cueText, String fontFamily, - String defaultFontFamily, int start, int end, int spanPriority) { - if (fontFamily != defaultFontFamily) { - cueText.setSpan(new TypefaceSpan(fontFamily), start, end, - Spanned.SPAN_EXCLUSIVE_EXCLUSIVE | spanPriority); + private static void attachFontFamily( + SpannableStringBuilder cueText, String fontFamily, int start, int end) { + if (fontFamily != Tx3gDecoder.DEFAULT_FONT_FAMILY) { + cueText.setSpan( + new TypefaceSpan(fontFamily), + start, + end, + Spanned.SPAN_EXCLUSIVE_EXCLUSIVE | Tx3gDecoder.SPAN_PRIORITY_LOW); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/AdaptiveTrackSelection.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/AdaptiveTrackSelection.java index 3173188cac3..bd2e18ad922 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/AdaptiveTrackSelection.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/AdaptiveTrackSelection.java @@ -15,32 +15,36 @@ */ package com.google.android.exoplayer2.trackselection; -import static java.lang.Math.max; import androidx.annotation.CallSuper; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.chunk.MediaChunk; import com.google.android.exoplayer2.source.chunk.MediaChunkIterator; import com.google.android.exoplayer2.upstream.BandwidthMeter; -import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Clock; import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; +import com.google.common.collect.Multimap; +import com.google.common.collect.MultimapBuilder; import java.util.ArrayList; +import java.util.Arrays; import java.util.List; import org.checkerframework.checker.nullness.compatqual.NullableType; /** - * A bandwidth based adaptive {@link TrackSelection}, whose selected track is updated to be the one - * of highest quality given the current network conditions and the state of the buffer. + * A bandwidth based adaptive {@link ExoTrackSelection}, whose selected track is updated to be the + * one of highest quality given the current network conditions and the state of the buffer. */ public class AdaptiveTrackSelection extends BaseTrackSelection { /** Factory for {@link AdaptiveTrackSelection} instances. */ - public static class Factory implements TrackSelection.Factory { + public static class Factory implements ExoTrackSelection.Factory { private final int minDurationForQualityIncreaseMs; private final int maxDurationForQualityDecreaseMs; @@ -128,50 +132,28 @@ public Factory( } @Override - public final @NullableType TrackSelection[] createTrackSelections( - @NullableType Definition[] definitions, BandwidthMeter bandwidthMeter) { - TrackSelection[] selections = new TrackSelection[definitions.length]; - int totalFixedBandwidth = 0; - for (int i = 0; i < definitions.length; i++) { - Definition definition = definitions[i]; - if (definition != null && definition.tracks.length == 1) { - // Make fixed selections first to know their total bandwidth. - selections[i] = - new FixedTrackSelection( - definition.group, definition.tracks[0], definition.reason, definition.data); - int trackBitrate = definition.group.getFormat(definition.tracks[0]).bitrate; - if (trackBitrate != Format.NO_VALUE) { - totalFixedBandwidth += trackBitrate; - } - } - } - List adaptiveSelections = new ArrayList<>(); + public final @NullableType ExoTrackSelection[] createTrackSelections( + @NullableType Definition[] definitions, + BandwidthMeter bandwidthMeter, + MediaPeriodId mediaPeriodId, + Timeline timeline) { + ImmutableList> adaptationCheckpoints = + getAdaptationCheckpoints(definitions); + ExoTrackSelection[] selections = new ExoTrackSelection[definitions.length]; for (int i = 0; i < definitions.length; i++) { - Definition definition = definitions[i]; - if (definition != null && definition.tracks.length > 1) { - AdaptiveTrackSelection adaptiveSelection = - createAdaptiveTrackSelection( - definition.group, bandwidthMeter, definition.tracks, totalFixedBandwidth); - adaptiveSelections.add(adaptiveSelection); - selections[i] = adaptiveSelection; - } - } - if (adaptiveSelections.size() > 1) { - long[][] adaptiveTrackBitrates = new long[adaptiveSelections.size()][]; - for (int i = 0; i < adaptiveSelections.size(); i++) { - AdaptiveTrackSelection adaptiveSelection = adaptiveSelections.get(i); - adaptiveTrackBitrates[i] = new long[adaptiveSelection.length()]; - for (int j = 0; j < adaptiveSelection.length(); j++) { - adaptiveTrackBitrates[i][j] = - adaptiveSelection.getFormat(adaptiveSelection.length() - j - 1).bitrate; - } - } - long[][][] bandwidthCheckpoints = getAllocationCheckpoints(adaptiveTrackBitrates); - for (int i = 0; i < adaptiveSelections.size(); i++) { - adaptiveSelections - .get(i) - .experimentalSetBandwidthAllocationCheckpoints(bandwidthCheckpoints[i]); + @Nullable Definition definition = definitions[i]; + if (definition == null || definition.tracks.length == 0) { + continue; } + selections[i] = + definition.tracks.length == 1 + ? new FixedTrackSelection( + definition.group, definition.tracks[0], definition.reason, definition.data) + : createAdaptiveTrackSelection( + definition.group, + bandwidthMeter, + definition.tracks, + adaptationCheckpoints.get(i)); } return selections; } @@ -182,23 +164,25 @@ public Factory( * @param group The {@link TrackGroup}. * @param bandwidthMeter A {@link BandwidthMeter} which can be used to select tracks. * @param tracks The indices of the selected tracks in the track group. - * @param totalFixedTrackBandwidth The total bandwidth used by all non-adaptive tracks, in bits - * per second. + * @param adaptationCheckpoints The {@link AdaptationCheckpoint checkpoints} that can be used to + * calculate available bandwidth for this selection. * @return An {@link AdaptiveTrackSelection} for the specified tracks. */ protected AdaptiveTrackSelection createAdaptiveTrackSelection( TrackGroup group, BandwidthMeter bandwidthMeter, int[] tracks, - int totalFixedTrackBandwidth) { + ImmutableList adaptationCheckpoints) { return new AdaptiveTrackSelection( group, tracks, - new DefaultBandwidthProvider(bandwidthMeter, bandwidthFraction, totalFixedTrackBandwidth), + bandwidthMeter, minDurationForQualityIncreaseMs, maxDurationForQualityDecreaseMs, minDurationToRetainAfterDiscardMs, + bandwidthFraction, bufferedFractionToLiveEdgeForQualityIncrease, + adaptationCheckpoints, clock); } } @@ -211,11 +195,13 @@ protected AdaptiveTrackSelection createAdaptiveTrackSelection( private static final long MIN_TIME_BETWEEN_BUFFER_REEVALUTATION_MS = 1000; - private final BandwidthProvider bandwidthProvider; + private final BandwidthMeter bandwidthMeter; private final long minDurationForQualityIncreaseUs; private final long maxDurationForQualityDecreaseUs; private final long minDurationToRetainAfterDiscardUs; + private final float bandwidthFraction; private final float bufferedFractionToLiveEdgeForQualityIncrease; + private final ImmutableList adaptationCheckpoints; private final Clock clock; private float playbackSpeed; @@ -230,18 +216,17 @@ protected AdaptiveTrackSelection createAdaptiveTrackSelection( * empty. May be in any order. * @param bandwidthMeter Provides an estimate of the currently available bandwidth. */ - public AdaptiveTrackSelection(TrackGroup group, int[] tracks, - BandwidthMeter bandwidthMeter) { + public AdaptiveTrackSelection(TrackGroup group, int[] tracks, BandwidthMeter bandwidthMeter) { this( group, tracks, bandwidthMeter, - /* reservedBandwidth= */ 0, DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, DEFAULT_BANDWIDTH_FRACTION, DEFAULT_BUFFERED_FRACTION_TO_LIVE_EDGE_FOR_QUALITY_INCREASE, + /* adaptationCheckpoints= */ ImmutableList.of(), Clock.DEFAULT); } @@ -250,8 +235,6 @@ public AdaptiveTrackSelection(TrackGroup group, int[] tracks, * @param tracks The indices of the selected tracks within the {@link TrackGroup}. Must not be * empty. May be in any order. * @param bandwidthMeter Provides an estimate of the currently available bandwidth. - * @param reservedBandwidth The reserved bandwidth, which shouldn't be considered available for - * use, in bits per second. * @param minDurationForQualityIncreaseMs The minimum duration of buffered data required for the * selected track to switch to one of higher quality. * @param maxDurationForQualityDecreaseMs The maximum duration of buffered data required for the @@ -269,62 +252,36 @@ public AdaptiveTrackSelection(TrackGroup group, int[] tracks, * when the playback position is closer to the live edge than {@code * minDurationForQualityIncreaseMs}, which would otherwise prevent switching to a higher * quality from happening. + * @param adaptationCheckpoints The {@link AdaptationCheckpoint checkpoints} that can be used to + * calculate available bandwidth for this selection. + * @param clock The {@link Clock}. */ - public AdaptiveTrackSelection( + protected AdaptiveTrackSelection( TrackGroup group, int[] tracks, BandwidthMeter bandwidthMeter, - long reservedBandwidth, long minDurationForQualityIncreaseMs, long maxDurationForQualityDecreaseMs, long minDurationToRetainAfterDiscardMs, float bandwidthFraction, float bufferedFractionToLiveEdgeForQualityIncrease, - Clock clock) { - this( - group, - tracks, - new DefaultBandwidthProvider(bandwidthMeter, bandwidthFraction, reservedBandwidth), - minDurationForQualityIncreaseMs, - maxDurationForQualityDecreaseMs, - minDurationToRetainAfterDiscardMs, - bufferedFractionToLiveEdgeForQualityIncrease, - clock); - } - - private AdaptiveTrackSelection( - TrackGroup group, - int[] tracks, - BandwidthProvider bandwidthProvider, - long minDurationForQualityIncreaseMs, - long maxDurationForQualityDecreaseMs, - long minDurationToRetainAfterDiscardMs, - float bufferedFractionToLiveEdgeForQualityIncrease, + List adaptationCheckpoints, Clock clock) { super(group, tracks); - this.bandwidthProvider = bandwidthProvider; + this.bandwidthMeter = bandwidthMeter; this.minDurationForQualityIncreaseUs = minDurationForQualityIncreaseMs * 1000L; this.maxDurationForQualityDecreaseUs = maxDurationForQualityDecreaseMs * 1000L; this.minDurationToRetainAfterDiscardUs = minDurationToRetainAfterDiscardMs * 1000L; + this.bandwidthFraction = bandwidthFraction; this.bufferedFractionToLiveEdgeForQualityIncrease = bufferedFractionToLiveEdgeForQualityIncrease; + this.adaptationCheckpoints = ImmutableList.copyOf(adaptationCheckpoints); this.clock = clock; playbackSpeed = 1f; reason = C.SELECTION_REASON_UNKNOWN; lastBufferEvaluationMs = C.TIME_UNSET; } - /** - * Sets checkpoints to determine the allocation bandwidth based on the total bandwidth. - * - * @param allocationCheckpoints List of checkpoints. Each element must be a long[2], with [0] - * being the total bandwidth and [1] being the allocated bandwidth. - */ - public void experimentalSetBandwidthAllocationCheckpoints(long[][] allocationCheckpoints) { - ((DefaultBandwidthProvider) bandwidthProvider) - .experimentalSetBandwidthAllocationCheckpoints(allocationCheckpoints); - } - @CallSuper @Override public void enable() { @@ -456,7 +413,7 @@ public int evaluateQueueSize(long playbackPositionUs, List * @param format The {@link Format} of the candidate track. * @param trackBitrate The estimated bitrate of the track. May differ from {@link Format#bitrate} * if a more accurate estimate of the current track bitrate is available. - * @param playbackSpeed The current playback speed. + * @param playbackSpeed The current factor by which playback is sped up. * @param effectiveBitrate The bitrate available to this selection. * @return Whether this {@link Format} can be selected. */ @@ -497,7 +454,7 @@ protected long getMinDurationToRetainAfterDiscardUs() { * Long#MIN_VALUE} to ignore track exclusion. */ private int determineIdealSelectedIndex(long nowMs) { - long effectiveBitrate = bandwidthProvider.getAllocatedBandwidth(); + long effectiveBitrate = getAllocatedBandwidth(); int lowestBitrateAllowedIndex = 0; for (int i = 0; i < length; i++) { if (nowMs == Long.MIN_VALUE || !isBlacklisted(i, nowMs)) { @@ -520,162 +477,181 @@ private long minDurationForQualityIncreaseUs(long availableDurationUs) { : minDurationForQualityIncreaseUs; } - /** Provides the allocated bandwidth. */ - private interface BandwidthProvider { - - /** Returns the allocated bitrate. */ - long getAllocatedBandwidth(); - } - - private static final class DefaultBandwidthProvider implements BandwidthProvider { - - private final BandwidthMeter bandwidthMeter; - private final float bandwidthFraction; - private final long reservedBandwidth; - - @Nullable private long[][] allocationCheckpoints; - - /* package */ DefaultBandwidthProvider( - BandwidthMeter bandwidthMeter, float bandwidthFraction, long reservedBandwidth) { - this.bandwidthMeter = bandwidthMeter; - this.bandwidthFraction = bandwidthFraction; - this.reservedBandwidth = reservedBandwidth; + private long getAllocatedBandwidth() { + long totalBandwidth = (long) (bandwidthMeter.getBitrateEstimate() * bandwidthFraction); + if (adaptationCheckpoints.isEmpty()) { + return totalBandwidth; } + int nextIndex = 1; + while (nextIndex < adaptationCheckpoints.size() - 1 + && adaptationCheckpoints.get(nextIndex).totalBandwidth < totalBandwidth) { + nextIndex++; + } + AdaptationCheckpoint previous = adaptationCheckpoints.get(nextIndex - 1); + AdaptationCheckpoint next = adaptationCheckpoints.get(nextIndex); + float fractionBetweenCheckpoints = + (float) (totalBandwidth - previous.totalBandwidth) + / (next.totalBandwidth - previous.totalBandwidth); + return previous.allocatedBandwidth + + (long) + (fractionBetweenCheckpoints * (next.allocatedBandwidth - previous.allocatedBandwidth)); + } - @Override - public long getAllocatedBandwidth() { - long totalBandwidth = (long) (bandwidthMeter.getBitrateEstimate() * bandwidthFraction); - long allocatableBandwidth = max(0L, totalBandwidth - reservedBandwidth); - if (allocationCheckpoints == null) { - return allocatableBandwidth; + /** + * Returns adaptation checkpoints for allocating bandwidth for adaptive track selections. + * + * @param definitions Array of track selection {@link Definition definitions}. Elements may be + * null. + * @return List of {@link AdaptationCheckpoint checkpoints} for each adaptive {@link Definition} + * with more than one selected track. + */ + private static ImmutableList> getAdaptationCheckpoints( + @NullableType Definition[] definitions) { + List> checkPointBuilders = + new ArrayList<>(); + for (int i = 0; i < definitions.length; i++) { + if (definitions[i] != null && definitions[i].tracks.length > 1) { + ImmutableList.Builder builder = ImmutableList.builder(); + // Add initial all-zero checkpoint. + builder.add(new AdaptationCheckpoint(/* totalBandwidth= */ 0, /* allocatedBandwidth= */ 0)); + checkPointBuilders.add(builder); + } else { + checkPointBuilders.add(null); } - int nextIndex = 1; - while (nextIndex < allocationCheckpoints.length - 1 - && allocationCheckpoints[nextIndex][0] < allocatableBandwidth) { - nextIndex++; + } + // Add minimum bitrate selection checkpoint. + long[][] trackBitrates = getSortedTrackBitrates(definitions); + int[] currentTrackIndices = new int[trackBitrates.length]; + long[] currentTrackBitrates = new long[trackBitrates.length]; + for (int i = 0; i < trackBitrates.length; i++) { + currentTrackBitrates[i] = trackBitrates[i].length == 0 ? 0 : trackBitrates[i][0]; + } + addCheckpoint(checkPointBuilders, currentTrackBitrates); + // Iterate through all adaptive checkpoints. + ImmutableList switchOrder = getSwitchOrder(trackBitrates); + for (int i = 0; i < switchOrder.size(); i++) { + int switchIndex = switchOrder.get(i); + int newTrackIndex = ++currentTrackIndices[switchIndex]; + currentTrackBitrates[switchIndex] = trackBitrates[switchIndex][newTrackIndex]; + addCheckpoint(checkPointBuilders, currentTrackBitrates); + } + // Add final checkpoint to extrapolate additional bandwidth for adaptive selections. + for (int i = 0; i < definitions.length; i++) { + if (checkPointBuilders.get(i) != null) { + currentTrackBitrates[i] *= 2; } - long[] previous = allocationCheckpoints[nextIndex - 1]; - long[] next = allocationCheckpoints[nextIndex]; - float fractionBetweenCheckpoints = - (float) (allocatableBandwidth - previous[0]) / (next[0] - previous[0]); - return previous[1] + (long) (fractionBetweenCheckpoints * (next[1] - previous[1])); } + addCheckpoint(checkPointBuilders, currentTrackBitrates); + ImmutableList.Builder> output = ImmutableList.builder(); + for (int i = 0; i < checkPointBuilders.size(); i++) { + @Nullable ImmutableList.Builder builder = checkPointBuilders.get(i); + output.add(builder == null ? ImmutableList.of() : builder.build()); + } + return output.build(); + } - /* package */ void experimentalSetBandwidthAllocationCheckpoints( - long[][] allocationCheckpoints) { - Assertions.checkArgument(allocationCheckpoints.length >= 2); - this.allocationCheckpoints = allocationCheckpoints; + /** Returns sorted track bitrates for all selected tracks. */ + private static long[][] getSortedTrackBitrates(@NullableType Definition[] definitions) { + long[][] trackBitrates = new long[definitions.length][]; + for (int i = 0; i < definitions.length; i++) { + @Nullable Definition definition = definitions[i]; + if (definition == null) { + trackBitrates[i] = new long[0]; + continue; + } + trackBitrates[i] = new long[definition.tracks.length]; + for (int j = 0; j < definition.tracks.length; j++) { + trackBitrates[i][j] = definition.group.getFormat(definition.tracks[j]).bitrate; + } + Arrays.sort(trackBitrates[i]); } + return trackBitrates; } /** - * Returns allocation checkpoints for allocating bandwidth between multiple adaptive track - * selections. + * Returns order of track indices in which the respective track should be switched up. * - * @param trackBitrates Array of [selectionIndex][trackIndex] -> trackBitrate. - * @return Array of allocation checkpoints [selectionIndex][checkpointIndex][2] with [0]=total - * bandwidth at checkpoint and [1]=allocated bandwidth at checkpoint. + * @param trackBitrates Sorted tracks bitrates for each selection. + * @return List of track indices indicating in which order tracks should be switched up. */ - private static long[][][] getAllocationCheckpoints(long[][] trackBitrates) { + private static ImmutableList getSwitchOrder(long[][] trackBitrates) { // Algorithm: - // 1. Use log bitrates to treat all resolution update steps equally. + // 1. Use log bitrates to treat all bitrate update steps equally. // 2. Distribute switch points for each selection equally in the same [0.0-1.0] range. // 3. Switch up one format at a time in the order of the switch points. - double[][] logBitrates = getLogArrayValues(trackBitrates); - double[][] switchPoints = getSwitchPoints(logBitrates); - - // There will be (count(switch point) + 3) checkpoints: - // [0] = all zero, [1] = minimum bitrates, [2-(end-1)] = up-switch points, - // [end] = extra point to set slope for additional bitrate. - int checkpointCount = countArrayElements(switchPoints) + 3; - long[][][] checkpoints = new long[logBitrates.length][checkpointCount][2]; - int[] currentSelection = new int[logBitrates.length]; - setCheckpointValues(checkpoints, /* checkpointIndex= */ 1, trackBitrates, currentSelection); - for (int checkpointIndex = 2; checkpointIndex < checkpointCount - 1; checkpointIndex++) { - int nextUpdateIndex = 0; - double nextUpdateSwitchPoint = Double.MAX_VALUE; - for (int i = 0; i < logBitrates.length; i++) { - if (currentSelection[i] + 1 == logBitrates[i].length) { - continue; - } - double switchPoint = switchPoints[i][currentSelection[i]]; - if (switchPoint < nextUpdateSwitchPoint) { - nextUpdateSwitchPoint = switchPoint; - nextUpdateIndex = i; - } + Multimap switchPoints = MultimapBuilder.treeKeys().arrayListValues().build(); + for (int i = 0; i < trackBitrates.length; i++) { + if (trackBitrates[i].length <= 1) { + continue; } - currentSelection[nextUpdateIndex]++; - setCheckpointValues(checkpoints, checkpointIndex, trackBitrates, currentSelection); - } - for (long[][] points : checkpoints) { - points[checkpointCount - 1][0] = 2 * points[checkpointCount - 2][0]; - points[checkpointCount - 1][1] = 2 * points[checkpointCount - 2][1]; - } - return checkpoints; - } - - /** Converts all input values to Math.log(value). */ - private static double[][] getLogArrayValues(long[][] values) { - double[][] logValues = new double[values.length][]; - for (int i = 0; i < values.length; i++) { - logValues[i] = new double[values[i].length]; - for (int j = 0; j < values[i].length; j++) { - logValues[i][j] = values[i][j] == Format.NO_VALUE ? 0 : Math.log(values[i][j]); + double[] logBitrates = new double[trackBitrates[i].length]; + for (int j = 0; j < trackBitrates[i].length; j++) { + logBitrates[j] = trackBitrates[i][j] == Format.NO_VALUE ? 0 : Math.log(trackBitrates[i][j]); + } + double totalBitrateDiff = logBitrates[logBitrates.length - 1] - logBitrates[0]; + for (int j = 0; j < logBitrates.length - 1; j++) { + double switchBitrate = 0.5 * (logBitrates[j] + logBitrates[j + 1]); + double switchPoint = + totalBitrateDiff == 0.0 ? 1.0 : (switchBitrate - logBitrates[0]) / totalBitrateDiff; + switchPoints.put(switchPoint, i); } } - return logValues; + return ImmutableList.copyOf(switchPoints.values()); } /** - * Returns idealized switch points for each switch between consecutive track selection bitrates. + * Add a checkpoint to the builders. * - * @param logBitrates Log bitrates with [selectionCount][formatCount]. - * @return Linearly distributed switch points in the range of [0.0-1.0]. + * @param checkPointBuilders Builders for adaptation checkpoints. May have null elements. + * @param checkpointBitrates The bitrates of each track at this checkpoint. */ - private static double[][] getSwitchPoints(double[][] logBitrates) { - double[][] switchPoints = new double[logBitrates.length][]; - for (int i = 0; i < logBitrates.length; i++) { - switchPoints[i] = new double[logBitrates[i].length - 1]; - if (switchPoints[i].length == 0) { + private static void addCheckpoint( + List> checkPointBuilders, + long[] checkpointBitrates) { + // Total bitrate includes all fixed tracks. + long totalBitrate = 0; + for (int i = 0; i < checkpointBitrates.length; i++) { + totalBitrate += checkpointBitrates[i]; + } + for (int i = 0; i < checkPointBuilders.size(); i++) { + @Nullable ImmutableList.Builder builder = checkPointBuilders.get(i); + if (builder == null) { continue; } - double totalBitrateDiff = logBitrates[i][logBitrates[i].length - 1] - logBitrates[i][0]; - for (int j = 0; j < logBitrates[i].length - 1; j++) { - double switchBitrate = 0.5 * (logBitrates[i][j] + logBitrates[i][j + 1]); - switchPoints[i][j] = - totalBitrateDiff == 0.0 ? 1.0 : (switchBitrate - logBitrates[i][0]) / totalBitrateDiff; - } + builder.add( + new AdaptationCheckpoint( + /* totalBandwidth= */ totalBitrate, /* allocatedBandwidth= */ checkpointBitrates[i])); } - return switchPoints; } - /** Returns total number of elements in a 2D array. */ - private static int countArrayElements(double[][] array) { - int count = 0; - for (double[] subArray : array) { - count += subArray.length; + /** Checkpoint to determine allocated bandwidth. */ + protected static final class AdaptationCheckpoint { + + /** Total bandwidth in bits per second at which this checkpoint applies. */ + public final long totalBandwidth; + /** Allocated bandwidth at this checkpoint in bits per second. */ + public final long allocatedBandwidth; + + public AdaptationCheckpoint(long totalBandwidth, long allocatedBandwidth) { + this.totalBandwidth = totalBandwidth; + this.allocatedBandwidth = allocatedBandwidth; } - return count; - } - /** - * Sets checkpoint bitrates. - * - * @param checkpoints Output checkpoints with [selectionIndex][checkpointIndex][2] where [0]=Total - * bitrate and [1]=Allocated bitrate. - * @param checkpointIndex The checkpoint index. - * @param trackBitrates The track bitrates with [selectionIndex][trackIndex]. - * @param selectedTracks The indices of selected tracks for each selection for this checkpoint. - */ - private static void setCheckpointValues( - long[][][] checkpoints, int checkpointIndex, long[][] trackBitrates, int[] selectedTracks) { - long totalBitrate = 0; - for (int i = 0; i < checkpoints.length; i++) { - checkpoints[i][checkpointIndex][1] = trackBitrates[i][selectedTracks[i]]; - totalBitrate += checkpoints[i][checkpointIndex][1]; + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (!(o instanceof AdaptationCheckpoint)) { + return false; + } + AdaptationCheckpoint that = (AdaptationCheckpoint) o; + return totalBandwidth == that.totalBandwidth && allocatedBandwidth == that.allocatedBandwidth; } - for (long[][] points : checkpoints) { - points[checkpointIndex][0] = totalBitrate; + + @Override + public int hashCode() { + return 31 * (int) totalBandwidth + (int) allocatedBandwidth; } } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/BaseTrackSelection.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/BaseTrackSelection.java index 4be4bf7075c..17c486b45ae 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/BaseTrackSelection.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/BaseTrackSelection.java @@ -28,27 +28,17 @@ import java.util.Arrays; import java.util.List; -/** - * An abstract base class suitable for most {@link TrackSelection} implementations. - */ -public abstract class BaseTrackSelection implements TrackSelection { +/** An abstract base class suitable for most {@link ExoTrackSelection} implementations. */ +public abstract class BaseTrackSelection implements ExoTrackSelection { - /** - * The selected {@link TrackGroup}. - */ + /** The selected {@link TrackGroup}. */ protected final TrackGroup group; - /** - * The number of selected tracks within the {@link TrackGroup}. Always greater than zero. - */ + /** The number of selected tracks within the {@link TrackGroup}. Always greater than zero. */ protected final int length; - /** - * The indices of the selected tracks in {@link #group}, in order of decreasing bandwidth. - */ + /** The indices of the selected tracks in {@link #group}, in order of decreasing bandwidth. */ protected final int[] tracks; - /** - * The {@link Format}s of the selected tracks, in order of decreasing bandwidth. - */ + /** The {@link Format}s of the selected tracks, in order of decreasing bandwidth. */ private final Format[] formats; /** Selected track exclusion timestamps, in order of decreasing bandwidth. */ private final long[] excludeUntilTimes; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelector.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelector.java index f61c0b14d45..627df86cf69 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelector.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelector.java @@ -25,6 +25,7 @@ import android.util.SparseBooleanArray; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.C.FormatSupport; import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.Player; @@ -32,13 +33,15 @@ import com.google.android.exoplayer2.RendererCapabilities; import com.google.android.exoplayer2.RendererCapabilities.AdaptiveSupport; import com.google.android.exoplayer2.RendererCapabilities.Capabilities; -import com.google.android.exoplayer2.RendererCapabilities.FormatSupport; import com.google.android.exoplayer2.RendererConfiguration; +import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Util; import com.google.common.collect.ComparisonChain; +import com.google.common.collect.ImmutableList; import com.google.common.collect.Ordering; import com.google.common.primitives.Ints; import java.util.ArrayList; @@ -50,6 +53,7 @@ import java.util.concurrent.atomic.AtomicReference; import org.checkerframework.checker.initialization.qual.UnderInitialization; import org.checkerframework.checker.nullness.compatqual.NullableType; +import org.checkerframework.checker.nullness.qual.EnsuresNonNull; /** * A default {@link TrackSelector} suitable for most use cases. Track selections are made according @@ -155,7 +159,7 @@ * * Tunneled playback can be enabled in cases where the combination of renderers and selected tracks * support it. Tunneled playback is enabled by passing an audio session ID to {@link - * ParametersBuilder#setTunnelingAudioSessionId(int)}. + * ParametersBuilder#setTunnelingEnabled(boolean)}. */ public class DefaultTrackSelector extends MappingTrackSelector { @@ -180,6 +184,7 @@ public static final class ParametersBuilder extends TrackSelectionParameters.Bui private int viewportWidth; private int viewportHeight; private boolean viewportOrientationMayChange; + private ImmutableList preferredVideoMimeTypes; // Audio private int maxAudioChannelCount; private int maxAudioBitrate; @@ -187,11 +192,13 @@ public static final class ParametersBuilder extends TrackSelectionParameters.Bui private boolean allowAudioMixedMimeTypeAdaptiveness; private boolean allowAudioMixedSampleRateAdaptiveness; private boolean allowAudioMixedChannelCountAdaptiveness; + private ImmutableList preferredAudioMimeTypes; // General private boolean forceLowestBitrate; private boolean forceHighestSupportedBitrate; private boolean exceedRendererCapabilitiesIfNecessary; - private int tunnelingAudioSessionId; + private boolean tunnelingEnabled; + private boolean allowMultipleAdaptiveSelections; private final SparseArray> selectionOverrides; @@ -215,7 +222,6 @@ public ParametersBuilder() { * * @param context Any context. */ - public ParametersBuilder(Context context) { super(context); setInitialValuesWithoutContext(); @@ -245,6 +251,7 @@ private ParametersBuilder(Parameters initialValues) { viewportWidth = initialValues.viewportWidth; viewportHeight = initialValues.viewportHeight; viewportOrientationMayChange = initialValues.viewportOrientationMayChange; + preferredVideoMimeTypes = initialValues.preferredVideoMimeTypes; // Audio maxAudioChannelCount = initialValues.maxAudioChannelCount; maxAudioBitrate = initialValues.maxAudioBitrate; @@ -253,11 +260,13 @@ private ParametersBuilder(Parameters initialValues) { allowAudioMixedSampleRateAdaptiveness = initialValues.allowAudioMixedSampleRateAdaptiveness; allowAudioMixedChannelCountAdaptiveness = initialValues.allowAudioMixedChannelCountAdaptiveness; + preferredAudioMimeTypes = initialValues.preferredAudioMimeTypes; // General forceLowestBitrate = initialValues.forceLowestBitrate; forceHighestSupportedBitrate = initialValues.forceHighestSupportedBitrate; exceedRendererCapabilitiesIfNecessary = initialValues.exceedRendererCapabilitiesIfNecessary; - tunnelingAudioSessionId = initialValues.tunnelingAudioSessionId; + tunnelingEnabled = initialValues.tunnelingEnabled; + allowMultipleAdaptiveSelections = initialValues.allowMultipleAdaptiveSelections; // Overrides selectionOverrides = cloneSelectionOverrides(initialValues.selectionOverrides); rendererDisabledFlags = initialValues.rendererDisabledFlags.clone(); @@ -442,6 +451,29 @@ public ParametersBuilder setViewportSize( return this; } + /** + * Sets the preferred sample MIME type for video tracks. + * + * @param mimeType The preferred MIME type for video tracks, or {@code null} to clear a + * previously set preference. + * @return This builder. + */ + public ParametersBuilder setPreferredVideoMimeType(@Nullable String mimeType) { + return mimeType == null ? setPreferredVideoMimeTypes() : setPreferredVideoMimeTypes(mimeType); + } + + /** + * Sets the preferred sample MIME types for video tracks. + * + * @param mimeTypes The preferred MIME types for video tracks in order of preference, or an + * empty list for no preference. + * @return This builder. + */ + public ParametersBuilder setPreferredVideoMimeTypes(String... mimeTypes) { + preferredVideoMimeTypes = ImmutableList.copyOf(mimeTypes); + return this; + } + // Audio @Override @@ -450,6 +482,18 @@ public ParametersBuilder setPreferredAudioLanguage(@Nullable String preferredAud return this; } + @Override + public ParametersBuilder setPreferredAudioLanguages(String... preferredAudioLanguages) { + super.setPreferredAudioLanguages(preferredAudioLanguages); + return this; + } + + @Override + public ParametersBuilder setPreferredAudioRoleFlags(@C.RoleFlags int preferredAudioRoleFlags) { + super.setPreferredAudioRoleFlags(preferredAudioRoleFlags); + return this; + } + /** * Sets the maximum allowed audio channel count. * @@ -531,6 +575,29 @@ public ParametersBuilder setAllowAudioMixedChannelCountAdaptiveness( return this; } + /** + * Sets the preferred sample MIME type for audio tracks. + * + * @param mimeType The preferred MIME type for audio tracks, or {@code null} to clear a + * previously set preference. + * @return This builder. + */ + public ParametersBuilder setPreferredAudioMimeType(@Nullable String mimeType) { + return mimeType == null ? setPreferredAudioMimeTypes() : setPreferredAudioMimeTypes(mimeType); + } + + /** + * Sets the preferred sample MIME types for audio tracks. + * + * @param mimeTypes The preferred MIME types for audio tracks in order of preference, or an + * empty list for no preference. + * @return This builder. + */ + public ParametersBuilder setPreferredAudioMimeTypes(String... mimeTypes) { + preferredAudioMimeTypes = ImmutableList.copyOf(mimeTypes); + return this; + } + // Text @Override @@ -546,6 +613,12 @@ public ParametersBuilder setPreferredTextLanguage(@Nullable String preferredText return this; } + @Override + public ParametersBuilder setPreferredTextLanguages(String... preferredTextLanguages) { + super.setPreferredTextLanguages(preferredTextLanguages); + return this; + } + @Override public ParametersBuilder setPreferredTextRoleFlags(@C.RoleFlags int preferredTextRoleFlags) { super.setPreferredTextRoleFlags(preferredTextRoleFlags); @@ -613,20 +686,26 @@ public ParametersBuilder setExceedRendererCapabilitiesIfNecessary( } /** - * Sets the audio session id to use when tunneling. - * - *

    Enables or disables tunneling. To enable tunneling, pass an audio session id to use when - * in tunneling mode. Session ids can be generated using {@link - * C#generateAudioSessionIdV21(Context)}. To disable tunneling pass {@link - * C#AUDIO_SESSION_ID_UNSET}. Tunneling will only be activated if it's both enabled and + * Sets whether to enable tunneling if possible. Tunneling will only be enabled if it's * supported by the audio and video renderers for the selected tracks. * - * @param tunnelingAudioSessionId The audio session id to use when tunneling, or {@link - * C#AUDIO_SESSION_ID_UNSET} to disable tunneling. + * @param tunnelingEnabled Whether to enable tunneling if possible. + * @return This builder. + */ + public ParametersBuilder setTunnelingEnabled(boolean tunnelingEnabled) { + this.tunnelingEnabled = tunnelingEnabled; + return this; + } + + /** + * Sets whether multiple adaptive selections with more than one track are allowed. + * + * @param allowMultipleAdaptiveSelections Whether multiple adaptive selections are allowed. * @return This builder. */ - public ParametersBuilder setTunnelingAudioSessionId(int tunnelingAudioSessionId) { - this.tunnelingAudioSessionId = tunnelingAudioSessionId; + public ParametersBuilder setAllowMultipleAdaptiveSelections( + boolean allowMultipleAdaptiveSelections) { + this.allowMultipleAdaptiveSelections = allowMultipleAdaptiveSelections; return this; } @@ -746,9 +825,7 @@ public final ParametersBuilder clearSelectionOverrides() { return this; } - /** - * Builds a {@link Parameters} instance with the selected values. - */ + /** Builds a {@link Parameters} instance with the selected values. */ public Parameters build() { return new Parameters( // Video @@ -766,16 +843,19 @@ public Parameters build() { viewportWidth, viewportHeight, viewportOrientationMayChange, + preferredVideoMimeTypes, // Audio - preferredAudioLanguage, + preferredAudioLanguages, + preferredAudioRoleFlags, maxAudioChannelCount, maxAudioBitrate, exceedAudioConstraintsIfNecessary, allowAudioMixedMimeTypeAdaptiveness, allowAudioMixedSampleRateAdaptiveness, allowAudioMixedChannelCountAdaptiveness, + preferredAudioMimeTypes, // Text - preferredTextLanguage, + preferredTextLanguages, preferredTextRoleFlags, selectUndeterminedTextLanguage, disabledTextTrackSelectionFlags, @@ -783,11 +863,13 @@ public Parameters build() { forceLowestBitrate, forceHighestSupportedBitrate, exceedRendererCapabilitiesIfNecessary, - tunnelingAudioSessionId, + tunnelingEnabled, + allowMultipleAdaptiveSelections, selectionOverrides, rendererDisabledFlags); } + @EnsuresNonNull({"preferredVideoMimeTypes", "preferredAudioMimeTypes"}) private void setInitialValuesWithoutContext(@UnderInitialization ParametersBuilder this) { // Video maxVideoWidth = Integer.MAX_VALUE; @@ -800,6 +882,7 @@ private void setInitialValuesWithoutContext(@UnderInitialization ParametersBuild viewportWidth = Integer.MAX_VALUE; viewportHeight = Integer.MAX_VALUE; viewportOrientationMayChange = true; + preferredVideoMimeTypes = ImmutableList.of(); // Audio maxAudioChannelCount = Integer.MAX_VALUE; maxAudioBitrate = Integer.MAX_VALUE; @@ -807,11 +890,13 @@ private void setInitialValuesWithoutContext(@UnderInitialization ParametersBuild allowAudioMixedMimeTypeAdaptiveness = false; allowAudioMixedSampleRateAdaptiveness = false; allowAudioMixedChannelCountAdaptiveness = false; + preferredAudioMimeTypes = ImmutableList.of(); // General forceLowestBitrate = false; forceHighestSupportedBitrate = false; exceedRendererCapabilitiesIfNecessary = true; - tunnelingAudioSessionId = C.AUDIO_SESSION_ID_UNSET; + tunnelingEnabled = false; + allowMultipleAdaptiveSelections = true; } private static SparseArray> @@ -932,6 +1017,11 @@ public static Parameters getDefaults(Context context) { * The default value is {@code true}. */ public final boolean viewportOrientationMayChange; + /** + * The preferred sample MIME types for video tracks in order of preference, or an empty list for + * no preference. The default is an empty list. + */ + public final ImmutableList preferredVideoMimeTypes; // Audio /** * Maximum allowed audio channel count. The default value is {@link Integer#MAX_VALUE} (i.e. no @@ -964,7 +1054,11 @@ public static Parameters getDefaults(Context context) { * false}. */ public final boolean allowAudioMixedChannelCountAdaptiveness; - + /** + * The preferred sample MIME types for audio tracks in order of preference, or an empty list for + * no preference. The default is an empty list. + */ + public final ImmutableList preferredAudioMimeTypes; // General /** * Whether to force selection of the single lowest bitrate audio and video tracks that comply @@ -986,12 +1080,17 @@ public static Parameters getDefaults(Context context) { * {@code true}. */ public final boolean exceedRendererCapabilitiesIfNecessary; + /** Whether to enable tunneling if possible. */ + public final boolean tunnelingEnabled; /** - * The audio session id to use when tunneling, or {@link C#AUDIO_SESSION_ID_UNSET} if tunneling - * is disabled. The default value is {@link C#AUDIO_SESSION_ID_UNSET} (i.e. tunneling is - * disabled). + * Whether multiple adaptive selections with more than one track are allowed. The default value + * is {@code true}. + * + *

    Note that tracks are only eligible for adaptation if they define a bitrate, the renderers + * support the tracks and allow adaptation between them, and they are not excluded based on + * other track selection parameters. */ - public final int tunnelingAudioSessionId; + public final boolean allowMultipleAdaptiveSelections; // Overrides private final SparseArray> @@ -1014,16 +1113,19 @@ public static Parameters getDefaults(Context context) { int viewportWidth, int viewportHeight, boolean viewportOrientationMayChange, + ImmutableList preferredVideoMimeTypes, // Audio - @Nullable String preferredAudioLanguage, + ImmutableList preferredAudioLanguages, + @C.RoleFlags int preferredAudioRoleFlags, int maxAudioChannelCount, int maxAudioBitrate, boolean exceedAudioConstraintsIfNecessary, boolean allowAudioMixedMimeTypeAdaptiveness, boolean allowAudioMixedSampleRateAdaptiveness, boolean allowAudioMixedChannelCountAdaptiveness, + ImmutableList preferredAudioMimeTypes, // Text - @Nullable String preferredTextLanguage, + ImmutableList preferredTextLanguages, @C.RoleFlags int preferredTextRoleFlags, boolean selectUndeterminedTextLanguage, @C.SelectionFlags int disabledTextTrackSelectionFlags, @@ -1031,13 +1133,15 @@ public static Parameters getDefaults(Context context) { boolean forceLowestBitrate, boolean forceHighestSupportedBitrate, boolean exceedRendererCapabilitiesIfNecessary, - int tunnelingAudioSessionId, + boolean tunnelingEnabled, + boolean allowMultipleAdaptiveSelections, // Overrides SparseArray> selectionOverrides, SparseBooleanArray rendererDisabledFlags) { super( - preferredAudioLanguage, - preferredTextLanguage, + preferredAudioLanguages, + preferredAudioRoleFlags, + preferredTextLanguages, preferredTextRoleFlags, selectUndeterminedTextLanguage, disabledTextTrackSelectionFlags); @@ -1056,6 +1160,7 @@ public static Parameters getDefaults(Context context) { this.viewportWidth = viewportWidth; this.viewportHeight = viewportHeight; this.viewportOrientationMayChange = viewportOrientationMayChange; + this.preferredVideoMimeTypes = preferredVideoMimeTypes; // Audio this.maxAudioChannelCount = maxAudioChannelCount; this.maxAudioBitrate = maxAudioBitrate; @@ -1063,18 +1168,19 @@ public static Parameters getDefaults(Context context) { this.allowAudioMixedMimeTypeAdaptiveness = allowAudioMixedMimeTypeAdaptiveness; this.allowAudioMixedSampleRateAdaptiveness = allowAudioMixedSampleRateAdaptiveness; this.allowAudioMixedChannelCountAdaptiveness = allowAudioMixedChannelCountAdaptiveness; + this.preferredAudioMimeTypes = preferredAudioMimeTypes; // General this.forceLowestBitrate = forceLowestBitrate; this.forceHighestSupportedBitrate = forceHighestSupportedBitrate; this.exceedRendererCapabilitiesIfNecessary = exceedRendererCapabilitiesIfNecessary; - this.tunnelingAudioSessionId = tunnelingAudioSessionId; + this.tunnelingEnabled = tunnelingEnabled; + this.allowMultipleAdaptiveSelections = allowMultipleAdaptiveSelections; // Overrides this.selectionOverrides = selectionOverrides; this.rendererDisabledFlags = rendererDisabledFlags; } - /* package */ - Parameters(Parcel in) { + /* package */ Parameters(Parcel in) { super(in); // Video this.maxVideoWidth = in.readInt(); @@ -1091,6 +1197,9 @@ public static Parameters getDefaults(Context context) { this.viewportWidth = in.readInt(); this.viewportHeight = in.readInt(); this.viewportOrientationMayChange = Util.readBoolean(in); + ArrayList preferredVideoMimeTypes = new ArrayList<>(); + in.readList(preferredVideoMimeTypes, /* loader= */ null); + this.preferredVideoMimeTypes = ImmutableList.copyOf(preferredVideoMimeTypes); // Audio this.maxAudioChannelCount = in.readInt(); this.maxAudioBitrate = in.readInt(); @@ -1098,11 +1207,15 @@ public static Parameters getDefaults(Context context) { this.allowAudioMixedMimeTypeAdaptiveness = Util.readBoolean(in); this.allowAudioMixedSampleRateAdaptiveness = Util.readBoolean(in); this.allowAudioMixedChannelCountAdaptiveness = Util.readBoolean(in); + ArrayList preferredAudioMimeTypes = new ArrayList<>(); + in.readList(preferredAudioMimeTypes, /* loader= */ null); + this.preferredAudioMimeTypes = ImmutableList.copyOf(preferredAudioMimeTypes); // General this.forceLowestBitrate = Util.readBoolean(in); this.forceHighestSupportedBitrate = Util.readBoolean(in); this.exceedRendererCapabilitiesIfNecessary = Util.readBoolean(in); - this.tunnelingAudioSessionId = in.readInt(); + this.tunnelingEnabled = Util.readBoolean(in); + this.allowMultipleAdaptiveSelections = Util.readBoolean(in); // Overrides this.selectionOverrides = readSelectionOverrides(in); this.rendererDisabledFlags = Util.castNonNull(in.readSparseBooleanArray()); @@ -1176,6 +1289,7 @@ public boolean equals(@Nullable Object obj) { && viewportOrientationMayChange == other.viewportOrientationMayChange && viewportWidth == other.viewportWidth && viewportHeight == other.viewportHeight + && preferredVideoMimeTypes.equals(other.preferredVideoMimeTypes) // Audio && maxAudioChannelCount == other.maxAudioChannelCount && maxAudioBitrate == other.maxAudioBitrate @@ -1184,11 +1298,13 @@ public boolean equals(@Nullable Object obj) { && allowAudioMixedSampleRateAdaptiveness == other.allowAudioMixedSampleRateAdaptiveness && allowAudioMixedChannelCountAdaptiveness == other.allowAudioMixedChannelCountAdaptiveness + && preferredAudioMimeTypes.equals(other.preferredAudioMimeTypes) // General && forceLowestBitrate == other.forceLowestBitrate && forceHighestSupportedBitrate == other.forceHighestSupportedBitrate && exceedRendererCapabilitiesIfNecessary == other.exceedRendererCapabilitiesIfNecessary - && tunnelingAudioSessionId == other.tunnelingAudioSessionId + && tunnelingEnabled == other.tunnelingEnabled + && allowMultipleAdaptiveSelections == other.allowMultipleAdaptiveSelections // Overrides && areRendererDisabledFlagsEqual(rendererDisabledFlags, other.rendererDisabledFlags) && areSelectionOverridesEqual(selectionOverrides, other.selectionOverrides); @@ -1212,6 +1328,7 @@ public int hashCode() { result = 31 * result + (viewportOrientationMayChange ? 1 : 0); result = 31 * result + viewportWidth; result = 31 * result + viewportHeight; + result = 31 * result + preferredVideoMimeTypes.hashCode(); // Audio result = 31 * result + maxAudioChannelCount; result = 31 * result + maxAudioBitrate; @@ -1219,11 +1336,13 @@ public int hashCode() { result = 31 * result + (allowAudioMixedMimeTypeAdaptiveness ? 1 : 0); result = 31 * result + (allowAudioMixedSampleRateAdaptiveness ? 1 : 0); result = 31 * result + (allowAudioMixedChannelCountAdaptiveness ? 1 : 0); + result = 31 * result + preferredAudioMimeTypes.hashCode(); // General result = 31 * result + (forceLowestBitrate ? 1 : 0); result = 31 * result + (forceHighestSupportedBitrate ? 1 : 0); result = 31 * result + (exceedRendererCapabilitiesIfNecessary ? 1 : 0); - result = 31 * result + tunnelingAudioSessionId; + result = 31 * result + (tunnelingEnabled ? 1 : 0); + result = 31 * result + (allowMultipleAdaptiveSelections ? 1 : 0); // Overrides (omitted from hashCode). return result; } @@ -1253,6 +1372,7 @@ public void writeToParcel(Parcel dest, int flags) { dest.writeInt(viewportWidth); dest.writeInt(viewportHeight); Util.writeBoolean(dest, viewportOrientationMayChange); + dest.writeList(preferredVideoMimeTypes); // Audio dest.writeInt(maxAudioChannelCount); dest.writeInt(maxAudioBitrate); @@ -1260,11 +1380,13 @@ public void writeToParcel(Parcel dest, int flags) { Util.writeBoolean(dest, allowAudioMixedMimeTypeAdaptiveness); Util.writeBoolean(dest, allowAudioMixedSampleRateAdaptiveness); Util.writeBoolean(dest, allowAudioMixedChannelCountAdaptiveness); + dest.writeList(preferredAudioMimeTypes); // General Util.writeBoolean(dest, forceLowestBitrate); Util.writeBoolean(dest, forceHighestSupportedBitrate); Util.writeBoolean(dest, exceedRendererCapabilitiesIfNecessary); - dest.writeInt(tunnelingAudioSessionId); + Util.writeBoolean(dest, tunnelingEnabled); + Util.writeBoolean(dest, allowMultipleAdaptiveSelections); // Overrides writeSelectionOverridesToParcel(dest, selectionOverrides); dest.writeSparseBooleanArray(rendererDisabledFlags); @@ -1489,6 +1611,7 @@ public SelectionOverride[] newArray(int size) { * dimension). */ private static final float FRACTION_TO_CONSIDER_FULLSCREEN = 0.98f; + private static final int[] NO_TRACKS = new int[0]; /** Ordering of two format values. A known value is considered greater than Format#NO_VALUE. */ private static final Ordering FORMAT_VALUE_ORDERING = @@ -1500,20 +1623,18 @@ public SelectionOverride[] newArray(int size) { /** Ordering where all elements are equal. */ private static final Ordering NO_ORDER = Ordering.from((first, second) -> 0); - private final TrackSelection.Factory trackSelectionFactory; + private final ExoTrackSelection.Factory trackSelectionFactory; private final AtomicReference parametersReference; - private boolean allowMultipleAdaptiveSelections; - /** @deprecated Use {@link #DefaultTrackSelector(Context)} instead. */ @Deprecated public DefaultTrackSelector() { this(Parameters.DEFAULT_WITHOUT_CONTEXT, new AdaptiveTrackSelection.Factory()); } - /** @deprecated Use {@link #DefaultTrackSelector(Context, TrackSelection.Factory)}. */ + /** @deprecated Use {@link #DefaultTrackSelector(Context, ExoTrackSelection.Factory)}. */ @Deprecated - public DefaultTrackSelector(TrackSelection.Factory trackSelectionFactory) { + public DefaultTrackSelector(ExoTrackSelection.Factory trackSelectionFactory) { this(Parameters.DEFAULT_WITHOUT_CONTEXT, trackSelectionFactory); } @@ -1524,17 +1645,18 @@ public DefaultTrackSelector(Context context) { /** * @param context Any {@link Context}. - * @param trackSelectionFactory A factory for {@link TrackSelection}s. + * @param trackSelectionFactory A factory for {@link ExoTrackSelection}s. */ - public DefaultTrackSelector(Context context, TrackSelection.Factory trackSelectionFactory) { + public DefaultTrackSelector(Context context, ExoTrackSelection.Factory trackSelectionFactory) { this(Parameters.getDefaults(context), trackSelectionFactory); } /** * @param parameters Initial {@link Parameters}. - * @param trackSelectionFactory A factory for {@link TrackSelection}s. + * @param trackSelectionFactory A factory for {@link ExoTrackSelection}s. */ - public DefaultTrackSelector(Parameters parameters, TrackSelection.Factory trackSelectionFactory) { + public DefaultTrackSelector( + Parameters parameters, ExoTrackSelection.Factory trackSelectionFactory) { this.trackSelectionFactory = trackSelectionFactory; parametersReference = new AtomicReference<>(parameters); } @@ -1574,27 +1696,20 @@ public ParametersBuilder buildUponParameters() { return getParameters().buildUpon(); } - /** - * Allows the creation of multiple adaptive track selections. - * - *

    This method is experimental, and will be renamed or removed in a future release. - */ - public void experimentalAllowMultipleAdaptiveSelections() { - this.allowMultipleAdaptiveSelections = true; - } - // MappingTrackSelector implementation. @Override - protected final Pair<@NullableType RendererConfiguration[], @NullableType TrackSelection[]> + protected final Pair<@NullableType RendererConfiguration[], @NullableType ExoTrackSelection[]> selectTracks( MappedTrackInfo mappedTrackInfo, @Capabilities int[][][] rendererFormatSupports, - @AdaptiveSupport int[] rendererMixedMimeTypeAdaptationSupports) + @AdaptiveSupport int[] rendererMixedMimeTypeAdaptationSupports, + MediaPeriodId mediaPeriodId, + Timeline timeline) throws ExoPlaybackException { Parameters params = parametersReference.get(); int rendererCount = mappedTrackInfo.getRendererCount(); - TrackSelection.@NullableType Definition[] definitions = + ExoTrackSelection.@NullableType Definition[] definitions = selectAllTracks( mappedTrackInfo, rendererFormatSupports, @@ -1613,7 +1728,7 @@ public void experimentalAllowMultipleAdaptiveSelections() { definitions[i] = override == null ? null - : new TrackSelection.Definition( + : new ExoTrackSelection.Definition( rendererTrackGroups.get(override.groupIndex), override.tracks, override.reason, @@ -1622,13 +1737,14 @@ public void experimentalAllowMultipleAdaptiveSelections() { } @NullableType - TrackSelection[] rendererTrackSelections = - trackSelectionFactory.createTrackSelections(definitions, getBandwidthMeter()); + ExoTrackSelection[] rendererTrackSelections = + trackSelectionFactory.createTrackSelections( + definitions, getBandwidthMeter(), mediaPeriodId, timeline); // Initialize the renderer configurations to the default configuration for all renderers with // selections, and null otherwise. - @NullableType RendererConfiguration[] rendererConfigurations = - new RendererConfiguration[rendererCount]; + @NullableType + RendererConfiguration[] rendererConfigurations = new RendererConfiguration[rendererCount]; for (int i = 0; i < rendererCount; i++) { boolean forceRendererDisabled = params.getRendererDisabled(i); boolean rendererEnabled = @@ -1639,12 +1755,10 @@ public void experimentalAllowMultipleAdaptiveSelections() { } // Configure audio and video renderers to use tunneling if appropriate. - maybeConfigureRenderersForTunneling( - mappedTrackInfo, - rendererFormatSupports, - rendererConfigurations, - rendererTrackSelections, - params.tunnelingAudioSessionId); + if (params.tunnelingEnabled) { + maybeConfigureRenderersForTunneling( + mappedTrackInfo, rendererFormatSupports, rendererConfigurations, rendererTrackSelections); + } return Pair.create(rendererConfigurations, rendererTrackSelections); } @@ -1652,8 +1766,9 @@ public void experimentalAllowMultipleAdaptiveSelections() { // Track selection prior to overrides and disabled flags being applied. /** - * Called from {@link #selectTracks(MappedTrackInfo, int[][][], int[])} to make a track selection - * for each renderer, prior to overrides and disabled flags being applied. + * Called from {@link #selectTracks(MappedTrackInfo, int[][][], int[], MediaPeriodId, Timeline)} + * to make a track selection for each renderer, prior to overrides and disabled flags being + * applied. * *

    The implementation should not account for overrides and disabled flags. Track selections * generated by this method will be overridden to account for these properties. @@ -1663,19 +1778,19 @@ public void experimentalAllowMultipleAdaptiveSelections() { * renderer, track group and track (in that order). * @param rendererMixedMimeTypeAdaptationSupports The {@link AdaptiveSupport} for mixed MIME type * adaptation for the renderer. - * @return The {@link TrackSelection.Definition}s for the renderers. A null entry indicates no + * @return The {@link ExoTrackSelection.Definition}s for the renderers. A null entry indicates no * selection was made. * @throws ExoPlaybackException If an error occurs while selecting the tracks. */ - protected TrackSelection.@NullableType Definition[] selectAllTracks( + protected ExoTrackSelection.@NullableType Definition[] selectAllTracks( MappedTrackInfo mappedTrackInfo, @Capabilities int[][][] rendererFormatSupports, @AdaptiveSupport int[] rendererMixedMimeTypeAdaptationSupports, Parameters params) throws ExoPlaybackException { int rendererCount = mappedTrackInfo.getRendererCount(); - TrackSelection.@NullableType Definition[] definitions = - new TrackSelection.Definition[rendererCount]; + ExoTrackSelection.@NullableType Definition[] definitions = + new ExoTrackSelection.Definition[rendererCount]; boolean seenVideoRendererWithMappedTracks = false; boolean selectedVideoTracks = false; @@ -1701,9 +1816,9 @@ public void experimentalAllowMultipleAdaptiveSelections() { for (int i = 0; i < rendererCount; i++) { if (C.TRACK_TYPE_AUDIO == mappedTrackInfo.getRendererType(i)) { boolean enableAdaptiveTrackSelection = - allowMultipleAdaptiveSelections || !seenVideoRendererWithMappedTracks; + params.allowMultipleAdaptiveSelections || !seenVideoRendererWithMappedTracks; @Nullable - Pair audioSelection = + Pair audioSelection = selectAudioTrack( mappedTrackInfo.getTrackGroups(i), rendererFormatSupports[i], @@ -1718,7 +1833,7 @@ public void experimentalAllowMultipleAdaptiveSelections() { // score. Clear the selection for that renderer. definitions[selectedAudioRendererIndex] = null; } - TrackSelection.Definition definition = audioSelection.first; + ExoTrackSelection.Definition definition = audioSelection.first; definitions[i] = definition; // We assume that audio tracks in the same group have matching language. selectedAudioLanguage = definition.group.getFormat(definition.tracks[0]).language; @@ -1739,7 +1854,7 @@ public void experimentalAllowMultipleAdaptiveSelections() { break; case C.TRACK_TYPE_TEXT: @Nullable - Pair textSelection = + Pair textSelection = selectTextTrack( mappedTrackInfo.getTrackGroups(i), rendererFormatSupports[i], @@ -1773,7 +1888,7 @@ public void experimentalAllowMultipleAdaptiveSelections() { /** * Called by {@link #selectAllTracks(MappedTrackInfo, int[][][], int[], Parameters)} to create a - * {@link TrackSelection} for a video renderer. + * {@link ExoTrackSelection} for a video renderer. * * @param groups The {@link TrackGroupArray} mapped to the renderer. * @param formatSupport The {@link Capabilities} for each mapped track, indexed by track group and @@ -1782,19 +1897,19 @@ public void experimentalAllowMultipleAdaptiveSelections() { * adaptation for the renderer. * @param params The selector's current constraint parameters. * @param enableAdaptiveTrackSelection Whether adaptive track selection is allowed. - * @return The {@link TrackSelection.Definition} for the renderer, or null if no selection was + * @return The {@link ExoTrackSelection.Definition} for the renderer, or null if no selection was * made. * @throws ExoPlaybackException If an error occurs while selecting the tracks. */ @Nullable - protected TrackSelection.Definition selectVideoTrack( + protected ExoTrackSelection.Definition selectVideoTrack( TrackGroupArray groups, @Capabilities int[][] formatSupport, @AdaptiveSupport int mixedMimeTypeAdaptationSupports, Parameters params, boolean enableAdaptiveTrackSelection) throws ExoPlaybackException { - TrackSelection.Definition definition = null; + ExoTrackSelection.Definition definition = null; if (!params.forceHighestSupportedBitrate && !params.forceLowestBitrate && enableAdaptiveTrackSelection) { @@ -1808,7 +1923,7 @@ protected TrackSelection.Definition selectVideoTrack( } @Nullable - private static TrackSelection.Definition selectAdaptiveVideoTrack( + private static ExoTrackSelection.Definition selectAdaptiveVideoTrack( TrackGroupArray groups, @Capabilities int[][] formatSupport, @AdaptiveSupport int mixedMimeTypeAdaptationSupports, @@ -1840,7 +1955,7 @@ private static TrackSelection.Definition selectAdaptiveVideoTrack( params.viewportHeight, params.viewportOrientationMayChange); if (adaptiveTracks.length > 0) { - return new TrackSelection.Definition(group, adaptiveTracks); + return new ExoTrackSelection.Definition(group, adaptiveTracks); } } return null; @@ -1866,8 +1981,9 @@ private static int[] getAdaptiveVideoTracksForGroup( return NO_TRACKS; } - List selectedTrackIndices = getViewportFilteredTrackIndices(group, viewportWidth, - viewportHeight, viewportOrientationMayChange); + List selectedTrackIndices = + getViewportFilteredTrackIndices( + group, viewportWidth, viewportHeight, viewportOrientationMayChange); if (selectedTrackIndices.size() < 2) { return NO_TRACKS; } @@ -2024,7 +2140,7 @@ private static boolean isSupportedAdaptiveVideoTrack( } @Nullable - private static TrackSelection.Definition selectFixedVideoTrack( + private static ExoTrackSelection.Definition selectFixedVideoTrack( TrackGroupArray groups, @Capabilities int[][] formatSupport, Parameters params) { int selectedTrackIndex = C.INDEX_UNSET; @Nullable TrackGroup selectedGroup = null; @@ -2044,8 +2160,8 @@ private static TrackSelection.Definition selectFixedVideoTrack( // Ignore trick-play tracks for now. continue; } - if (isSupported(trackFormatSupport[trackIndex], - params.exceedRendererCapabilitiesIfNecessary)) { + if (isSupported( + trackFormatSupport[trackIndex], params.exceedRendererCapabilitiesIfNecessary)) { VideoTrackScore trackScore = new VideoTrackScore( format, @@ -2067,14 +2183,14 @@ private static TrackSelection.Definition selectFixedVideoTrack( return selectedGroup == null ? null - : new TrackSelection.Definition(selectedGroup, selectedTrackIndex); + : new ExoTrackSelection.Definition(selectedGroup, selectedTrackIndex); } // Audio track selection implementation. /** * Called by {@link #selectAllTracks(MappedTrackInfo, int[][][], int[], Parameters)} to create a - * {@link TrackSelection} for an audio renderer. + * {@link ExoTrackSelection} for an audio renderer. * * @param groups The {@link TrackGroupArray} mapped to the renderer. * @param formatSupport The {@link Capabilities} for each mapped track, indexed by track group and @@ -2083,13 +2199,13 @@ private static TrackSelection.Definition selectFixedVideoTrack( * adaptation for the renderer. * @param params The selector's current constraint parameters. * @param enableAdaptiveTrackSelection Whether adaptive track selection is allowed. - * @return The {@link TrackSelection.Definition} and corresponding {@link AudioTrackScore}, or + * @return The {@link ExoTrackSelection.Definition} and corresponding {@link AudioTrackScore}, or * null if no selection was made. * @throws ExoPlaybackException If an error occurs while selecting the tracks. */ @SuppressWarnings("unused") @Nullable - protected Pair selectAudioTrack( + protected Pair selectAudioTrack( TrackGroupArray groups, @Capabilities int[][] formatSupport, @AdaptiveSupport int mixedMimeTypeAdaptationSupports, @@ -2103,8 +2219,8 @@ protected Pair selectAudioTrack( TrackGroup trackGroup = groups.get(groupIndex); @Capabilities int[] trackFormatSupport = formatSupport[groupIndex]; for (int trackIndex = 0; trackIndex < trackGroup.length; trackIndex++) { - if (isSupported(trackFormatSupport[trackIndex], - params.exceedRendererCapabilitiesIfNecessary)) { + if (isSupported( + trackFormatSupport[trackIndex], params.exceedRendererCapabilitiesIfNecessary)) { Format format = trackGroup.getFormat(trackIndex); AudioTrackScore trackScore = new AudioTrackScore(format, params, trackFormatSupport[trackIndex]); @@ -2127,7 +2243,7 @@ protected Pair selectAudioTrack( TrackGroup selectedGroup = groups.get(selectedGroupIndex); - TrackSelection.Definition definition = null; + ExoTrackSelection.Definition definition = null; if (!params.forceHighestSupportedBitrate && !params.forceLowestBitrate && enableAdaptiveTrackSelection) { @@ -2142,12 +2258,12 @@ protected Pair selectAudioTrack( params.allowAudioMixedSampleRateAdaptiveness, params.allowAudioMixedChannelCountAdaptiveness); if (adaptiveTracks.length > 1) { - definition = new TrackSelection.Definition(selectedGroup, adaptiveTracks); + definition = new ExoTrackSelection.Definition(selectedGroup, adaptiveTracks); } } if (definition == null) { // We didn't make an adaptive selection, so make a fixed one instead. - definition = new TrackSelection.Definition(selectedGroup, selectedTrackIndex); + definition = new ExoTrackSelection.Definition(selectedGroup, selectedTrackIndex); } return Pair.create(definition, Assertions.checkNotNull(selectedTrackScore)); @@ -2189,7 +2305,8 @@ private static boolean isSupportedAdaptiveAudioTrack( boolean allowMixedSampleRateAdaptiveness, boolean allowAudioMixedChannelCountAdaptiveness) { return isSupported(formatSupport, /* allowExceedsCapabilities= */ false) - && (format.bitrate == Format.NO_VALUE || format.bitrate <= maxAudioBitrate) + && format.bitrate != Format.NO_VALUE + && format.bitrate <= maxAudioBitrate && (allowAudioMixedChannelCountAdaptiveness || (format.channelCount != Format.NO_VALUE && format.channelCount == primaryFormat.channelCount)) @@ -2205,7 +2322,7 @@ private static boolean isSupportedAdaptiveAudioTrack( /** * Called by {@link #selectAllTracks(MappedTrackInfo, int[][][], int[], Parameters)} to create a - * {@link TrackSelection} for a text renderer. + * {@link ExoTrackSelection} for a text renderer. * * @param groups The {@link TrackGroupArray} mapped to the renderer. * @param formatSupport The {@link Capabilities} for each mapped track, indexed by track group and @@ -2213,12 +2330,12 @@ private static boolean isSupportedAdaptiveAudioTrack( * @param params The selector's current constraint parameters. * @param selectedAudioLanguage The language of the selected audio track. May be null if the * selected text track declares no language or no text track was selected. - * @return The {@link TrackSelection.Definition} and corresponding {@link TextTrackScore}, or null - * if no selection was made. + * @return The {@link ExoTrackSelection.Definition} and corresponding {@link TextTrackScore}, or + * null if no selection was made. * @throws ExoPlaybackException If an error occurs while selecting the tracks. */ @Nullable - protected Pair selectTextTrack( + protected Pair selectTextTrack( TrackGroupArray groups, @Capabilities int[][] formatSupport, Parameters params, @@ -2231,8 +2348,8 @@ protected Pair selectTextTrack( TrackGroup trackGroup = groups.get(groupIndex); @Capabilities int[] trackFormatSupport = formatSupport[groupIndex]; for (int trackIndex = 0; trackIndex < trackGroup.length; trackIndex++) { - if (isSupported(trackFormatSupport[trackIndex], - params.exceedRendererCapabilitiesIfNecessary)) { + if (isSupported( + trackFormatSupport[trackIndex], params.exceedRendererCapabilitiesIfNecessary)) { Format format = trackGroup.getFormat(trackIndex); TextTrackScore trackScore = new TextTrackScore( @@ -2249,7 +2366,7 @@ protected Pair selectTextTrack( return selectedGroup == null ? null : Pair.create( - new TrackSelection.Definition(selectedGroup, selectedTrackIndex), + new ExoTrackSelection.Definition(selectedGroup, selectedTrackIndex), Assertions.checkNotNull(selectedTrackScore)); } @@ -2257,18 +2374,18 @@ protected Pair selectTextTrack( /** * Called by {@link #selectAllTracks(MappedTrackInfo, int[][][], int[], Parameters)} to create a - * {@link TrackSelection} for a renderer whose type is neither video, audio or text. + * {@link ExoTrackSelection} for a renderer whose type is neither video, audio or text. * * @param trackType The type of the renderer. * @param groups The {@link TrackGroupArray} mapped to the renderer. * @param formatSupport The {@link Capabilities} for each mapped track, indexed by track group and * track (in that order). * @param params The selector's current constraint parameters. - * @return The {@link TrackSelection} for the renderer, or null if no selection was made. + * @return The {@link ExoTrackSelection} for the renderer, or null if no selection was made. * @throws ExoPlaybackException If an error occurs while selecting the tracks. */ @Nullable - protected TrackSelection.Definition selectOtherTrack( + protected ExoTrackSelection.Definition selectOtherTrack( int trackType, TrackGroupArray groups, @Capabilities int[][] formatSupport, Parameters params) throws ExoPlaybackException { @Nullable TrackGroup selectedGroup = null; @@ -2278,8 +2395,8 @@ protected TrackSelection.Definition selectOtherTrack( TrackGroup trackGroup = groups.get(groupIndex); @Capabilities int[] trackFormatSupport = formatSupport[groupIndex]; for (int trackIndex = 0; trackIndex < trackGroup.length; trackIndex++) { - if (isSupported(trackFormatSupport[trackIndex], - params.exceedRendererCapabilitiesIfNecessary)) { + if (isSupported( + trackFormatSupport[trackIndex], params.exceedRendererCapabilitiesIfNecessary)) { Format format = trackGroup.getFormat(trackIndex); OtherTrackScore trackScore = new OtherTrackScore(format, trackFormatSupport[trackIndex]); if (selectedTrackScore == null || trackScore.compareTo(selectedTrackScore) > 0) { @@ -2292,15 +2409,15 @@ protected TrackSelection.Definition selectOtherTrack( } return selectedGroup == null ? null - : new TrackSelection.Definition(selectedGroup, selectedTrackIndex); + : new ExoTrackSelection.Definition(selectedGroup, selectedTrackIndex); } // Utility methods. /** - * Determines whether tunneling should be enabled, replacing {@link RendererConfiguration}s in - * {@code rendererConfigurations} with configurations that enable tunneling on the appropriate - * renderers if so. + * Determines whether tunneling can be enabled, replacing {@link RendererConfiguration}s in {@code + * rendererConfigurations} with configurations that enable tunneling on the appropriate renderers + * if so. * * @param mappedTrackInfo Mapped track information. * @param renderererFormatSupports The {@link Capabilities} for each mapped track, indexed by @@ -2308,18 +2425,12 @@ protected TrackSelection.Definition selectOtherTrack( * @param rendererConfigurations The renderer configurations. Configurations may be replaced with * ones that enable tunneling as a result of this call. * @param trackSelections The renderer track selections. - * @param tunnelingAudioSessionId The audio session id to use when tunneling, or {@link - * C#AUDIO_SESSION_ID_UNSET} if tunneling should not be enabled. */ private static void maybeConfigureRenderersForTunneling( MappedTrackInfo mappedTrackInfo, @Capabilities int[][][] renderererFormatSupports, @NullableType RendererConfiguration[] rendererConfigurations, - @NullableType TrackSelection[] trackSelections, - int tunnelingAudioSessionId) { - if (tunnelingAudioSessionId == C.AUDIO_SESSION_ID_UNSET) { - return; - } + @NullableType ExoTrackSelection[] trackSelections) { // Check whether we can enable tunneling. To enable tunneling we require exactly one audio and // one video renderer to support tunneling and have a selection. int tunnelingAudioRendererIndex = -1; @@ -2327,7 +2438,7 @@ private static void maybeConfigureRenderersForTunneling( boolean enableTunneling = true; for (int i = 0; i < mappedTrackInfo.getRendererCount(); i++) { int rendererType = mappedTrackInfo.getRendererType(i); - TrackSelection trackSelection = trackSelections[i]; + ExoTrackSelection trackSelection = trackSelections[i]; if ((rendererType == C.TRACK_TYPE_AUDIO || rendererType == C.TRACK_TYPE_VIDEO) && trackSelection != null) { if (rendererSupportsTunneling( @@ -2353,23 +2464,25 @@ private static void maybeConfigureRenderersForTunneling( enableTunneling &= tunnelingAudioRendererIndex != -1 && tunnelingVideoRendererIndex != -1; if (enableTunneling) { RendererConfiguration tunnelingRendererConfiguration = - new RendererConfiguration(tunnelingAudioSessionId); + new RendererConfiguration(/* tunneling= */ true); rendererConfigurations[tunnelingAudioRendererIndex] = tunnelingRendererConfiguration; rendererConfigurations[tunnelingVideoRendererIndex] = tunnelingRendererConfiguration; } } /** - * Returns whether a renderer supports tunneling for a {@link TrackSelection}. + * Returns whether a renderer supports tunneling for a {@link ExoTrackSelection}. * * @param formatSupport The {@link Capabilities} for each track, indexed by group index and track * index (in that order). * @param trackGroups The {@link TrackGroupArray}s for the renderer. * @param selection The track selection. - * @return Whether the renderer supports tunneling for the {@link TrackSelection}. + * @return Whether the renderer supports tunneling for the {@link ExoTrackSelection}. */ private static boolean rendererSupportsTunneling( - @Capabilities int[][] formatSupport, TrackGroupArray trackGroups, TrackSelection selection) { + @Capabilities int[][] formatSupport, + TrackGroupArray trackGroups, + ExoTrackSelection selection) { if (selection == null) { return false; } @@ -2387,21 +2500,21 @@ private static boolean rendererSupportsTunneling( /** * Returns true if the {@link FormatSupport} in the given {@link Capabilities} is {@link - * RendererCapabilities#FORMAT_HANDLED} or if {@code allowExceedsCapabilities} is set and the - * format support is {@link RendererCapabilities#FORMAT_EXCEEDS_CAPABILITIES}. + * C#FORMAT_HANDLED} or if {@code allowExceedsCapabilities} is set and the format support is + * {@link C#FORMAT_EXCEEDS_CAPABILITIES}. * * @param formatSupport {@link Capabilities}. * @param allowExceedsCapabilities Whether to return true if {@link FormatSupport} is {@link - * RendererCapabilities#FORMAT_EXCEEDS_CAPABILITIES}. - * @return True if {@link FormatSupport} is {@link RendererCapabilities#FORMAT_HANDLED}, or if - * {@code allowExceedsCapabilities} is set and the format support is {@link - * RendererCapabilities#FORMAT_EXCEEDS_CAPABILITIES}. + * C#FORMAT_EXCEEDS_CAPABILITIES}. + * @return True if {@link FormatSupport} is {@link C#FORMAT_HANDLED}, or if {@code + * allowExceedsCapabilities} is set and the format support is {@link + * C#FORMAT_EXCEEDS_CAPABILITIES}. */ protected static boolean isSupported( @Capabilities int formatSupport, boolean allowExceedsCapabilities) { @FormatSupport int maskedSupport = RendererCapabilities.getFormatSupport(formatSupport); - return maskedSupport == RendererCapabilities.FORMAT_HANDLED || (allowExceedsCapabilities - && maskedSupport == RendererCapabilities.FORMAT_EXCEEDS_CAPABILITIES); + return maskedSupport == C.FORMAT_HANDLED + || (allowExceedsCapabilities && maskedSupport == C.FORMAT_EXCEEDS_CAPABILITIES); } /** @@ -2454,8 +2567,8 @@ protected static int getFormatLanguageScore( return 0; } - private static List getViewportFilteredTrackIndices(TrackGroup group, int viewportWidth, - int viewportHeight, boolean orientationMayChange) { + private static List getViewportFilteredTrackIndices( + TrackGroup group, int viewportWidth, int viewportHeight, boolean orientationMayChange) { // Initially include all indices. ArrayList selectedTrackIndices = new ArrayList<>(group.length); for (int i = 0; i < group.length; i++) { @@ -2474,8 +2587,9 @@ private static List getViewportFilteredTrackIndices(TrackGroup group, i // smallest to exceed the maximum size at which it can be displayed within the viewport. // We'll discard formats of higher resolution. if (format.width > 0 && format.height > 0) { - Point maxVideoSizeInViewport = getMaxVideoSizeInViewport(orientationMayChange, - viewportWidth, viewportHeight, format.width, format.height); + Point maxVideoSizeInViewport = + getMaxVideoSizeInViewport( + orientationMayChange, viewportWidth, viewportHeight, format.width, format.height); int videoPixels = format.width * format.height; if (format.width >= (int) (maxVideoSizeInViewport.x * FRACTION_TO_CONSIDER_FULLSCREEN) && format.height >= (int) (maxVideoSizeInViewport.y * FRACTION_TO_CONSIDER_FULLSCREEN) @@ -2505,8 +2619,12 @@ private static List getViewportFilteredTrackIndices(TrackGroup group, i * Given viewport dimensions and video dimensions, computes the maximum size of the video as it * will be rendered to fit inside of the viewport. */ - private static Point getMaxVideoSizeInViewport(boolean orientationMayChange, int viewportWidth, - int viewportHeight, int videoWidth, int videoHeight) { + private static Point getMaxVideoSizeInViewport( + boolean orientationMayChange, + int viewportWidth, + int viewportHeight, + int videoWidth, + int videoHeight) { if (orientationMayChange && (videoWidth > videoHeight) != (viewportWidth > viewportHeight)) { // Rotation is allowed, and the video will be larger in the rotated viewport. int tempViewportWidth = viewportWidth; @@ -2537,6 +2655,7 @@ protected static final class VideoTrackScore implements Comparable 0) { + bestLanguageIndex = i; + bestLanguageScore = score; + break; + } + } + preferredLanguageIndex = bestLanguageIndex; + preferredLanguageScore = bestLanguageScore; + preferredRoleFlagsScore = + Integer.bitCount(format.roleFlags & parameters.preferredAudioRoleFlags); isDefaultSelectionFlag = (format.selectionFlags & C.SELECTION_FLAG_DEFAULT) != 0; channelCount = format.channelCount; sampleRate = format.sampleRate; @@ -2633,20 +2781,29 @@ public AudioTrackScore(Format format, Parameters parameters, @Capabilities int f && (format.channelCount == Format.NO_VALUE || format.channelCount <= parameters.maxAudioChannelCount); String[] localeLanguages = Util.getSystemLanguageCodes(); - int bestMatchIndex = Integer.MAX_VALUE; - int bestMatchScore = 0; + int bestLocaleMatchIndex = Integer.MAX_VALUE; + int bestLocaleMatchScore = 0; for (int i = 0; i < localeLanguages.length; i++) { int score = getFormatLanguageScore( format, localeLanguages[i], /* allowUndeterminedFormatLanguage= */ false); if (score > 0) { - bestMatchIndex = i; - bestMatchScore = score; + bestLocaleMatchIndex = i; + bestLocaleMatchScore = score; + break; + } + } + localeLanguageMatchIndex = bestLocaleMatchIndex; + localeLanguageScore = bestLocaleMatchScore; + int bestMimeTypeMatchIndex = Integer.MAX_VALUE; + for (int i = 0; i < parameters.preferredAudioMimeTypes.size(); i++) { + if (format.sampleMimeType != null + && format.sampleMimeType.equals(parameters.preferredAudioMimeTypes.get(i))) { + bestMimeTypeMatchIndex = i; break; } } - localeLanguageMatchIndex = bestMatchIndex; - localeLanguageScore = bestMatchScore; + preferredMimeTypeMatchIndex = bestMimeTypeMatchIndex; } /** @@ -2666,8 +2823,17 @@ public int compareTo(AudioTrackScore other) { : FORMAT_VALUE_ORDERING.reverse(); return ComparisonChain.start() .compareFalseFirst(this.isWithinRendererCapabilities, other.isWithinRendererCapabilities) + .compare( + this.preferredLanguageIndex, + other.preferredLanguageIndex, + Ordering.natural().reverse()) .compare(this.preferredLanguageScore, other.preferredLanguageScore) + .compare(this.preferredRoleFlagsScore, other.preferredRoleFlagsScore) .compareFalseFirst(this.isWithinConstraints, other.isWithinConstraints) + .compare( + this.preferredMimeTypeMatchIndex, + other.preferredMimeTypeMatchIndex, + Ordering.natural().reverse()) .compare( this.bitrate, other.bitrate, @@ -2701,6 +2867,7 @@ protected static final class TextTrackScore implements Comparable preferredLanguages = + parameters.preferredTextLanguages.isEmpty() + ? ImmutableList.of("") + : parameters.preferredTextLanguages; + for (int i = 0; i < preferredLanguages.size(); i++) { + int score = + getFormatLanguageScore( + format, preferredLanguages.get(i), parameters.selectUndeterminedTextLanguage); + if (score > 0) { + bestLanguageIndex = i; + bestLanguageScore = score; + break; + } + } + preferredLanguageIndex = bestLanguageIndex; + preferredLanguageScore = bestLanguageScore; preferredRoleFlagsScore = Integer.bitCount(format.roleFlags & parameters.preferredTextRoleFlags); hasCaptionRoleFlags = @@ -2730,7 +2914,7 @@ public TextTrackScore( getFormatLanguageScore(format, selectedAudioLanguage, selectedAudioLanguageUndetermined); isWithinConstraints = preferredLanguageScore > 0 - || (parameters.preferredTextLanguage == null && preferredRoleFlagsScore > 0) + || (parameters.preferredTextLanguages.isEmpty() && preferredRoleFlagsScore > 0) || isDefault || (isForced && selectedAudioLanguageScore > 0); } @@ -2748,6 +2932,10 @@ public int compareTo(TextTrackScore other) { ComparisonChain.start() .compareFalseFirst( this.isWithinRendererCapabilities, other.isWithinRendererCapabilities) + .compare( + this.preferredLanguageIndex, + other.preferredLanguageIndex, + Ordering.natural().reverse()) .compare(this.preferredLanguageScore, other.preferredLanguageScore) .compare(this.preferredRoleFlagsScore, other.preferredRoleFlagsScore) .compareFalseFirst(this.isDefault, other.isDefault) diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelection.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/ExoTrackSelection.java similarity index 80% rename from library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelection.java rename to library/core/src/main/java/com/google/android/exoplayer2/trackselection/ExoTrackSelection.java index 5e703438f86..e6816ec8847 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelection.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/ExoTrackSelection.java @@ -18,6 +18,8 @@ import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.chunk.Chunk; import com.google.android.exoplayer2.source.chunk.MediaChunk; @@ -27,15 +29,12 @@ import org.checkerframework.checker.nullness.compatqual.NullableType; /** - * A track selection consisting of a static subset of selected tracks belonging to a {@link - * TrackGroup}, and a possibly varying individual selected track from the subset. - * - *

    Tracks belonging to the subset are exposed in decreasing bandwidth order. The individual - * selected track may change dynamically as a result of calling {@link #updateSelectedTrack(long, - * long, long, List, MediaChunkIterator[])} or {@link #evaluateQueueSize(long, List)}. This only - * happens between calls to {@link #enable()} and {@link #disable()}. + * A {@link TrackSelection} that can change the individually selected track as a result of calling + * {@link #updateSelectedTrack(long, long, long, List, MediaChunkIterator[])} or {@link + * #evaluateQueueSize(long, List)}. This only happens between calls to {@link #enable()} and {@link + * #disable()}. */ -public interface TrackSelection { +public interface ExoTrackSelection extends TrackSelection { /** Contains of a subset of selected tracks belonging to a {@link TrackGroup}. */ final class Definition { @@ -71,9 +70,7 @@ public Definition(TrackGroup group, int[] tracks, int reason, @Nullable Object d } } - /** - * Factory for {@link TrackSelection} instances. - */ + /** Factory for {@link ExoTrackSelection} instances. */ interface Factory { /** @@ -84,12 +81,18 @@ interface Factory { * * @param definitions A {@link Definition} array. May include null values. * @param bandwidthMeter A {@link BandwidthMeter} which can be used to select tracks. + * @param mediaPeriodId The {@link MediaPeriodId} of the period for which tracks are to be + * selected. + * @param timeline The {@link Timeline} holding the period for which tracks are to be selected. * @return The created selections. Must have the same length as {@code definitions} and may * include null values. */ @NullableType - TrackSelection[] createTrackSelections( - @NullableType Definition[] definitions, BandwidthMeter bandwidthMeter); + ExoTrackSelection[] createTrackSelections( + @NullableType Definition[] definitions, + BandwidthMeter bandwidthMeter, + MediaPeriodId mediaPeriodId, + Timeline timeline); } /** @@ -110,78 +113,23 @@ TrackSelection[] createTrackSelections( */ void disable(); - /** - * Returns the {@link TrackGroup} to which the selected tracks belong. - */ - TrackGroup getTrackGroup(); - - // Static subset of selected tracks. - - /** - * Returns the number of tracks in the selection. - */ - int length(); - - /** - * Returns the format of the track at a given index in the selection. - * - * @param index The index in the selection. - * @return The format of the selected track. - */ - Format getFormat(int index); - - /** - * Returns the index in the track group of the track at a given index in the selection. - * - * @param index The index in the selection. - * @return The index of the selected track. - */ - int getIndexInTrackGroup(int index); - - /** - * Returns the index in the selection of the track with the specified format. The format is - * located by identity so, for example, {@code selection.indexOf(selection.getFormat(index)) == - * index} even if multiple selected tracks have formats that contain the same values. - * - * @param format The format. - * @return The index in the selection, or {@link C#INDEX_UNSET} if the track with the specified - * format is not part of the selection. - */ - int indexOf(Format format); - - /** - * Returns the index in the selection of the track with the specified index in the track group. - * - * @param indexInTrackGroup The index in the track group. - * @return The index in the selection, or {@link C#INDEX_UNSET} if the track with the specified - * index is not part of the selection. - */ - int indexOf(int indexInTrackGroup); - // Individual selected track. - /** - * Returns the {@link Format} of the individual selected track. - */ + /** Returns the {@link Format} of the individual selected track. */ Format getSelectedFormat(); - /** - * Returns the index in the track group of the individual selected track. - */ + /** Returns the index in the track group of the individual selected track. */ int getSelectedIndexInTrackGroup(); - /** - * Returns the index of the selected track. - */ + /** Returns the index of the selected track. */ int getSelectedIndex(); - /** - * Returns the reason for the current track selection. - */ + /** Returns the reason for the current track selection. */ int getSelectionReason(); /** Returns optional data associated with the current track selection. */ - @Nullable Object getSelectionData(); + @Nullable + Object getSelectionData(); // Adaptation. @@ -189,7 +137,7 @@ TrackSelection[] createTrackSelections( * Called to notify the selection of the current playback speed. The playback speed may affect * adaptive track selection. * - * @param speed The playback speed. + * @param speed The factor by which playback is sped up. */ void onPlaybackSpeed(float speed); @@ -200,6 +148,22 @@ TrackSelection[] createTrackSelections( */ default void onDiscontinuity() {} + /** + * Called to notify when a rebuffer occurred. + * + *

    A rebuffer is defined to be caused by buffer depletion rather than a user action. Hence this + * method is not called during initial buffering or when buffering as a result of a seek + * operation. + */ + default void onRebuffer() {} + + /** + * Called to notify when the playback is paused or resumed. + * + * @param playWhenReady Whether playback will proceed when ready. + */ + default void onPlayWhenReadyChanged(boolean playWhenReady) {} + /** * Updates the selected track for sources that load media in discrete {@link MediaChunk}s. * diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/FixedTrackSelection.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/FixedTrackSelection.java index fefad00cbd1..62e9a3dc71b 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/FixedTrackSelection.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/FixedTrackSelection.java @@ -20,51 +20,13 @@ import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.chunk.MediaChunk; import com.google.android.exoplayer2.source.chunk.MediaChunkIterator; -import com.google.android.exoplayer2.upstream.BandwidthMeter; import java.util.List; -import org.checkerframework.checker.nullness.compatqual.NullableType; /** * A {@link TrackSelection} consisting of a single track. */ public final class FixedTrackSelection extends BaseTrackSelection { - /** - * @deprecated Don't use as adaptive track selection factory as it will throw when multiple tracks - * are selected. If you would like to disable adaptive selection in {@link - * DefaultTrackSelector}, enable the {@link - * DefaultTrackSelector.Parameters#forceHighestSupportedBitrate} flag instead. - */ - @Deprecated - public static final class Factory implements TrackSelection.Factory { - - private final int reason; - @Nullable private final Object data; - - public Factory() { - this.reason = C.SELECTION_REASON_UNKNOWN; - this.data = null; - } - - /** - * @param reason A reason for the track selection. - * @param data Optional data associated with the track selection. - */ - public Factory(int reason, @Nullable Object data) { - this.reason = reason; - this.data = data; - } - - @Override - public @NullableType TrackSelection[] createTrackSelections( - @NullableType Definition[] definitions, BandwidthMeter bandwidthMeter) { - return TrackSelectionUtil.createTrackSelectionsForDefinitions( - definitions, - definition -> - new FixedTrackSelection(definition.group, definition.tracks[0], reason, data)); - } - } - private final int reason; @Nullable private final Object data; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/MappingTrackSelector.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/MappingTrackSelector.java index 16c63353ee4..05ea4bb3c47 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/MappingTrackSelector.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/MappingTrackSelector.java @@ -22,12 +22,12 @@ import androidx.annotation.IntDef; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.C.FormatSupport; import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.Renderer; import com.google.android.exoplayer2.RendererCapabilities; import com.google.android.exoplayer2.RendererCapabilities.AdaptiveSupport; import com.google.android.exoplayer2.RendererCapabilities.Capabilities; -import com.google.android.exoplayer2.RendererCapabilities.FormatSupport; import com.google.android.exoplayer2.RendererConfiguration; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; @@ -43,14 +43,12 @@ /** * Base class for {@link TrackSelector}s that first establish a mapping between {@link TrackGroup}s - * and {@link Renderer}s, and then from that mapping create a {@link TrackSelection} for each + * and {@link Renderer}s, and then from that mapping create a {@link ExoTrackSelection} for each * renderer. */ public abstract class MappingTrackSelector extends TrackSelector { - /** - * Provides mapped track information for each renderer. - */ + /** Provides mapped track information for each renderer. */ public static final class MappedTrackInfo { /** @@ -71,29 +69,25 @@ public static final class MappedTrackInfo { public static final int RENDERER_SUPPORT_NO_TRACKS = 0; /** * The renderer has tracks mapped to it, but all are unsupported. In other words, {@link - * #getTrackSupport(int, int, int)} returns {@link RendererCapabilities#FORMAT_UNSUPPORTED_DRM}, - * {@link RendererCapabilities#FORMAT_UNSUPPORTED_SUBTYPE} or {@link - * RendererCapabilities#FORMAT_UNSUPPORTED_TYPE} for all tracks mapped to the renderer. + * #getTrackSupport(int, int, int)} returns {@link C#FORMAT_UNSUPPORTED_DRM}, {@link + * C#FORMAT_UNSUPPORTED_SUBTYPE} or {@link C#FORMAT_UNSUPPORTED_TYPE} for all tracks mapped to + * the renderer. */ public static final int RENDERER_SUPPORT_UNSUPPORTED_TRACKS = 1; /** * The renderer has tracks mapped to it and at least one is of a supported type, but all such * tracks exceed the renderer's capabilities. In other words, {@link #getTrackSupport(int, int, - * int)} returns {@link RendererCapabilities#FORMAT_EXCEEDS_CAPABILITIES} for at least one - * track mapped to the renderer, but does not return {@link - * RendererCapabilities#FORMAT_HANDLED} for any tracks mapped to the renderer. + * int)} returns {@link C#FORMAT_EXCEEDS_CAPABILITIES} for at least one track mapped to the + * renderer, but does not return {@link C#FORMAT_HANDLED} for any tracks mapped to the renderer. */ public static final int RENDERER_SUPPORT_EXCEEDS_CAPABILITIES_TRACKS = 2; /** * The renderer has tracks mapped to it, and at least one such track is playable. In other - * words, {@link #getTrackSupport(int, int, int)} returns {@link - * RendererCapabilities#FORMAT_HANDLED} for at least one track mapped to the renderer. + * words, {@link #getTrackSupport(int, int, int)} returns {@link C#FORMAT_HANDLED} for at least + * one track mapped to the renderer. */ public static final int RENDERER_SUPPORT_PLAYABLE_TRACKS = 3; - /** @deprecated Use {@link #getRendererCount()}. */ - @Deprecated public final int length; - private final int rendererCount; private final String[] rendererNames; private final int[] rendererTrackTypes; @@ -127,7 +121,6 @@ public static final class MappedTrackInfo { this.rendererMixedMimeTypeAdaptiveSupports = rendererMixedMimeTypeAdaptiveSupports; this.unmappedTrackGroups = unmappedTrackGroups; this.rendererCount = rendererTrackTypes.length; - this.length = rendererCount; } /** Returns the number of renderers. */ @@ -181,14 +174,14 @@ public int getRendererSupport(int rendererIndex) { for (@Capabilities int trackFormatSupport : trackGroupFormatSupport) { int trackRendererSupport; switch (RendererCapabilities.getFormatSupport(trackFormatSupport)) { - case RendererCapabilities.FORMAT_HANDLED: + case C.FORMAT_HANDLED: return RENDERER_SUPPORT_PLAYABLE_TRACKS; - case RendererCapabilities.FORMAT_EXCEEDS_CAPABILITIES: + case C.FORMAT_EXCEEDS_CAPABILITIES: trackRendererSupport = RENDERER_SUPPORT_EXCEEDS_CAPABILITIES_TRACKS; break; - case RendererCapabilities.FORMAT_UNSUPPORTED_TYPE: - case RendererCapabilities.FORMAT_UNSUPPORTED_SUBTYPE: - case RendererCapabilities.FORMAT_UNSUPPORTED_DRM: + case C.FORMAT_UNSUPPORTED_TYPE: + case C.FORMAT_UNSUPPORTED_SUBTYPE: + case C.FORMAT_UNSUPPORTED_DRM: trackRendererSupport = RENDERER_SUPPORT_UNSUPPORTED_TRACKS; break; default: @@ -200,13 +193,6 @@ public int getRendererSupport(int rendererIndex) { return bestRendererSupport; } - /** @deprecated Use {@link #getTypeSupport(int)}. */ - @Deprecated - @RendererSupport - public int getTrackTypeRendererSupport(int trackType) { - return getTypeSupport(trackType); - } - /** * Returns the extent to which tracks of a specified type are supported. This is the best level * of support obtained from {@link #getRendererSupport(int)} for all renderers that handle the @@ -227,13 +213,6 @@ public int getTypeSupport(int trackType) { return bestRendererSupport; } - /** @deprecated Use {@link #getTrackSupport(int, int, int)}. */ - @Deprecated - @FormatSupport - public int getTrackFormatSupport(int rendererIndex, int groupIndex, int trackIndex) { - return getTrackSupport(rendererIndex, groupIndex, trackIndex); - } - /** * Returns the extent to which an individual track is supported by the renderer. * @@ -252,14 +231,12 @@ public int getTrackSupport(int rendererIndex, int groupIndex, int trackIndex) { * Returns the extent to which a renderer supports adaptation between supported tracks in a * specified {@link TrackGroup}. * - *

    Tracks for which {@link #getTrackSupport(int, int, int)} returns {@link - * RendererCapabilities#FORMAT_HANDLED} are always considered. Tracks for which {@link - * #getTrackSupport(int, int, int)} returns {@link - * RendererCapabilities#FORMAT_EXCEEDS_CAPABILITIES} are also considered if {@code + *

    Tracks for which {@link #getTrackSupport(int, int, int)} returns {@link C#FORMAT_HANDLED} + * are always considered. Tracks for which {@link #getTrackSupport(int, int, int)} returns + * {@link C#FORMAT_EXCEEDS_CAPABILITIES} are also considered if {@code * includeCapabilitiesExceededTracks} is set to {@code true}. Tracks for which {@link - * #getTrackSupport(int, int, int)} returns {@link RendererCapabilities#FORMAT_UNSUPPORTED_DRM}, - * {@link RendererCapabilities#FORMAT_UNSUPPORTED_TYPE} or {@link - * RendererCapabilities#FORMAT_UNSUPPORTED_SUBTYPE} are never considered. + * #getTrackSupport(int, int, int)} returns {@link C#FORMAT_UNSUPPORTED_DRM}, {@link + * C#FORMAT_UNSUPPORTED_TYPE} or {@link C#FORMAT_UNSUPPORTED_SUBTYPE} are never considered. * * @param rendererIndex The renderer index. * @param groupIndex The index of the track group. @@ -276,9 +253,9 @@ public int getAdaptiveSupport( int trackIndexCount = 0; for (int i = 0; i < trackCount; i++) { @FormatSupport int fixedSupport = getTrackSupport(rendererIndex, groupIndex, i); - if (fixedSupport == RendererCapabilities.FORMAT_HANDLED + if (fixedSupport == C.FORMAT_HANDLED || (includeCapabilitiesExceededTracks - && fixedSupport == RendererCapabilities.FORMAT_EXCEEDS_CAPABILITIES)) { + && fixedSupport == C.FORMAT_EXCEEDS_CAPABILITIES)) { trackIndices[trackIndexCount++] = i; } } @@ -320,17 +297,10 @@ public int getAdaptiveSupport(int rendererIndex, int groupIndex, int[] trackIndi : adaptiveSupport; } - /** @deprecated Use {@link #getUnmappedTrackGroups()}. */ - @Deprecated - public TrackGroupArray getUnassociatedTrackGroups() { - return getUnmappedTrackGroups(); - } - /** Returns {@link TrackGroup}s not mapped to any renderer. */ public TrackGroupArray getUnmappedTrackGroups() { return unmappedTrackGroups; } - } @Nullable private MappedTrackInfo currentMappedTrackInfo; @@ -355,7 +325,7 @@ public final void onSelectionActivated(@Nullable Object info) { public final TrackSelectorResult selectTracks( RendererCapabilities[] rendererCapabilities, TrackGroupArray trackGroups, - MediaPeriodId periodId, + MediaPeriodId mediaPeriodId, Timeline timeline) throws ExoPlaybackException { // Structures into which data will be written during the selection. The extra item at the end @@ -429,9 +399,13 @@ public final TrackSelectorResult selectTracks( rendererFormatSupports, unmappedTrackGroupArray); - Pair<@NullableType RendererConfiguration[], @NullableType TrackSelection[]> result = + Pair<@NullableType RendererConfiguration[], @NullableType ExoTrackSelection[]> result = selectTracks( - mappedTrackInfo, rendererFormatSupports, rendererMixedMimeTypeAdaptationSupports); + mappedTrackInfo, + rendererFormatSupports, + rendererMixedMimeTypeAdaptationSupports, + mediaPeriodId, + timeline); return new TrackSelectorResult(result.first, result.second, mappedTrackInfo); } @@ -443,27 +417,30 @@ public final TrackSelectorResult selectTracks( * renderer, track group and track (in that order). * @param rendererMixedMimeTypeAdaptationSupport The {@link AdaptiveSupport} for mixed MIME type * adaptation for the renderer. + * @param mediaPeriodId The {@link MediaPeriodId} of the period for which tracks are to be + * selected. + * @param timeline The {@link Timeline} holding the period for which tracks are to be selected. * @return A pair consisting of the track selections and configurations for each renderer. A null * configuration indicates the renderer should be disabled, in which case the track selection * will also be null. A track selection may also be null for a non-disabled renderer if {@link * RendererCapabilities#getTrackType()} is {@link C#TRACK_TYPE_NONE}. * @throws ExoPlaybackException If an error occurs while selecting the tracks. */ - protected abstract Pair<@NullableType RendererConfiguration[], @NullableType TrackSelection[]> + protected abstract Pair<@NullableType RendererConfiguration[], @NullableType ExoTrackSelection[]> selectTracks( MappedTrackInfo mappedTrackInfo, @Capabilities int[][][] rendererFormatSupports, - @AdaptiveSupport int[] rendererMixedMimeTypeAdaptationSupport) + @AdaptiveSupport int[] rendererMixedMimeTypeAdaptationSupport, + MediaPeriodId mediaPeriodId, + Timeline timeline) throws ExoPlaybackException; /** * Finds the renderer to which the provided {@link TrackGroup} should be mapped. * *

    A {@link TrackGroup} is mapped to the renderer that reports the highest of (listed in - * decreasing order of support) {@link RendererCapabilities#FORMAT_HANDLED}, {@link - * RendererCapabilities#FORMAT_EXCEEDS_CAPABILITIES}, {@link - * RendererCapabilities#FORMAT_UNSUPPORTED_DRM} and {@link - * RendererCapabilities#FORMAT_UNSUPPORTED_SUBTYPE}. + * decreasing order of support) {@link C#FORMAT_HANDLED}, {@link C#FORMAT_EXCEEDS_CAPABILITIES}, + * {@link C#FORMAT_UNSUPPORTED_DRM} and {@link C#FORMAT_UNSUPPORTED_SUBTYPE}. * *

    In the case that two or more renderers report the same level of support, the assignment * depends on {@code preferUnassociatedRenderer}. @@ -476,9 +453,9 @@ public final TrackSelectorResult selectTracks( * available renderers have already mapped track groups. * * - *

    If all renderers report {@link RendererCapabilities#FORMAT_UNSUPPORTED_TYPE} for all of the - * tracks in the group, then {@code renderers.length} is returned to indicate that the group was - * not mapped to any renderer. + *

    If all renderers report {@link C#FORMAT_UNSUPPORTED_TYPE} for all of the tracks in the + * group, then {@code renderers.length} is returned to indicate that the group was not mapped to + * any renderer. * * @param rendererCapabilities The {@link RendererCapabilities} of the renderers. * @param group The track group to map to a renderer. @@ -496,11 +473,11 @@ private static int findRenderer( boolean preferUnassociatedRenderer) throws ExoPlaybackException { int bestRendererIndex = rendererCapabilities.length; - @FormatSupport int bestFormatSupportLevel = RendererCapabilities.FORMAT_UNSUPPORTED_TYPE; + @FormatSupport int bestFormatSupportLevel = C.FORMAT_UNSUPPORTED_TYPE; boolean bestRendererIsUnassociated = true; for (int rendererIndex = 0; rendererIndex < rendererCapabilities.length; rendererIndex++) { RendererCapabilities rendererCapability = rendererCapabilities[rendererIndex]; - @FormatSupport int formatSupportLevel = RendererCapabilities.FORMAT_UNSUPPORTED_TYPE; + @FormatSupport int formatSupportLevel = C.FORMAT_UNSUPPORTED_TYPE; for (int trackIndex = 0; trackIndex < group.length; trackIndex++) { @FormatSupport int trackFormatSupportLevel = @@ -559,5 +536,4 @@ private static int[] getMixedMimeTypeAdaptationSupports( } return mixedMimeTypeAdaptationSupport; } - } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/RandomTrackSelection.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/RandomTrackSelection.java index 4b9b72715a4..3dcb73de21f 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/RandomTrackSelection.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/RandomTrackSelection.java @@ -18,6 +18,8 @@ import android.os.SystemClock; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.chunk.MediaChunk; import com.google.android.exoplayer2.source.chunk.MediaChunkIterator; @@ -26,15 +28,11 @@ import java.util.Random; import org.checkerframework.checker.nullness.compatqual.NullableType; -/** - * A {@link TrackSelection} whose selected track is updated randomly. - */ +/** An {@link ExoTrackSelection} whose selected track is updated randomly. */ public final class RandomTrackSelection extends BaseTrackSelection { - /** - * Factory for {@link RandomTrackSelection} instances. - */ - public static final class Factory implements TrackSelection.Factory { + /** Factory for {@link RandomTrackSelection} instances. */ + public static final class Factory implements ExoTrackSelection.Factory { private final Random random; @@ -42,16 +40,17 @@ public Factory() { random = new Random(); } - /** - * @param seed A seed for the {@link Random} instance used by the factory. - */ + /** @param seed A seed for the {@link Random} instance used by the factory. */ public Factory(int seed) { random = new Random(seed); } @Override - public @NullableType TrackSelection[] createTrackSelections( - @NullableType Definition[] definitions, BandwidthMeter bandwidthMeter) { + public @NullableType ExoTrackSelection[] createTrackSelections( + @NullableType Definition[] definitions, + BandwidthMeter bandwidthMeter, + MediaPeriodId mediaPeriodId, + Timeline timeline) { return TrackSelectionUtil.createTrackSelectionsForDefinitions( definitions, definition -> new RandomTrackSelection(definition.group, definition.tracks, random)); @@ -139,5 +138,4 @@ public int getSelectionReason() { public Object getSelectionData() { return null; } - } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionParameters.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionParameters.java index 3871a31a3be..88719bc0abe 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionParameters.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionParameters.java @@ -15,16 +15,19 @@ */ package com.google.android.exoplayer2.trackselection; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; + import android.content.Context; import android.os.Looper; import android.os.Parcel; import android.os.Parcelable; -import android.text.TextUtils; import android.view.accessibility.CaptioningManager; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.ImmutableList; +import java.util.ArrayList; import java.util.Locale; /** Constraint parameters for track selection. */ @@ -36,8 +39,9 @@ public class TrackSelectionParameters implements Parcelable { */ public static class Builder { - @Nullable /* package */ String preferredAudioLanguage; - @Nullable /* package */ String preferredTextLanguage; + /* package */ ImmutableList preferredAudioLanguages; + @C.RoleFlags /* package */ int preferredAudioRoleFlags; + /* package */ ImmutableList preferredTextLanguages; @C.RoleFlags /* package */ int preferredTextRoleFlags; /* package */ boolean selectUndeterminedTextLanguage; @C.SelectionFlags /* package */ int disabledTextTrackSelectionFlags; @@ -59,8 +63,9 @@ public Builder(Context context) { */ @Deprecated public Builder() { - preferredAudioLanguage = null; - preferredTextLanguage = null; + preferredAudioLanguages = ImmutableList.of(); + preferredAudioRoleFlags = 0; + preferredTextLanguages = ImmutableList.of(); preferredTextRoleFlags = 0; selectUndeterminedTextLanguage = false; disabledTextTrackSelectionFlags = 0; @@ -71,8 +76,9 @@ public Builder() { * the builder are obtained. */ /* package */ Builder(TrackSelectionParameters initialValues) { - preferredAudioLanguage = initialValues.preferredAudioLanguage; - preferredTextLanguage = initialValues.preferredTextLanguage; + preferredAudioLanguages = initialValues.preferredAudioLanguages; + preferredAudioRoleFlags = initialValues.preferredAudioRoleFlags; + preferredTextLanguages = initialValues.preferredTextLanguages; preferredTextRoleFlags = initialValues.preferredTextRoleFlags; selectUndeterminedTextLanguage = initialValues.selectUndeterminedTextLanguage; disabledTextTrackSelectionFlags = initialValues.disabledTextTrackSelectionFlags; @@ -86,7 +92,36 @@ public Builder() { * @return This builder. */ public Builder setPreferredAudioLanguage(@Nullable String preferredAudioLanguage) { - this.preferredAudioLanguage = preferredAudioLanguage; + return preferredAudioLanguage == null + ? setPreferredAudioLanguages() + : setPreferredAudioLanguages(preferredAudioLanguage); + } + + /** + * Sets the preferred languages for audio and forced text tracks. + * + * @param preferredAudioLanguages Preferred audio languages as IETF BCP 47 conformant tags in + * order of preference, or an empty array to select the default track, or the first track if + * there's no default. + * @return This builder. + */ + public Builder setPreferredAudioLanguages(String... preferredAudioLanguages) { + ImmutableList.Builder listBuilder = ImmutableList.builder(); + for (String language : checkNotNull(preferredAudioLanguages)) { + listBuilder.add(Util.normalizeLanguageCode(checkNotNull(language))); + } + this.preferredAudioLanguages = listBuilder.build(); + return this; + } + + /** + * Sets the preferred {@link C.RoleFlags} for audio tracks. + * + * @param preferredAudioRoleFlags Preferred audio role flags. + * @return This builder. + */ + public Builder setPreferredAudioRoleFlags(@C.RoleFlags int preferredAudioRoleFlags) { + this.preferredAudioRoleFlags = preferredAudioRoleFlags; return this; } @@ -115,7 +150,25 @@ public Builder setPreferredTextLanguageAndRoleFlagsToCaptioningManagerSettings( * @return This builder. */ public Builder setPreferredTextLanguage(@Nullable String preferredTextLanguage) { - this.preferredTextLanguage = preferredTextLanguage; + return preferredTextLanguage == null + ? setPreferredTextLanguages() + : setPreferredTextLanguages(preferredTextLanguage); + } + + /** + * Sets the preferred languages for text tracks. + * + * @param preferredTextLanguages Preferred text languages as IETF BCP 47 conformant tags in + * order of preference, or an empty array to select the default track if there is one, or no + * track otherwise. + * @return This builder. + */ + public Builder setPreferredTextLanguages(String... preferredTextLanguages) { + ImmutableList.Builder listBuilder = ImmutableList.builder(); + for (String language : checkNotNull(preferredTextLanguages)) { + listBuilder.add(Util.normalizeLanguageCode(checkNotNull(language))); + } + this.preferredTextLanguages = listBuilder.build(); return this; } @@ -132,8 +185,8 @@ public Builder setPreferredTextRoleFlags(@C.RoleFlags int preferredTextRoleFlags /** * Sets whether a text track with undetermined language should be selected if no track with - * {@link #setPreferredTextLanguage(String)} is available, or if the preferred language is - * unset. + * {@link #setPreferredTextLanguages(String...) a preferred language} is available, or if the + * preferred language is unset. * * @param selectUndeterminedTextLanguage Whether a text track with undetermined language should * be selected if no preferred language track is available. @@ -161,9 +214,10 @@ public Builder setDisabledTextTrackSelectionFlags( public TrackSelectionParameters build() { return new TrackSelectionParameters( // Audio - preferredAudioLanguage, + preferredAudioLanguages, + preferredAudioRoleFlags, // Text - preferredTextLanguage, + preferredTextLanguages, preferredTextRoleFlags, selectUndeterminedTextLanguage, disabledTextTrackSelectionFlags); @@ -185,7 +239,7 @@ private void setPreferredTextLanguageAndRoleFlagsToCaptioningManagerSettingsV19( preferredTextRoleFlags = C.ROLE_FLAG_CAPTION | C.ROLE_FLAG_DESCRIBES_MUSIC_AND_SOUND; Locale preferredLocale = captioningManager.getLocale(); if (preferredLocale != null) { - preferredTextLanguage = Util.getLocaleLanguageTag(preferredLocale); + preferredTextLanguages = ImmutableList.of(Util.getLocaleLanguageTag(preferredLocale)); } } } @@ -218,17 +272,23 @@ public static TrackSelectionParameters getDefaults(Context context) { } /** - * The preferred language for audio and forced text tracks as an IETF BCP 47 conformant tag. - * {@code null} selects the default track, or the first track if there's no default. The default - * value is {@code null}. + * The preferred languages for audio and forced text tracks as IETF BCP 47 conformant tags in + * order of preference. An empty list selects the default track, or the first track if there's no + * default. The default value is an empty list. + */ + public final ImmutableList preferredAudioLanguages; + /** + * The preferred {@link C.RoleFlags} for audio tracks. {@code 0} selects the default track if + * there is one, or the first track if there's no default. The default value is {@code 0}. */ - @Nullable public final String preferredAudioLanguage; + @C.RoleFlags public final int preferredAudioRoleFlags; /** - * The preferred language for text tracks as an IETF BCP 47 conformant tag. {@code null} selects - * the default track if there is one, or no track otherwise. The default value is {@code null}, or - * the language of the accessibility {@link CaptioningManager} if enabled. + * The preferred languages for text tracks as IETF BCP 47 conformant tags in order of preference. + * An empty list selects the default track if there is one, or no track otherwise. The default + * value is an empty list, or the language of the accessibility {@link CaptioningManager} if + * enabled. */ - @Nullable public final String preferredTextLanguage; + public final ImmutableList preferredTextLanguages; /** * The preferred {@link C.RoleFlags} for text tracks. {@code 0} selects the default track if there * is one, or no track otherwise. The default value is {@code 0}, or {@link C#ROLE_FLAG_SUBTITLE} @@ -238,7 +298,7 @@ public static TrackSelectionParameters getDefaults(Context context) { @C.RoleFlags public final int preferredTextRoleFlags; /** * Whether a text track with undetermined language should be selected if no track with {@link - * #preferredTextLanguage} is available, or if {@link #preferredTextLanguage} is unset. The + * #preferredTextLanguages} is available, or if {@link #preferredTextLanguages} is unset. The * default value is {@code false}. */ public final boolean selectUndeterminedTextLanguage; @@ -249,23 +309,30 @@ public static TrackSelectionParameters getDefaults(Context context) { @C.SelectionFlags public final int disabledTextTrackSelectionFlags; /* package */ TrackSelectionParameters( - @Nullable String preferredAudioLanguage, - @Nullable String preferredTextLanguage, + ImmutableList preferredAudioLanguages, + @C.RoleFlags int preferredAudioRoleFlags, + ImmutableList preferredTextLanguages, @C.RoleFlags int preferredTextRoleFlags, boolean selectUndeterminedTextLanguage, @C.SelectionFlags int disabledTextTrackSelectionFlags) { // Audio - this.preferredAudioLanguage = Util.normalizeLanguageCode(preferredAudioLanguage); + this.preferredAudioLanguages = preferredAudioLanguages; + this.preferredAudioRoleFlags = preferredAudioRoleFlags; // Text - this.preferredTextLanguage = Util.normalizeLanguageCode(preferredTextLanguage); + this.preferredTextLanguages = preferredTextLanguages; this.preferredTextRoleFlags = preferredTextRoleFlags; this.selectUndeterminedTextLanguage = selectUndeterminedTextLanguage; this.disabledTextTrackSelectionFlags = disabledTextTrackSelectionFlags; } /* package */ TrackSelectionParameters(Parcel in) { - this.preferredAudioLanguage = in.readString(); - this.preferredTextLanguage = in.readString(); + ArrayList preferredAudioLanguages = new ArrayList<>(); + in.readList(preferredAudioLanguages, /* loader= */ null); + this.preferredAudioLanguages = ImmutableList.copyOf(preferredAudioLanguages); + this.preferredAudioRoleFlags = in.readInt(); + ArrayList preferredTextLanguages = new ArrayList<>(); + in.readList(preferredTextLanguages, /* loader= */ null); + this.preferredTextLanguages = ImmutableList.copyOf(preferredTextLanguages); this.preferredTextRoleFlags = in.readInt(); this.selectUndeterminedTextLanguage = Util.readBoolean(in); this.disabledTextTrackSelectionFlags = in.readInt(); @@ -286,8 +353,9 @@ public boolean equals(@Nullable Object obj) { return false; } TrackSelectionParameters other = (TrackSelectionParameters) obj; - return TextUtils.equals(preferredAudioLanguage, other.preferredAudioLanguage) - && TextUtils.equals(preferredTextLanguage, other.preferredTextLanguage) + return preferredAudioLanguages.equals(other.preferredAudioLanguages) + && preferredAudioRoleFlags == other.preferredAudioRoleFlags + && preferredTextLanguages.equals(other.preferredTextLanguages) && preferredTextRoleFlags == other.preferredTextRoleFlags && selectUndeterminedTextLanguage == other.selectUndeterminedTextLanguage && disabledTextTrackSelectionFlags == other.disabledTextTrackSelectionFlags; @@ -296,8 +364,9 @@ public boolean equals(@Nullable Object obj) { @Override public int hashCode() { int result = 1; - result = 31 * result + (preferredAudioLanguage == null ? 0 : preferredAudioLanguage.hashCode()); - result = 31 * result + (preferredTextLanguage == null ? 0 : preferredTextLanguage.hashCode()); + result = 31 * result + preferredAudioLanguages.hashCode(); + result = 31 * result + preferredAudioRoleFlags; + result = 31 * result + preferredTextLanguages.hashCode(); result = 31 * result + preferredTextRoleFlags; result = 31 * result + (selectUndeterminedTextLanguage ? 1 : 0); result = 31 * result + disabledTextTrackSelectionFlags; @@ -313,8 +382,9 @@ public int describeContents() { @Override public void writeToParcel(Parcel dest, int flags) { - dest.writeString(preferredAudioLanguage); - dest.writeString(preferredTextLanguage); + dest.writeList(preferredAudioLanguages); + dest.writeInt(preferredAudioRoleFlags); + dest.writeList(preferredTextLanguages); dest.writeInt(preferredTextRoleFlags); Util.writeBoolean(dest, selectUndeterminedTextLanguage); dest.writeInt(disabledTextTrackSelectionFlags); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionUtil.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionUtil.java index 0f2748b1ac8..0dac7259a34 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionUtil.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectionUtil.java @@ -18,7 +18,7 @@ import androidx.annotation.Nullable; import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.trackselection.DefaultTrackSelector.SelectionOverride; -import com.google.android.exoplayer2.trackselection.TrackSelection.Definition; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection.Definition; import org.checkerframework.checker.nullness.compatqual.NullableType; /** Track selection related utility methods. */ @@ -35,7 +35,7 @@ public interface AdaptiveTrackSelectionFactory { * @param trackSelectionDefinition A {@link Definition} for the track selection. * @return The created track selection. */ - TrackSelection createAdaptiveTrackSelection(Definition trackSelectionDefinition); + ExoTrackSelection createAdaptiveTrackSelection(Definition trackSelectionDefinition); } /** @@ -48,10 +48,10 @@ public interface AdaptiveTrackSelectionFactory { * @return The array of created track selection. For null entries in {@code definitions} returns * null values. */ - public static @NullableType TrackSelection[] createTrackSelectionsForDefinitions( + public static @NullableType ExoTrackSelection[] createTrackSelectionsForDefinitions( @NullableType Definition[] definitions, AdaptiveTrackSelectionFactory adaptiveTrackSelectionFactory) { - TrackSelection[] selections = new TrackSelection[definitions.length]; + ExoTrackSelection[] selections = new ExoTrackSelection[definitions.length]; boolean createdAdaptiveTrackSelection = false; for (int i = 0; i < definitions.length; i++) { Definition definition = definitions[i]; diff --git a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectorResult.java b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectorResult.java index 67623c2cf6b..e7f0caaedfc 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectorResult.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/trackselection/TrackSelectorResult.java @@ -20,9 +20,7 @@ import com.google.android.exoplayer2.util.Util; import org.checkerframework.checker.nullness.compatqual.NullableType; -/** - * The result of a {@link TrackSelector} operation. - */ +/** The result of a {@link TrackSelector} operation. */ public final class TrackSelectorResult { /** The number of selections in the result. Greater than or equal to zero. */ @@ -32,10 +30,8 @@ public final class TrackSelectorResult { * renderer should be disabled. */ public final @NullableType RendererConfiguration[] rendererConfigurations; - /** - * A {@link TrackSelectionArray} containing the track selection for each renderer. - */ - public final TrackSelectionArray selections; + /** A {@link ExoTrackSelection} array containing the track selection for each renderer. */ + public final @NullableType ExoTrackSelection[] selections; /** * An opaque object that will be returned to {@link TrackSelector#onSelectionActivated(Object)} * should the selections be activated. @@ -45,17 +41,17 @@ public final class TrackSelectorResult { /** * @param rendererConfigurations A {@link RendererConfiguration} for each renderer. A null entry * indicates the corresponding renderer should be disabled. - * @param selections A {@link TrackSelectionArray} containing the selection for each renderer. + * @param selections A {@link ExoTrackSelection} array containing the selection for each renderer. * @param info An opaque object that will be returned to {@link * TrackSelector#onSelectionActivated(Object)} should the selection be activated. May be * {@code null}. */ public TrackSelectorResult( @NullableType RendererConfiguration[] rendererConfigurations, - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, @Nullable Object info) { this.rendererConfigurations = rendererConfigurations; - this.selections = new TrackSelectionArray(selections); + this.selections = selections.clone(); this.info = info; length = rendererConfigurations.length; } @@ -100,7 +96,6 @@ public boolean isEquivalent(@Nullable TrackSelectorResult other, int index) { return false; } return Util.areEqual(rendererConfigurations[index], other.rendererConfigurations[index]) - && Util.areEqual(selections.get(index), other.selections.get(index)); + && Util.areEqual(selections[index], other.selections[index]); } - } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/BandwidthMeter.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/BandwidthMeter.java index d520fcfa60f..f35d745892b 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/BandwidthMeter.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/BandwidthMeter.java @@ -35,7 +35,7 @@ interface EventListener { * changed. * *

    Note: The estimated bitrate is typically derived from more information than just {@code - * bytes} and {@code elapsedMs}. + * bytesTransferred} and {@code elapsedMs}. * * @param elapsedMs The time taken to transfer {@code bytesTransferred}, in milliseconds. This * is at most the elapsed time since the last callback, but may be less if there were diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DataSchemeDataSource.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/DataSchemeDataSource.java index 2c3670f52a6..2b9cf00e479 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DataSchemeDataSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/DataSchemeDataSource.java @@ -59,7 +59,6 @@ public long open(DataSpec dataSpec) throws IOException { String dataString = uriParts[1]; if (uriParts[0].contains(";base64")) { try { - // TODO(internal: b/169937045): Consider passing Base64.URL_SAFE flag. data = Base64.decode(dataString, /* flags= */ Base64.DEFAULT); } catch (IllegalArgumentException e) { throw new ParserException("Error while parsing Base64 encoded string: " + dataString, e); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultDataSource.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultDataSource.java index 12fea3898cb..cf89180b431 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultDataSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultDataSource.java @@ -19,7 +19,6 @@ import android.content.Context; import android.net.Uri; import androidx.annotation.Nullable; -import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.Util; @@ -89,7 +88,7 @@ public final class DefaultDataSource implements DataSource { public DefaultDataSource(Context context, boolean allowCrossProtocolRedirects) { this( context, - ExoPlayerLibraryInfo.DEFAULT_USER_AGENT, + /* userAgent= */ null, DefaultHttpDataSource.DEFAULT_CONNECT_TIMEOUT_MILLIS, DefaultHttpDataSource.DEFAULT_READ_TIMEOUT_MILLIS, allowCrossProtocolRedirects); @@ -99,11 +98,13 @@ public DefaultDataSource(Context context, boolean allowCrossProtocolRedirects) { * Constructs a new instance, optionally configured to follow cross-protocol redirects. * * @param context A context. - * @param userAgent The User-Agent to use when requesting remote data. + * @param userAgent The user agent that will be used when requesting remote data, or {@code null} + * to use the default user agent of the underlying platform. * @param allowCrossProtocolRedirects Whether cross-protocol redirects (i.e. redirects from HTTP * to HTTPS and vice versa) are enabled when fetching remote data. */ - public DefaultDataSource(Context context, String userAgent, boolean allowCrossProtocolRedirects) { + public DefaultDataSource( + Context context, @Nullable String userAgent, boolean allowCrossProtocolRedirects) { this( context, userAgent, @@ -116,7 +117,8 @@ public DefaultDataSource(Context context, String userAgent, boolean allowCrossPr * Constructs a new instance, optionally configured to follow cross-protocol redirects. * * @param context A context. - * @param userAgent The User-Agent to use when requesting remote data. + * @param userAgent The user agent that will be used when requesting remote data, or {@code null} + * to use the default user agent of the underlying platform. * @param connectTimeoutMillis The connection timeout that should be used when requesting remote * data, in milliseconds. A timeout of zero is interpreted as an infinite timeout. * @param readTimeoutMillis The read timeout that should be used when requesting remote data, in @@ -126,18 +128,18 @@ public DefaultDataSource(Context context, String userAgent, boolean allowCrossPr */ public DefaultDataSource( Context context, - String userAgent, + @Nullable String userAgent, int connectTimeoutMillis, int readTimeoutMillis, boolean allowCrossProtocolRedirects) { this( context, - new DefaultHttpDataSource( - userAgent, - connectTimeoutMillis, - readTimeoutMillis, - allowCrossProtocolRedirects, - /* defaultRequestProperties= */ null)); + new DefaultHttpDataSource.Factory() + .setUserAgent(userAgent) + .setConnectTimeoutMs(connectTimeoutMillis) + .setReadTimeoutMs(readTimeoutMillis) + .setAllowCrossProtocolRedirects(allowCrossProtocolRedirects) + .createDataSource()); } /** diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultDataSourceFactory.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultDataSourceFactory.java index 68ce25c47fc..0c6d2105173 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultDataSourceFactory.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultDataSourceFactory.java @@ -15,8 +15,6 @@ */ package com.google.android.exoplayer2.upstream; -import static com.google.android.exoplayer2.ExoPlayerLibraryInfo.DEFAULT_USER_AGENT; - import android.content.Context; import androidx.annotation.Nullable; import com.google.android.exoplayer2.upstream.DataSource.Factory; @@ -37,16 +35,17 @@ public final class DefaultDataSourceFactory implements Factory { * @param context A context. */ public DefaultDataSourceFactory(Context context) { - this(context, DEFAULT_USER_AGENT, /* listener= */ null); + this(context, /* userAgent= */ (String) null, /* listener= */ null); } /** * Creates an instance. * * @param context A context. - * @param userAgent The User-Agent string that should be used. + * @param userAgent The user agent that will be used when requesting remote data, or {@code null} + * to use the default user agent of the underlying platform. */ - public DefaultDataSourceFactory(Context context, String userAgent) { + public DefaultDataSourceFactory(Context context, @Nullable String userAgent) { this(context, userAgent, /* listener= */ null); } @@ -54,12 +53,13 @@ public DefaultDataSourceFactory(Context context, String userAgent) { * Creates an instance. * * @param context A context. - * @param userAgent The User-Agent string that should be used. + * @param userAgent The user agent that will be used when requesting remote data, or {@code null} + * to use the default user agent of the underlying platform. * @param listener An optional listener. */ public DefaultDataSourceFactory( - Context context, String userAgent, @Nullable TransferListener listener) { - this(context, listener, new DefaultHttpDataSourceFactory(userAgent, listener)); + Context context, @Nullable String userAgent, @Nullable TransferListener listener) { + this(context, listener, new DefaultHttpDataSource.Factory().setUserAgent(userAgent)); } /** diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceFactory.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceFactory.java index 0a0650a4b1c..c23e569c6ce 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceFactory.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/DefaultHttpDataSourceFactory.java @@ -15,17 +15,15 @@ */ package com.google.android.exoplayer2.upstream; -import static com.google.android.exoplayer2.ExoPlayerLibraryInfo.DEFAULT_USER_AGENT; import androidx.annotation.Nullable; import com.google.android.exoplayer2.upstream.HttpDataSource.BaseFactory; -import com.google.android.exoplayer2.upstream.HttpDataSource.Factory; -import com.google.android.exoplayer2.util.Assertions; -/** A {@link Factory} that produces {@link DefaultHttpDataSource} instances. */ +/** @deprecated Use {@link DefaultHttpDataSource.Factory} instead. */ +@Deprecated public final class DefaultHttpDataSourceFactory extends BaseFactory { - private final String userAgent; + @Nullable private final String userAgent; @Nullable private final TransferListener listener; private final int connectTimeoutMillis; private final int readTimeoutMillis; @@ -37,7 +35,7 @@ public final class DefaultHttpDataSourceFactory extends BaseFactory { * timeout and disables cross-protocol redirects. */ public DefaultHttpDataSourceFactory() { - this(DEFAULT_USER_AGENT); + this(/* userAgent= */ null); } /** @@ -45,9 +43,10 @@ public DefaultHttpDataSourceFactory() { * connection timeout, {@link DefaultHttpDataSource#DEFAULT_READ_TIMEOUT_MILLIS} as the read * timeout and disables cross-protocol redirects. * - * @param userAgent The User-Agent string that should be used. + * @param userAgent The user agent that will be used, or {@code null} to use the default user + * agent of the underlying platform. */ - public DefaultHttpDataSourceFactory(String userAgent) { + public DefaultHttpDataSourceFactory(@Nullable String userAgent) { this(userAgent, null); } @@ -56,17 +55,20 @@ public DefaultHttpDataSourceFactory(String userAgent) { * connection timeout, {@link DefaultHttpDataSource#DEFAULT_READ_TIMEOUT_MILLIS} as the read * timeout and disables cross-protocol redirects. * - * @param userAgent The User-Agent string that should be used. + * @param userAgent The user agent that will be used, or {@code null} to use the default user + * agent of the underlying platform. * @param listener An optional listener. * @see #DefaultHttpDataSourceFactory(String, TransferListener, int, int, boolean) */ - public DefaultHttpDataSourceFactory(String userAgent, @Nullable TransferListener listener) { + public DefaultHttpDataSourceFactory( + @Nullable String userAgent, @Nullable TransferListener listener) { this(userAgent, listener, DefaultHttpDataSource.DEFAULT_CONNECT_TIMEOUT_MILLIS, DefaultHttpDataSource.DEFAULT_READ_TIMEOUT_MILLIS, false); } /** - * @param userAgent The User-Agent string that should be used. + * @param userAgent The user agent that will be used, or {@code null} to use the default user + * agent of the underlying platform. * @param connectTimeoutMillis The connection timeout that should be used when requesting remote * data, in milliseconds. A timeout of zero is interpreted as an infinite timeout. * @param readTimeoutMillis The read timeout that should be used when requesting remote data, in @@ -75,7 +77,7 @@ public DefaultHttpDataSourceFactory(String userAgent, @Nullable TransferListener * to HTTPS and vice versa) are enabled. */ public DefaultHttpDataSourceFactory( - String userAgent, + @Nullable String userAgent, int connectTimeoutMillis, int readTimeoutMillis, boolean allowCrossProtocolRedirects) { @@ -88,7 +90,8 @@ public DefaultHttpDataSourceFactory( } /** - * @param userAgent The User-Agent string that should be used. + * @param userAgent The user agent that will be used, or {@code null} to use the default user + * agent of the underlying platform. * @param listener An optional listener. * @param connectTimeoutMillis The connection timeout that should be used when requesting remote * data, in milliseconds. A timeout of zero is interpreted as an infinite timeout. @@ -98,18 +101,20 @@ public DefaultHttpDataSourceFactory( * to HTTPS and vice versa) are enabled. */ public DefaultHttpDataSourceFactory( - String userAgent, + @Nullable String userAgent, @Nullable TransferListener listener, int connectTimeoutMillis, int readTimeoutMillis, boolean allowCrossProtocolRedirects) { - this.userAgent = Assertions.checkNotEmpty(userAgent); + this.userAgent = userAgent; this.listener = listener; this.connectTimeoutMillis = connectTimeoutMillis; this.readTimeoutMillis = readTimeoutMillis; this.allowCrossProtocolRedirects = allowCrossProtocolRedirects; } + // Calls deprecated constructor. + @SuppressWarnings("deprecation") @Override protected DefaultHttpDataSource createDataSourceInternal( HttpDataSource.RequestProperties defaultRequestProperties) { diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/UdpDataSource.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/UdpDataSource.java index e2b8ba1b31a..2da837e788f 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/UdpDataSource.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/UdpDataSource.java @@ -31,25 +31,22 @@ /** A UDP {@link DataSource}. */ public final class UdpDataSource extends BaseDataSource { - /** - * Thrown when an error is encountered when trying to read from a {@link UdpDataSource}. - */ + /** Thrown when an error is encountered when trying to read from a {@link UdpDataSource}. */ public static final class UdpDataSourceException extends IOException { public UdpDataSourceException(IOException cause) { super(cause); } - } - /** - * The default maximum datagram packet size, in bytes. - */ + /** The default maximum datagram packet size, in bytes. */ public static final int DEFAULT_MAX_PACKET_SIZE = 2000; /** The default socket timeout, in milliseconds. */ public static final int DEFAULT_SOCKET_TIMEOUT_MILLIS = 8 * 1000; + public static final int UDP_PORT_UNSET = -1; + private final int socketTimeoutMillis; private final byte[] packetBuffer; private final DatagramPacket packet; @@ -175,4 +172,14 @@ public void close() { } } + /** + * Returns the local port number opened for the UDP connection, or {@link #UDP_PORT_UNSET} if no + * connection is open + */ + public int getLocalPort() { + if (socket == null) { + return UDP_PORT_UNSET; + } + return socket.getLocalPort(); + } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/Cache.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/Cache.java index c917929111b..eb782bd334f 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/Cache.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/Cache.java @@ -178,10 +178,10 @@ public CacheException(String message, Throwable cause) { * @param key The cache key of the resource. * @param position The starting position in the resource from which data is required. * @param length The length of the data being requested, or {@link C#LENGTH_UNSET} if unbounded. - * The length is ignored in the case of a cache hit. In the case of a cache miss, it defines - * the maximum length of the hole {@link CacheSpan} that's returned. Cache implementations may - * support parallel writes into non-overlapping holes, and so passing the actual required - * length should be preferred to passing {@link C#LENGTH_UNSET} when possible. + * The length is ignored if there is a cache entry that overlaps the position. Else, it + * defines the maximum length of the hole {@link CacheSpan} that's returned. Cache + * implementations may support parallel writes into non-overlapping holes, and so passing the + * actual required length should be preferred to passing {@link C#LENGTH_UNSET} when possible. * @return The {@link CacheSpan}. * @throws InterruptedException If the thread was interrupted. * @throws CacheException If an error is encountered. @@ -199,8 +199,8 @@ CacheSpan startReadWrite(String key, long position, long length) * @param key The cache key of the resource. * @param position The starting position in the resource from which data is required. * @param length The length of the data being requested, or {@link C#LENGTH_UNSET} if unbounded. - * The length is ignored in the case of a cache hit. In the case of a cache miss, it defines - * the range of data locked by the returned {@link CacheSpan}. + * The length is ignored if there is a cache entry that overlaps the position. Else, it + * defines the range of data locked by the returned {@link CacheSpan}. * @return The {@link CacheSpan}. Or null if the cache entry is locked. * @throws CacheException If an error is encountered. */ diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/CachedContentIndex.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/CachedContentIndex.java index 850ac59f048..9f4fcd4c11f 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/CachedContentIndex.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/CachedContentIndex.java @@ -38,6 +38,7 @@ import com.google.android.exoplayer2.util.AtomicFile; import com.google.android.exoplayer2.util.ReusableBufferedOutputStream; import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.ImmutableSet; import java.io.BufferedInputStream; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; @@ -301,9 +302,8 @@ public void maybeRemove(String key) { /** Removes all resources whose {@link CachedContent CachedContents} are empty and unlocked. */ public void removeEmpty() { - String[] keys = new String[keyToContent.size()]; - keyToContent.keySet().toArray(keys); - for (String key : keys) { + // Create a copy of the keys as the underlying map is modified by maybeRemove(key). + for (String key : ImmutableSet.copyOf(keyToContent.keySet())) { maybeRemove(key); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/SimpleCache.java b/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/SimpleCache.java index cc1e5a8e5ed..0f1da881156 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/SimpleCache.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/upstream/cache/SimpleCache.java @@ -837,7 +837,8 @@ private static long parseUid(String fileName) { } private static void createCacheDirectories(File cacheDir) throws CacheException { - if (!cacheDir.mkdirs()) { + // If mkdirs() returns false, double check that the directory doesn't exist before throwing. + if (!cacheDir.mkdirs() && !cacheDir.isDirectory()) { String message = "Failed to create cache directory: " + cacheDir; Log.e(TAG, message); throw new CacheException(message); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/util/EventLogger.java b/library/core/src/main/java/com/google/android/exoplayer2/util/EventLogger.java index 53afd7a7a3f..6e25f1f0a21 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/util/EventLogger.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/util/EventLogger.java @@ -34,6 +34,7 @@ import com.google.android.exoplayer2.analytics.AnalyticsListener; import com.google.android.exoplayer2.audio.AudioAttributes; import com.google.android.exoplayer2.decoder.DecoderCounters; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.source.LoadEventInfo; import com.google.android.exoplayer2.source.MediaLoadData; @@ -45,6 +46,7 @@ import com.google.android.exoplayer2.trackselection.TrackSelectionArray; import java.io.IOException; import java.text.NumberFormat; +import java.util.List; import java.util.Locale; /** Logs events from {@link Player} and other core components using {@link Log}. */ @@ -237,7 +239,7 @@ public void onTracksChanged( for (int trackIndex = 0; trackIndex < trackGroup.length; trackIndex++) { String status = getTrackStatusString(trackSelection, trackGroup, trackIndex); String formatSupport = - RendererCapabilities.getFormatSupportString( + C.getFormatSupportString( mappedTrackInfo.getTrackSupport(rendererIndex, groupIndex, trackIndex)); logd( " " @@ -275,9 +277,7 @@ public void onTracksChanged( TrackGroup trackGroup = unassociatedTrackGroups.get(groupIndex); for (int trackIndex = 0; trackIndex < trackGroup.length; trackIndex++) { String status = getTrackStatusString(false); - String formatSupport = - RendererCapabilities.getFormatSupportString( - RendererCapabilities.FORMAT_UNSUPPORTED_TYPE); + String formatSupport = C.getFormatSupportString(C.FORMAT_UNSUPPORTED_TYPE); logd( " " + status @@ -295,6 +295,20 @@ public void onTracksChanged( logd("]"); } + @Override + public void onStaticMetadataChanged(EventTime eventTime, List metadataList) { + logd("staticMetadata [" + getEventTimeString(eventTime)); + for (int i = 0; i < metadataList.size(); i++) { + Metadata metadata = metadataList.get(i); + if (metadata.length() != 0) { + logd(" Metadata:" + i + " ["); + printMetadata(metadata, " "); + logd(" ]"); + } + } + logd("]"); + } + @Override public void onMetadata(EventTime eventTime, Metadata metadata) { logd("metadata [" + getEventTimeString(eventTime)); @@ -314,7 +328,8 @@ public void onAudioDecoderInitialized( } @Override - public void onAudioInputFormatChanged(EventTime eventTime, Format format) { + public void onAudioInputFormatChanged( + EventTime eventTime, Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { logd(eventTime, "audioInputFormat", Format.toLogString(format)); } @@ -328,13 +343,18 @@ public void onAudioUnderrun( /* throwable= */ null); } + @Override + public void onAudioDecoderReleased(EventTime eventTime, String decoderName) { + logd(eventTime, "audioDecoderReleased", decoderName); + } + @Override public void onAudioDisabled(EventTime eventTime, DecoderCounters counters) { logd(eventTime, "audioDisabled"); } @Override - public void onAudioSessionId(EventTime eventTime, int audioSessionId) { + public void onAudioSessionIdChanged(EventTime eventTime, int audioSessionId) { logd(eventTime, "audioSessionId", Integer.toString(audioSessionId)); } @@ -374,7 +394,8 @@ public void onVideoDecoderInitialized( } @Override - public void onVideoInputFormatChanged(EventTime eventTime, Format format) { + public void onVideoInputFormatChanged( + EventTime eventTime, Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { logd(eventTime, "videoInputFormat", Format.toLogString(format)); } @@ -383,6 +404,11 @@ public void onDroppedVideoFrames(EventTime eventTime, int count, long elapsedMs) logd(eventTime, "droppedFrames", Integer.toString(count)); } + @Override + public void onVideoDecoderReleased(EventTime eventTime, String decoderName) { + logd(eventTime, "videoDecoderReleased", decoderName); + } + @Override public void onVideoDisabled(EventTime eventTime, DecoderCounters counters) { logd(eventTime, "videoDisabled"); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/DecoderVideoRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/video/DecoderVideoRenderer.java index 6fda6d2e9c9..5310bfd624b 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/video/DecoderVideoRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/video/DecoderVideoRenderer.java @@ -15,6 +15,9 @@ */ package com.google.android.exoplayer2.video; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_DRM_SESSION_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_REUSE_NOT_IMPLEMENTED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_NO; import static java.lang.Math.max; import android.os.Handler; @@ -34,6 +37,7 @@ import com.google.android.exoplayer2.decoder.DecoderCounters; import com.google.android.exoplayer2.decoder.DecoderException; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.drm.DrmSession; import com.google.android.exoplayer2.drm.DrmSession.DrmSessionException; import com.google.android.exoplayer2.drm.ExoMediaCrypto; @@ -97,9 +101,12 @@ public abstract class DecoderVideoRenderer extends BaseRenderer { private Format inputFormat; private Format outputFormat; + + @Nullable private Decoder< VideoDecoderInputBuffer, ? extends VideoDecoderOutputBuffer, ? extends DecoderException> decoder; + private VideoDecoderInputBuffer inputBuffer; private VideoDecoderOutputBuffer outputBuffer; @Nullable private Surface surface; @@ -311,22 +318,6 @@ protected void onStreamChanged(Format[] formats, long startPositionUs, long offs super.onStreamChanged(formats, startPositionUs, offsetUs); } - /** - * Called when a decoder has been created and configured. - * - *

    The default implementation is a no-op. - * - * @param name The name of the decoder that was initialized. - * @param initializedTimestampMs {@link SystemClock#elapsedRealtime()} when initialization - * finished. - * @param initializationDurationMs The time taken to initialize the decoder, in milliseconds. - */ - @CallSuper - protected void onDecoderInitialized( - String name, long initializedTimestampMs, long initializationDurationMs) { - eventDispatcher.decoderInitialized(name, initializedTimestampMs, initializationDurationMs); - } - /** * Flushes the decoder. * @@ -358,9 +349,10 @@ protected void releaseDecoder() { decoderReceivedBuffers = false; buffersInCodecCount = 0; if (decoder != null) { + decoderCounters.decoderReleaseCount++; decoder.release(); + eventDispatcher.decoderReleased(decoder.getName()); decoder = null; - decoderCounters.decoderReleaseCount++; } setDecoderDrmSession(null); } @@ -381,7 +373,24 @@ protected void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybac if (decoder == null) { maybeInitDecoder(); - } else if (sourceDrmSession != decoderDrmSession || !canKeepCodec(oldFormat, inputFormat)) { + eventDispatcher.inputFormatChanged(inputFormat, /* decoderReuseEvaluation= */ null); + return; + } + + DecoderReuseEvaluation evaluation; + if (sourceDrmSession != decoderDrmSession) { + evaluation = + new DecoderReuseEvaluation( + decoder.getName(), + oldFormat, + newFormat, + REUSE_RESULT_NO, + DISCARD_REASON_DRM_SESSION_CHANGED); + } else { + evaluation = canReuseDecoder(decoder.getName(), oldFormat, newFormat); + } + + if (evaluation.result == REUSE_RESULT_NO) { if (decoderReceivedBuffers) { // Signal end of stream and wait for any final output buffers before re-initialization. decoderReinitializationState = REINITIALIZATION_STATE_SIGNAL_END_OF_STREAM; @@ -391,8 +400,7 @@ protected void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybac maybeInitDecoder(); } } - - eventDispatcher.inputFormatChanged(inputFormat); + eventDispatcher.inputFormatChanged(inputFormat, evaluation); } /** @@ -641,14 +649,18 @@ protected final void setOutputBufferRenderer( protected abstract void setDecoderOutputMode(@C.VideoOutputMode int outputMode); /** - * Returns whether the existing decoder can be kept for a new format. + * Evaluates whether the existing decoder can be reused for a new {@link Format}. + * + *

    The default implementation does not allow decoder reuse. * * @param oldFormat The previous format. * @param newFormat The new format. - * @return Whether the existing decoder can be kept. + * @return The result of the evaluation. */ - protected boolean canKeepCodec(Format oldFormat, Format newFormat) { - return false; + protected DecoderReuseEvaluation canReuseDecoder( + String decoderName, Format oldFormat, Format newFormat) { + return new DecoderReuseEvaluation( + decoderName, oldFormat, newFormat, REUSE_RESULT_NO, DISCARD_REASON_REUSE_NOT_IMPLEMENTED); } // Internal methods. @@ -676,8 +688,8 @@ private void maybeInitDecoder() throws ExoPlaybackException { if (mediaCrypto == null) { DrmSessionException drmError = decoderDrmSession.getError(); if (drmError != null) { - // Continue for now. We may be able to avoid failure if the session recovers, or if a new - // input format causes the session to be replaced before it's used. + // Continue for now. We may be able to avoid failure if a new input format causes the + // session to be replaced without it having been used. } else { // The drm session isn't open yet. return; @@ -690,12 +702,12 @@ private void maybeInitDecoder() throws ExoPlaybackException { decoder = createDecoder(inputFormat, mediaCrypto); setDecoderOutputMode(outputMode); long decoderInitializedTimestamp = SystemClock.elapsedRealtime(); - onDecoderInitialized( + eventDispatcher.decoderInitialized( decoder.getName(), decoderInitializedTimestamp, decoderInitializedTimestamp - decoderInitializingTimestamp); decoderCounters.decoderInitCount++; - } catch (DecoderException e) { + } catch (DecoderException | OutOfMemoryError e) { throw createRendererException(e, inputFormat); } } diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/FixedFrameRateEstimator.java b/library/core/src/main/java/com/google/android/exoplayer2/video/FixedFrameRateEstimator.java new file mode 100644 index 00000000000..a4315327f3e --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/video/FixedFrameRateEstimator.java @@ -0,0 +1,220 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.video; + +import androidx.annotation.VisibleForTesting; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import java.util.Arrays; + +/** + * Attempts to detect and refine a fixed frame rate estimate based on frame presentation timestamps. + */ +/* package */ final class FixedFrameRateEstimator { + + /** The number of consecutive matching frame durations required to detect a fixed frame rate. */ + public static final int CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC = 15; + /** + * The maximum amount frame durations can differ for them to be considered matching, in + * nanoseconds. + * + *

    This constant is set to 1ms to account for container formats that only represent frame + * presentation timestamps to the nearest millisecond. In such cases, frame durations need to + * switch between values that are 1ms apart to achieve common fixed frame rates (e.g., 30fps + * content will need frames that are 33ms and 34ms). + */ + @VisibleForTesting static final long MAX_MATCHING_FRAME_DIFFERENCE_NS = 1_000_000; + + private Matcher currentMatcher; + private Matcher candidateMatcher; + private boolean candidateMatcherActive; + private boolean switchToCandidateMatcherWhenSynced; + private long lastFramePresentationTimeNs; + private int framesWithoutSyncCount; + + public FixedFrameRateEstimator() { + currentMatcher = new Matcher(); + candidateMatcher = new Matcher(); + lastFramePresentationTimeNs = C.TIME_UNSET; + } + + /** Resets the estimator. */ + public void reset() { + currentMatcher.reset(); + candidateMatcher.reset(); + candidateMatcherActive = false; + lastFramePresentationTimeNs = C.TIME_UNSET; + framesWithoutSyncCount = 0; + } + + /** + * Called with each frame presentation timestamp. + * + * @param framePresentationTimeNs The frame presentation timestamp, in nanoseconds. + */ + public void onNextFrame(long framePresentationTimeNs) { + currentMatcher.onNextFrame(framePresentationTimeNs); + if (currentMatcher.isSynced() && !switchToCandidateMatcherWhenSynced) { + candidateMatcherActive = false; + } else if (lastFramePresentationTimeNs != C.TIME_UNSET) { + if (!candidateMatcherActive || candidateMatcher.isLastFrameOutlier()) { + // Reset the candidate with the last and current frame presentation timestamps, so that it + // will try and match against the duration of the previous frame. + candidateMatcher.reset(); + candidateMatcher.onNextFrame(lastFramePresentationTimeNs); + } + candidateMatcherActive = true; + candidateMatcher.onNextFrame(framePresentationTimeNs); + } + if (candidateMatcherActive && candidateMatcher.isSynced()) { + // The candidate matcher should be promoted to be the current matcher. The current matcher + // can be re-used as the next candidate matcher. + Matcher previousMatcher = currentMatcher; + currentMatcher = candidateMatcher; + candidateMatcher = previousMatcher; + candidateMatcherActive = false; + switchToCandidateMatcherWhenSynced = false; + } + lastFramePresentationTimeNs = framePresentationTimeNs; + framesWithoutSyncCount = currentMatcher.isSynced() ? 0 : framesWithoutSyncCount + 1; + } + + /** Returns whether the estimator has detected a fixed frame rate. */ + public boolean isSynced() { + return currentMatcher.isSynced(); + } + + /** Returns the number of frames since the estimator last detected a fixed frame rate. */ + public int getFramesWithoutSyncCount() { + return framesWithoutSyncCount; + } + + /** + * Returns the sum of all frame durations used to calculate the current fixed frame rate estimate, + * or {@link C#TIME_UNSET} if {@link #isSynced()} is {@code false}. + */ + public long getMatchingFrameDurationSumNs() { + return isSynced() ? currentMatcher.getMatchingFrameDurationSumNs() : C.TIME_UNSET; + } + + /** + * The currently detected fixed frame duration estimate in nanoseconds, or {@link C#TIME_UNSET} if + * {@link #isSynced()} is {@code false}. Whilst synced, the estimate is refined each time {@link + * #onNextFrame} is called with a new frame presentation timestamp. + */ + public long getFrameDurationNs() { + return isSynced() ? currentMatcher.getFrameDurationNs() : C.TIME_UNSET; + } + + /** + * The currently detected fixed frame rate estimate, or {@link Format#NO_VALUE} if {@link + * #isSynced()} is {@code false}. Whilst synced, the estimate is refined each time {@link + * #onNextFrame} is called with a new frame presentation timestamp. + */ + public float getFrameRate() { + return isSynced() + ? (float) ((double) C.NANOS_PER_SECOND / currentMatcher.getFrameDurationNs()) + : Format.NO_VALUE; + } + + /** Tries to match frame durations against the duration of the first frame it receives. */ + private static final class Matcher { + + private long firstFramePresentationTimeNs; + private long firstFrameDurationNs; + private long lastFramePresentationTimeNs; + private long frameCount; + + /** The total number of frames that have matched the frame duration being tracked. */ + private long matchingFrameCount; + /** The sum of the frame durations of all matching frames. */ + private long matchingFrameDurationSumNs; + /** Cyclic buffer of flags indicating whether the most recent frame durations were outliers. */ + private final boolean[] recentFrameOutlierFlags; + /** + * The number of recent frame durations that were outliers. Equal to the number of {@code true} + * values in {@link #recentFrameOutlierFlags}. + */ + private int recentFrameOutlierCount; + + public Matcher() { + recentFrameOutlierFlags = new boolean[CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC]; + } + + public void reset() { + frameCount = 0; + matchingFrameCount = 0; + matchingFrameDurationSumNs = 0; + recentFrameOutlierCount = 0; + Arrays.fill(recentFrameOutlierFlags, false); + } + + public boolean isSynced() { + return frameCount > CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC + && recentFrameOutlierCount == 0; + } + + public boolean isLastFrameOutlier() { + if (frameCount == 0) { + return false; + } + return recentFrameOutlierFlags[getRecentFrameOutlierIndex(frameCount - 1)]; + } + + public long getMatchingFrameDurationSumNs() { + return matchingFrameDurationSumNs; + } + + public long getFrameDurationNs() { + return matchingFrameCount == 0 ? 0 : (matchingFrameDurationSumNs / matchingFrameCount); + } + + public void onNextFrame(long framePresentationTimeNs) { + if (frameCount == 0) { + firstFramePresentationTimeNs = framePresentationTimeNs; + } else if (frameCount == 1) { + // This is the frame duration that the tracker will match against. + firstFrameDurationNs = framePresentationTimeNs - firstFramePresentationTimeNs; + matchingFrameDurationSumNs = firstFrameDurationNs; + matchingFrameCount = 1; + } else { + long lastFrameDurationNs = framePresentationTimeNs - lastFramePresentationTimeNs; + int recentFrameOutlierIndex = getRecentFrameOutlierIndex(frameCount); + if (Math.abs(lastFrameDurationNs - firstFrameDurationNs) + <= MAX_MATCHING_FRAME_DIFFERENCE_NS) { + matchingFrameCount++; + matchingFrameDurationSumNs += lastFrameDurationNs; + if (recentFrameOutlierFlags[recentFrameOutlierIndex]) { + recentFrameOutlierFlags[recentFrameOutlierIndex] = false; + recentFrameOutlierCount--; + } + } else { + if (!recentFrameOutlierFlags[recentFrameOutlierIndex]) { + recentFrameOutlierFlags[recentFrameOutlierIndex] = true; + recentFrameOutlierCount++; + } + } + } + + frameCount++; + lastFramePresentationTimeNs = framePresentationTimeNs; + } + + private static int getRecentFrameOutlierIndex(long frameCount) { + return (int) (frameCount % CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC); + } + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java index 8da7866d0ca..5908c6ff8fd 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/video/MediaCodecVideoRenderer.java @@ -15,6 +15,9 @@ */ package com.google.android.exoplayer2.video; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_MAX_INPUT_SIZE_EXCEEDED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_MAX_RESOLUTION_EXCEEDED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_NO; import static java.lang.Math.max; import static java.lang.Math.min; @@ -45,6 +48,8 @@ import com.google.android.exoplayer2.RendererCapabilities; import com.google.android.exoplayer2.decoder.DecoderCounters; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DecoderDiscardReasons; import com.google.android.exoplayer2.drm.DrmInitData; import com.google.android.exoplayer2.mediacodec.MediaCodecAdapter; import com.google.android.exoplayer2.mediacodec.MediaCodecDecoderException; @@ -60,7 +65,6 @@ import com.google.android.exoplayer2.util.TraceUtil; import com.google.android.exoplayer2.util.Util; import com.google.android.exoplayer2.video.VideoRendererEventListener.EventDispatcher; -import java.lang.reflect.Method; import java.nio.ByteBuffer; import java.util.Collections; import java.util.List; @@ -75,7 +79,7 @@ *

  • Message with type {@link #MSG_SET_SURFACE} to set the output surface. The message payload * should be the target {@link Surface}, or null. *
  • Message with type {@link #MSG_SET_SCALING_MODE} to set the video scaling mode. The message - * payload should be one of the integer scaling modes in {@link VideoScalingMode}. Note that + * payload should be one of the integer scaling modes in {@link C.VideoScalingMode}. Note that * the scaling mode only applies if the {@link Surface} targeted by this renderer is owned by * a {@link android.view.SurfaceView}. *
  • Message with type {@link #MSG_SET_VIDEO_FRAME_METADATA_LISTENER} to set a listener for @@ -104,29 +108,11 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer { /** Magic frame render timestamp that indicates the EOS in tunneling mode. */ private static final long TUNNELING_EOS_PRESENTATION_TIME_US = Long.MAX_VALUE; - // TODO: Remove reflection once we target API level 30. - @Nullable private static final Method surfaceSetFrameRateMethod; - - static { - @Nullable Method setFrameRateMethod = null; - if (Util.SDK_INT >= 30) { - try { - setFrameRateMethod = Surface.class.getMethod("setFrameRate", float.class, int.class); - } catch (NoSuchMethodException e) { - // Do nothing. - } - } - surfaceSetFrameRateMethod = setFrameRateMethod; - } - // TODO: Remove these constants and use those defined by Surface once we target API level 30. - private static final int SURFACE_FRAME_RATE_COMPATIBILITY_DEFAULT = 0; - private static final int SURFACE_FRAME_RATE_COMPATIBILITY_FIXED_SOURCE = 1; - private static boolean evaluatedDeviceNeedsSetOutputSurfaceWorkaround; private static boolean deviceNeedsSetOutputSurfaceWorkaround; private final Context context; - private final VideoFrameReleaseTimeHelper frameReleaseTimeHelper; + private final VideoFrameReleaseHelper frameReleaseHelper; private final EventDispatcher eventDispatcher; private final long allowedJoiningTimeMs; private final int maxDroppedFramesToNotify; @@ -136,11 +122,10 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer { private boolean codecNeedsSetOutputSurfaceWorkaround; private boolean codecHandlesHdr10PlusOutOfBandMetadata; - private Surface surface; - private float surfaceFrameRate; - private Surface dummySurface; + @Nullable private Surface surface; + @Nullable private Surface dummySurface; private boolean haveReportedFirstFrameRenderedForCurrentSurface; - @VideoScalingMode private int scalingMode; + @C.VideoScalingMode private int scalingMode; private boolean renderedFirstFrameAfterReset; private boolean mayRenderFirstFrameAfterEnableIfNotStarted; private boolean renderedFirstFrameAfterEnable; @@ -150,7 +135,8 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer { private int droppedFrames; private int consecutiveDroppedFrameCount; private int buffersInCodecCount; - private long lastRenderTimeUs; + private long lastBufferPresentationTimeUs; + private long lastRenderRealtimeUs; private long totalVideoFrameProcessingOffsetUs; private int videoFrameProcessingOffsetCount; @@ -158,7 +144,6 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer { private int currentHeight; private int currentUnappliedRotationDegrees; private float currentPixelWidthHeightRatio; - private float currentFrameRate; private int reportedWidth; private int reportedHeight; private int reportedUnappliedRotationDegrees; @@ -214,6 +199,7 @@ public MediaCodecVideoRenderer( int maxDroppedFramesToNotify) { this( context, + MediaCodecAdapter.Factory.DEFAULT, mediaCodecSelector, allowedJoiningTimeMs, /* enableDecoderFallback= */ false, @@ -244,22 +230,62 @@ public MediaCodecVideoRenderer( @Nullable Handler eventHandler, @Nullable VideoRendererEventListener eventListener, int maxDroppedFramesToNotify) { + this( + context, + MediaCodecAdapter.Factory.DEFAULT, + mediaCodecSelector, + allowedJoiningTimeMs, + enableDecoderFallback, + eventHandler, + eventListener, + maxDroppedFramesToNotify); + } + + /** + * Creates a new instance. + * + * @param context A context. + * @param codecAdapterFactory The {@link MediaCodecAdapter.Factory} used to create {@link + * MediaCodecAdapter} instances. + * @param mediaCodecSelector A decoder selector. + * @param allowedJoiningTimeMs The maximum duration in milliseconds for which this video renderer + * can attempt to seamlessly join an ongoing playback. + * @param enableDecoderFallback Whether to enable fallback to lower-priority decoders if decoder + * initialization fails. This may result in using a decoder that is slower/less efficient than + * the primary decoder. + * @param eventHandler A handler to use when delivering events to {@code eventListener}. May be + * null if delivery of events is not required. + * @param eventListener A listener of events. May be null if delivery of events is not required. + * @param maxDroppedFramesToNotify The maximum number of frames that can be dropped between + * invocations of {@link VideoRendererEventListener#onDroppedFrames(int, long)}. + */ + public MediaCodecVideoRenderer( + Context context, + MediaCodecAdapter.Factory codecAdapterFactory, + MediaCodecSelector mediaCodecSelector, + long allowedJoiningTimeMs, + boolean enableDecoderFallback, + @Nullable Handler eventHandler, + @Nullable VideoRendererEventListener eventListener, + int maxDroppedFramesToNotify) { super( C.TRACK_TYPE_VIDEO, + codecAdapterFactory, mediaCodecSelector, enableDecoderFallback, /* assumedMinimumCodecOperatingRate= */ 30); this.allowedJoiningTimeMs = allowedJoiningTimeMs; this.maxDroppedFramesToNotify = maxDroppedFramesToNotify; this.context = context.getApplicationContext(); - frameReleaseTimeHelper = new VideoFrameReleaseTimeHelper(this.context); + frameReleaseHelper = new VideoFrameReleaseHelper(this.context); eventDispatcher = new EventDispatcher(eventHandler, eventListener); deviceNeedsNoPostProcessWorkaround = deviceNeedsNoPostProcessWorkaround(); joiningDeadlineMs = C.TIME_UNSET; currentWidth = Format.NO_VALUE; currentHeight = Format.NO_VALUE; currentPixelWidthHeightRatio = Format.NO_VALUE; - scalingMode = VIDEO_SCALING_MODE_DEFAULT; + scalingMode = C.VIDEO_SCALING_MODE_DEFAULT; + tunnelingAudioSessionId = C.AUDIO_SESSION_ID_UNSET; clearReportedVideoSize(); } @@ -274,7 +300,7 @@ protected int supportsFormat(MediaCodecSelector mediaCodecSelector, Format forma throws DecoderQueryException { String mimeType = format.sampleMimeType; if (!MimeTypes.isVideo(mimeType)) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } @Nullable DrmInitData drmInitData = format.drmInitData; // Assume encrypted content requires secure decoders. @@ -295,10 +321,10 @@ protected int supportsFormat(MediaCodecSelector mediaCodecSelector, Format forma /* requiresTunnelingDecoder= */ false); } if (decoderInfos.isEmpty()) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_SUBTYPE); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_SUBTYPE); } if (!supportsFormatDrm(format)) { - return RendererCapabilities.create(FORMAT_UNSUPPORTED_DRM); + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_DRM); } // Check capabilities for the first decoder in the list, which takes priority. MediaCodecInfo decoderInfo = decoderInfos.get(0); @@ -324,8 +350,8 @@ protected int supportsFormat(MediaCodecSelector mediaCodecSelector, Format forma } } } - @FormatSupport - int formatSupport = isFormatSupported ? FORMAT_HANDLED : FORMAT_EXCEEDS_CAPABILITIES; + @C.FormatSupport + int formatSupport = isFormatSupported ? C.FORMAT_HANDLED : C.FORMAT_EXCEEDS_CAPABILITIES; return RendererCapabilities.create(formatSupport, adaptiveSupport, tunnelingSupport); } @@ -375,14 +401,14 @@ private static List getDecoderInfos( protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) throws ExoPlaybackException { super.onEnabled(joining, mayRenderStartOfStream); - int oldTunnelingAudioSessionId = tunnelingAudioSessionId; - tunnelingAudioSessionId = getConfiguration().tunnelingAudioSessionId; - tunneling = tunnelingAudioSessionId != C.AUDIO_SESSION_ID_UNSET; - if (tunnelingAudioSessionId != oldTunnelingAudioSessionId) { + boolean tunneling = getConfiguration().tunneling; + Assertions.checkState(!tunneling || tunnelingAudioSessionId != C.AUDIO_SESSION_ID_UNSET); + if (this.tunneling != tunneling) { + this.tunneling = tunneling; releaseCodec(); } eventDispatcher.enabled(decoderCounters); - frameReleaseTimeHelper.enable(); + frameReleaseHelper.onEnabled(); mayRenderFirstFrameAfterEnableIfNotStarted = mayRenderStartOfStream; renderedFirstFrameAfterEnable = false; } @@ -391,6 +417,8 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) protected void onPositionReset(long positionUs, boolean joining) throws ExoPlaybackException { super.onPositionReset(positionUs, joining); clearRenderedFirstFrame(); + frameReleaseHelper.onPositionReset(); + lastBufferPresentationTimeUs = C.TIME_UNSET; initialPositionUs = C.TIME_UNSET; consecutiveDroppedFrameCount = 0; if (joining) { @@ -428,10 +456,10 @@ protected void onStarted() { super.onStarted(); droppedFrames = 0; droppedFrameAccumulationStartTimeMs = SystemClock.elapsedRealtime(); - lastRenderTimeUs = SystemClock.elapsedRealtime() * 1000; + lastRenderRealtimeUs = SystemClock.elapsedRealtime() * 1000; totalVideoFrameProcessingOffsetUs = 0; videoFrameProcessingOffsetCount = 0; - updateSurfaceFrameRate(/* isNewSurface= */ false); + frameReleaseHelper.onStarted(); } @Override @@ -439,7 +467,7 @@ protected void onStopped() { joiningDeadlineMs = C.TIME_UNSET; maybeNotifyDroppedFrames(); maybeNotifyVideoFrameProcessingOffset(); - clearSurfaceFrameRate(); + frameReleaseHelper.onStopped(); super.onStopped(); } @@ -448,7 +476,7 @@ protected void onDisabled() { clearReportedVideoSize(); clearRenderedFirstFrame(); haveReportedFirstFrameRenderedForCurrentSurface = false; - frameReleaseTimeHelper.disable(); + frameReleaseHelper.onDisabled(); tunnelingOnFrameRenderedListener = null; try { super.onDisabled(); @@ -474,18 +502,31 @@ protected void onReset() { @Override public void handleMessage(int messageType, @Nullable Object message) throws ExoPlaybackException { - if (messageType == MSG_SET_SURFACE) { - setSurface((Surface) message); - } else if (messageType == MSG_SET_SCALING_MODE) { - scalingMode = (Integer) message; - MediaCodec codec = getCodec(); - if (codec != null) { - codec.setVideoScalingMode(scalingMode); - } - } else if (messageType == MSG_SET_VIDEO_FRAME_METADATA_LISTENER) { - frameMetadataListener = (VideoFrameMetadataListener) message; - } else { - super.handleMessage(messageType, message); + switch (messageType) { + case MSG_SET_SURFACE: + setSurface((Surface) message); + break; + case MSG_SET_SCALING_MODE: + scalingMode = (Integer) message; + @Nullable MediaCodecAdapter codec = getCodec(); + if (codec != null) { + codec.setVideoScalingMode(scalingMode); + } + break; + case MSG_SET_VIDEO_FRAME_METADATA_LISTENER: + frameMetadataListener = (VideoFrameMetadataListener) message; + break; + case MSG_SET_AUDIO_SESSION_ID: + int tunnelingAudioSessionId = (int) message; + if (this.tunnelingAudioSessionId != tunnelingAudioSessionId) { + this.tunnelingAudioSessionId = tunnelingAudioSessionId; + if (tunneling) { + releaseCodec(); + } + } + break; + default: + super.handleMessage(messageType, message); } } @@ -504,13 +545,12 @@ private void setSurface(Surface surface) throws ExoPlaybackException { } // We only need to update the codec if the surface has changed. if (this.surface != surface) { - clearSurfaceFrameRate(); this.surface = surface; + frameReleaseHelper.onSurfaceChanged(surface); haveReportedFirstFrameRenderedForCurrentSurface = false; - updateSurfaceFrameRate(/* isNewSurface= */ true); @State int state = getState(); - MediaCodec codec = getCodec(); + @Nullable MediaCodecAdapter codec = getCodec(); if (codec != null) { if (Util.SDK_INT >= 23 && surface != null && !codecNeedsSetOutputSurfaceWorkaround) { setOutputSurfaceV23(codec, surface); @@ -554,7 +594,7 @@ protected boolean getCodecNeedsEosPropagation() { @Override protected void configureCodec( MediaCodecInfo codecInfo, - MediaCodecAdapter codecAdapter, + MediaCodecAdapter codec, Format format, @Nullable MediaCrypto crypto, float codecOperatingRate) { @@ -567,7 +607,7 @@ protected void configureCodec( codecMaxValues, codecOperatingRate, deviceNeedsNoPostProcessWorkaround, - tunnelingAudioSessionId); + tunneling ? tunnelingAudioSessionId : C.AUDIO_SESSION_ID_UNSET); if (surface == null) { if (!shouldUseDummySurface(codecInfo)) { throw new IllegalStateException(); @@ -577,25 +617,31 @@ protected void configureCodec( } surface = dummySurface; } - codecAdapter.configure(mediaFormat, surface, crypto, 0); + codec.configure(mediaFormat, surface, crypto, 0); if (Util.SDK_INT >= 23 && tunneling) { - tunnelingOnFrameRenderedListener = new OnFrameRenderedListenerV23(codecAdapter.getCodec()); + tunnelingOnFrameRenderedListener = new OnFrameRenderedListenerV23(codec); } } @Override - protected @KeepCodecResult int canKeepCodec( - MediaCodec codec, MediaCodecInfo codecInfo, Format oldFormat, Format newFormat) { - if (codecInfo.isSeamlessAdaptationSupported( - oldFormat, newFormat, /* isNewFormatComplete= */ true) - && newFormat.width <= codecMaxValues.width - && newFormat.height <= codecMaxValues.height - && getMaxInputSize(codecInfo, newFormat) <= codecMaxValues.inputSize) { - return oldFormat.initializationDataEquals(newFormat) - ? KEEP_CODEC_RESULT_YES_WITHOUT_RECONFIGURATION - : KEEP_CODEC_RESULT_YES_WITH_RECONFIGURATION; + protected DecoderReuseEvaluation canReuseCodec( + MediaCodecInfo codecInfo, Format oldFormat, Format newFormat) { + DecoderReuseEvaluation evaluation = codecInfo.canReuseCodec(oldFormat, newFormat); + + @DecoderDiscardReasons int discardReasons = evaluation.discardReasons; + if (newFormat.width > codecMaxValues.width || newFormat.height > codecMaxValues.height) { + discardReasons |= DISCARD_REASON_VIDEO_MAX_RESOLUTION_EXCEEDED; + } + if (getMaxInputSize(codecInfo, newFormat) > codecMaxValues.inputSize) { + discardReasons |= DISCARD_REASON_MAX_INPUT_SIZE_EXCEEDED; } - return KEEP_CODEC_RESULT_NO; + + return new DecoderReuseEvaluation( + codecInfo.name, + oldFormat, + newFormat, + discardReasons != 0 ? REUSE_RESULT_NO : evaluation.result, + discardReasons); } @CallSuper @@ -606,14 +652,15 @@ protected void resetCodecStateForFlush() { } @Override - public void setOperatingRate(float operatingRate) throws ExoPlaybackException { - super.setOperatingRate(operatingRate); - updateSurfaceFrameRate(/* isNewSurface= */ false); + public void setPlaybackSpeed(float currentPlaybackSpeed, float targetPlaybackSpeed) + throws ExoPlaybackException { + super.setPlaybackSpeed(currentPlaybackSpeed, targetPlaybackSpeed); + frameReleaseHelper.onPlaybackSpeed(currentPlaybackSpeed); } @Override protected float getCodecOperatingRateV23( - float operatingRate, Format format, Format[] streamFormats) { + float targetPlaybackSpeed, Format format, Format[] streamFormats) { // Use the highest known stream frame-rate up front, to avoid having to reconfigure the codec // should an adaptive switch to that stream occur. float maxFrameRate = -1; @@ -623,7 +670,7 @@ protected float getCodecOperatingRateV23( maxFrameRate = max(maxFrameRate, streamFrameRate); } } - return maxFrameRate == -1 ? CODEC_OPERATING_RATE_UNSET : (maxFrameRate * operatingRate); + return maxFrameRate == -1 ? CODEC_OPERATING_RATE_UNSET : (maxFrameRate * targetPlaybackSpeed); } @Override @@ -636,9 +683,17 @@ protected void onCodecInitialized(String name, long initializedTimestampMs, } @Override - protected void onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybackException { - super.onInputFormatChanged(formatHolder); - eventDispatcher.inputFormatChanged(formatHolder.format); + protected void onCodecReleased(String name) { + eventDispatcher.decoderReleased(name); + } + + @Override + @Nullable + protected DecoderReuseEvaluation onInputFormatChanged(FormatHolder formatHolder) + throws ExoPlaybackException { + @Nullable DecoderReuseEvaluation evaluation = super.onInputFormatChanged(formatHolder); + eventDispatcher.inputFormatChanged(formatHolder.format, evaluation); + return evaluation; } /** @@ -666,7 +721,7 @@ protected void onQueueInputBuffer(DecoderInputBuffer buffer) throws ExoPlaybackE @Override protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaFormat) { - @Nullable MediaCodec codec = getCodec(); + @Nullable MediaCodecAdapter codec = getCodec(); if (codec != null) { // Must be applied each time the output format changes. codec.setVideoScalingMode(scalingMode); @@ -705,8 +760,7 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF // On API level 20 and below the decoder does not apply the rotation. currentUnappliedRotationDegrees = format.rotationDegrees; } - currentFrameRate = format.frameRate; - updateSurfaceFrameRate(/* isNewSurface= */ false); + frameReleaseHelper.onFormatChanged(format.frameRate); } @Override @@ -744,7 +798,7 @@ protected void handleInputBufferSupplementalData(DecoderInputBuffer buffer) protected boolean processOutputBuffer( long positionUs, long elapsedRealtimeUs, - @Nullable MediaCodec codec, + @Nullable MediaCodecAdapter codec, @Nullable ByteBuffer buffer, int bufferIndex, int bufferFlags, @@ -760,6 +814,11 @@ protected boolean processOutputBuffer( initialPositionUs = positionUs; } + if (bufferPresentationTimeUs != lastBufferPresentationTimeUs) { + frameReleaseHelper.onNextFrame(bufferPresentationTimeUs); + this.lastBufferPresentationTimeUs = bufferPresentationTimeUs; + } + long outputStreamOffsetUs = getOutputStreamOffsetUs(); long presentationTimeUs = bufferPresentationTimeUs - outputStreamOffsetUs; @@ -768,7 +827,20 @@ protected boolean processOutputBuffer( return true; } - long earlyUs = bufferPresentationTimeUs - positionUs; + // Note: Use of double rather than float is intentional for accuracy in the calculations below. + double playbackSpeed = getPlaybackSpeed(); + boolean isStarted = getState() == STATE_STARTED; + long elapsedRealtimeNowUs = SystemClock.elapsedRealtime() * 1000; + + // Calculate how early we are. In other words, the realtime duration that needs to elapse whilst + // the renderer is started before the frame should be rendered. A negative value means that + // we're already late. + long earlyUs = (long) ((bufferPresentationTimeUs - positionUs) / playbackSpeed); + if (isStarted) { + // Account for the elapsed time since the start of this iteration of the rendering loop. + earlyUs -= elapsedRealtimeNowUs - elapsedRealtimeUs; + } + if (surface == dummySurface) { // Skip frames in sync with playback, so we'll be at the right frame if the mode changes. if (isBufferLate(earlyUs)) { @@ -779,9 +851,7 @@ protected boolean processOutputBuffer( return false; } - long elapsedRealtimeNowUs = SystemClock.elapsedRealtime() * 1000; - long elapsedSinceLastRenderUs = elapsedRealtimeNowUs - lastRenderTimeUs; - boolean isStarted = getState() == STATE_STARTED; + long elapsedSinceLastRenderUs = elapsedRealtimeNowUs - lastRenderRealtimeUs; boolean shouldRenderFirstFrame = !renderedFirstFrameAfterEnable ? (isStarted || mayRenderFirstFrameAfterEnableIfNotStarted) @@ -808,24 +878,17 @@ protected boolean processOutputBuffer( return false; } - // Fine-grained adjustment of earlyUs based on the elapsed time since the start of the current - // iteration of the rendering loop. - long elapsedSinceStartOfLoopUs = elapsedRealtimeNowUs - elapsedRealtimeUs; - earlyUs -= elapsedSinceStartOfLoopUs; - // Compute the buffer's desired release time in nanoseconds. long systemTimeNs = System.nanoTime(); long unadjustedFrameReleaseTimeNs = systemTimeNs + (earlyUs * 1000); // Apply a timestamp adjustment, if there is one. - long adjustedReleaseTimeNs = frameReleaseTimeHelper.adjustReleaseTime( - bufferPresentationTimeUs, unadjustedFrameReleaseTimeNs); + long adjustedReleaseTimeNs = frameReleaseHelper.adjustReleaseTime(unadjustedFrameReleaseTimeNs); earlyUs = (adjustedReleaseTimeNs - systemTimeNs) / 1000; boolean treatDroppedBuffersAsSkipped = joiningDeadlineMs != C.TIME_UNSET; if (shouldDropBuffersToKeyframe(earlyUs, elapsedRealtimeUs, isLastBuffer) - && maybeDropBuffersToKeyframe( - codec, bufferIndex, presentationTimeUs, positionUs, treatDroppedBuffersAsSkipped)) { + && maybeDropBuffersToKeyframe(positionUs, treatDroppedBuffersAsSkipped)) { return false; } else if (shouldDropOutputBuffer(earlyUs, elapsedRealtimeUs, isLastBuffer)) { if (treatDroppedBuffersAsSkipped) { @@ -957,7 +1020,7 @@ protected boolean shouldForceRenderOutputBuffer(long earlyUs, long elapsedSinceL * @param index The index of the output buffer to skip. * @param presentationTimeUs The presentation time of the output buffer, in microseconds. */ - protected void skipOutputBuffer(MediaCodec codec, int index, long presentationTimeUs) { + protected void skipOutputBuffer(MediaCodecAdapter codec, int index, long presentationTimeUs) { TraceUtil.beginSection("skipVideoBuffer"); codec.releaseOutputBuffer(index, false); TraceUtil.endSection(); @@ -971,7 +1034,7 @@ protected void skipOutputBuffer(MediaCodec codec, int index, long presentationTi * @param index The index of the output buffer to drop. * @param presentationTimeUs The presentation time of the output buffer, in microseconds. */ - protected void dropOutputBuffer(MediaCodec codec, int index, long presentationTimeUs) { + protected void dropOutputBuffer(MediaCodecAdapter codec, int index, long presentationTimeUs) { TraceUtil.beginSection("dropVideoBuffer"); codec.releaseOutputBuffer(index, false); TraceUtil.endSection(); @@ -983,9 +1046,6 @@ protected void dropOutputBuffer(MediaCodec codec, int index, long presentationTi * position. If no such keyframe exists, as the playback position is inside the same group of * pictures as the buffer being processed, returns {@code false}. Returns {@code true} otherwise. * - * @param codec The codec that owns the output buffer. - * @param index The index of the output buffer to drop. - * @param presentationTimeUs The presentation time of the output buffer, in microseconds. * @param positionUs The current playback position, in microseconds. * @param treatDroppedBuffersAsSkipped Whether dropped buffers should be treated as intentionally * skipped. @@ -993,9 +1053,6 @@ protected void dropOutputBuffer(MediaCodec codec, int index, long presentationTi * @throws ExoPlaybackException If an error occurs flushing the codec. */ protected boolean maybeDropBuffersToKeyframe( - MediaCodec codec, - int index, - long presentationTimeUs, long positionUs, boolean treatDroppedBuffersAsSkipped) throws ExoPlaybackException { @@ -1052,12 +1109,12 @@ protected void updateVideoFrameProcessingOffsetCounters(long processingOffsetUs) * @param index The index of the output buffer to drop. * @param presentationTimeUs The presentation time of the output buffer, in microseconds. */ - protected void renderOutputBuffer(MediaCodec codec, int index, long presentationTimeUs) { + protected void renderOutputBuffer(MediaCodecAdapter codec, int index, long presentationTimeUs) { maybeNotifyVideoSizeChanged(); TraceUtil.beginSection("releaseOutputBuffer"); codec.releaseOutputBuffer(index, true); TraceUtil.endSection(); - lastRenderTimeUs = SystemClock.elapsedRealtime() * 1000; + lastRenderRealtimeUs = SystemClock.elapsedRealtime() * 1000; decoderCounters.renderedOutputBufferCount++; consecutiveDroppedFrameCount = 0; maybeNotifyRenderedFirstFrame(); @@ -1074,63 +1131,17 @@ protected void renderOutputBuffer(MediaCodec codec, int index, long presentation */ @RequiresApi(21) protected void renderOutputBufferV21( - MediaCodec codec, int index, long presentationTimeUs, long releaseTimeNs) { + MediaCodecAdapter codec, int index, long presentationTimeUs, long releaseTimeNs) { maybeNotifyVideoSizeChanged(); TraceUtil.beginSection("releaseOutputBuffer"); codec.releaseOutputBuffer(index, releaseTimeNs); TraceUtil.endSection(); - lastRenderTimeUs = SystemClock.elapsedRealtime() * 1000; + lastRenderRealtimeUs = SystemClock.elapsedRealtime() * 1000; decoderCounters.renderedOutputBufferCount++; consecutiveDroppedFrameCount = 0; maybeNotifyRenderedFirstFrame(); } - /** - * Updates the frame-rate of the current {@link #surface} based on the renderer operating rate, - * frame-rate of the content, and whether the renderer is started. - * - * @param isNewSurface Whether the current {@link #surface} is new. - */ - private void updateSurfaceFrameRate(boolean isNewSurface) { - if (Util.SDK_INT < 30 || surface == null || surface == dummySurface) { - return; - } - boolean shouldSetFrameRate = getState() == STATE_STARTED && currentFrameRate != Format.NO_VALUE; - float surfaceFrameRate = shouldSetFrameRate ? currentFrameRate * getOperatingRate() : 0; - // We always set the frame-rate if we have a new surface, since we have no way of knowing what - // it might have been set to previously. - if (this.surfaceFrameRate == surfaceFrameRate && !isNewSurface) { - return; - } - this.surfaceFrameRate = surfaceFrameRate; - setSurfaceFrameRateV30(surface, surfaceFrameRate); - } - - /** Clears the frame-rate of the current {@link #surface}. */ - private void clearSurfaceFrameRate() { - if (Util.SDK_INT < 30 || surface == null || surface == dummySurface || surfaceFrameRate == 0) { - return; - } - surfaceFrameRate = 0; - setSurfaceFrameRateV30(surface, /* frameRate= */ 0); - } - - @RequiresApi(30) - private void setSurfaceFrameRateV30(Surface surface, float frameRate) { - if (surfaceSetFrameRateMethod == null) { - Log.e(TAG, "Failed to call Surface.setFrameRate (method does not exist)"); - } - int compatibility = - frameRate == 0 - ? SURFACE_FRAME_RATE_COMPATIBILITY_DEFAULT - : SURFACE_FRAME_RATE_COMPATIBILITY_FIXED_SOURCE; - try { - surfaceSetFrameRateMethod.invoke(surface, frameRate, compatibility); - } catch (Exception e) { - Log.e(TAG, "Failed to call Surface.setFrameRate", e); - } - } - private boolean shouldUseDummySurface(MediaCodecInfo codecInfo) { return Util.SDK_INT >= 23 && !tunneling @@ -1150,7 +1161,7 @@ private void clearRenderedFirstFrame() { // OnFrameRenderedListenerV23.onFrameRenderedListener for tunneled playback on API level 23 and // above. if (Util.SDK_INT >= 23 && tunneling) { - MediaCodec codec = getCodec(); + @Nullable MediaCodecAdapter codec = getCodec(); // If codec is null then the listener will be instantiated in configureCodec. if (codec != null) { tunnelingOnFrameRenderedListener = new OnFrameRenderedListenerV23(codec); @@ -1231,14 +1242,14 @@ private static boolean isBufferVeryLate(long earlyUs) { } @RequiresApi(29) - private static void setHdr10PlusInfoV29(MediaCodec codec, byte[] hdr10PlusInfo) { + private static void setHdr10PlusInfoV29(MediaCodecAdapter codec, byte[] hdr10PlusInfo) { Bundle codecParameters = new Bundle(); codecParameters.putByteArray(MediaCodec.PARAMETER_KEY_HDR10_PLUS_INFO, hdr10PlusInfo); codec.setParameters(codecParameters); } @RequiresApi(23) - protected void setOutputSurfaceV23(MediaCodec codec, Surface surface) { + protected void setOutputSurfaceV23(MediaCodecAdapter codec, Surface surface) { codec.setOutputSurface(surface); } @@ -1346,8 +1357,12 @@ protected CodecMaxValues getCodecMaxValues( } boolean haveUnknownDimensions = false; for (Format streamFormat : streamFormats) { - if (codecInfo.isSeamlessAdaptationSupported( - format, streamFormat, /* isNewFormatComplete= */ false)) { + if (format.colorInfo != null && streamFormat.colorInfo == null) { + // streamFormat likely has incomplete color information. Copy the complete color information + // from format to avoid codec re-use being ruled out for only this reason. + streamFormat = streamFormat.buildUpon().setColorInfo(format.colorInfo).build(); + } + if (codecInfo.canReuseCodec(format, streamFormat).result != REUSE_RESULT_NO) { haveUnknownDimensions |= (streamFormat.width == Format.NO_VALUE || streamFormat.height == Format.NO_VALUE); maxWidth = max(maxWidth, streamFormat.width); @@ -1778,19 +1793,19 @@ private static boolean evaluateDeviceNeedsSetOutputSurfaceWorkaround() { @RequiresApi(23) private final class OnFrameRenderedListenerV23 - implements MediaCodec.OnFrameRenderedListener, Handler.Callback { + implements MediaCodecAdapter.OnFrameRenderedListener, Handler.Callback { private static final int HANDLE_FRAME_RENDERED = 0; private final Handler handler; - public OnFrameRenderedListenerV23(MediaCodec codec) { + public OnFrameRenderedListenerV23(MediaCodecAdapter codec) { handler = Util.createHandlerForCurrentLooper(/* callback= */ this); codec.setOnFrameRenderedListener(/* listener= */ this, handler); } @Override - public void onFrameRendered(MediaCodec codec, long presentationTimeUs, long nanoTime) { + public void onFrameRendered(MediaCodecAdapter codec, long presentationTimeUs, long nanoTime) { // Workaround bug in MediaCodec that causes deadlock if you call directly back into the // MediaCodec from this listener method. // Deadlock occurs because MediaCodec calls this listener method holding a lock, diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/VideoFrameReleaseHelper.java b/library/core/src/main/java/com/google/android/exoplayer2/video/VideoFrameReleaseHelper.java new file mode 100644 index 00000000000..1778ed69762 --- /dev/null +++ b/library/core/src/main/java/com/google/android/exoplayer2/video/VideoFrameReleaseHelper.java @@ -0,0 +1,565 @@ +/* + * Copyright (C) 2016 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.video; + +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; + +import android.annotation.TargetApi; +import android.content.Context; +import android.hardware.display.DisplayManager; +import android.os.Handler; +import android.os.HandlerThread; +import android.os.Message; +import android.view.Choreographer; +import android.view.Choreographer.FrameCallback; +import android.view.Display; +import android.view.Surface; +import android.view.WindowManager; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.Renderer; +import com.google.android.exoplayer2.util.Log; +import com.google.android.exoplayer2.util.Util; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; + +/** + * Helps a video {@link Renderer} release frames to a {@link Surface}. The helper: + * + *
      + *
    • Adjusts frame release timestamps to achieve a smoother visual result. The release + * timestamps are smoothed, and aligned with the default display's vsync signal. + *
    • Adjusts the {@link Surface} frame rate to inform the underlying platform of a fixed frame + * rate, when there is one. + *
    + */ +public final class VideoFrameReleaseHelper { + + private static final String TAG = "VideoFrameReleaseHelper"; + + /** + * The minimum sum of frame durations used to calculate the current fixed frame rate estimate, for + * the estimate to be treated as a high confidence estimate. + */ + private static final long MINIMUM_MATCHING_FRAME_DURATION_FOR_HIGH_CONFIDENCE_NS = 5_000_000_000L; + + /** + * The minimum change in media frame rate that will trigger a change in surface frame rate, given + * a high confidence estimate. + */ + private static final float MINIMUM_MEDIA_FRAME_RATE_CHANGE_FOR_UPDATE_HIGH_CONFIDENCE = 0.02f; + + /** + * The minimum change in media frame rate that will trigger a change in surface frame rate, given + * a low confidence estimate. + */ + private static final float MINIMUM_MEDIA_FRAME_RATE_CHANGE_FOR_UPDATE_LOW_CONFIDENCE = 1f; + + /** + * The minimum number of frames without a frame rate estimate, for the surface frame rate to be + * cleared. + */ + private static final int MINIMUM_FRAMES_WITHOUT_SYNC_TO_CLEAR_SURFACE_FRAME_RATE = + 2 * FixedFrameRateEstimator.CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC; + + /** The period between sampling display VSYNC timestamps, in milliseconds. */ + private static final long VSYNC_SAMPLE_UPDATE_PERIOD_MS = 500; + /** + * The maximum adjustment that can be made to a frame release timestamp, in nanoseconds, excluding + * the part of the adjustment that aligns frame release timestamps with the display VSYNC. + */ + private static final long MAX_ALLOWED_ADJUSTMENT_NS = 20_000_000; + /** + * If a frame is targeted to a display VSYNC with timestamp {@code vsyncTime}, the adjusted frame + * release timestamp will be calculated as {@code releaseTime = vsyncTime - ((vsyncDuration * + * VSYNC_OFFSET_PERCENTAGE) / 100)}. + */ + private static final long VSYNC_OFFSET_PERCENTAGE = 80; + + private final FixedFrameRateEstimator frameRateEstimator; + @Nullable private final WindowManager windowManager; + @Nullable private final VSyncSampler vsyncSampler; + @Nullable private final DefaultDisplayListener displayListener; + + private boolean started; + @Nullable private Surface surface; + + /** The media frame rate specified in the {@link Format}. */ + private float formatFrameRate; + /** + * The media frame rate used to calculate the playback frame rate of the {@link Surface}. This may + * be different to {@link #formatFrameRate} if {@link #formatFrameRate} is unspecified or + * inaccurate. + */ + private float surfaceMediaFrameRate; + /** The playback frame rate set on the {@link Surface}. */ + private float surfacePlaybackFrameRate; + + private float playbackSpeed; + + private long vsyncDurationNs; + private long vsyncOffsetNs; + + private long frameIndex; + private long pendingLastAdjustedFrameIndex; + private long pendingLastAdjustedReleaseTimeNs; + private long lastAdjustedFrameIndex; + private long lastAdjustedReleaseTimeNs; + + /** + * Constructs an instance. + * + * @param context A context from which information about the default display can be retrieved. + */ + public VideoFrameReleaseHelper(@Nullable Context context) { + frameRateEstimator = new FixedFrameRateEstimator(); + if (context != null) { + context = context.getApplicationContext(); + windowManager = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE); + } else { + windowManager = null; + } + if (windowManager != null) { + displayListener = + Util.SDK_INT >= 17 ? maybeBuildDefaultDisplayListenerV17(checkNotNull(context)) : null; + vsyncSampler = VSyncSampler.getInstance(); + } else { + displayListener = null; + vsyncSampler = null; + } + vsyncDurationNs = C.TIME_UNSET; + vsyncOffsetNs = C.TIME_UNSET; + formatFrameRate = Format.NO_VALUE; + playbackSpeed = 1f; + } + + /** Called when the renderer is enabled. */ + @TargetApi(17) // displayListener is null if Util.SDK_INT < 17. + public void onEnabled() { + if (windowManager != null) { + checkNotNull(vsyncSampler).addObserver(); + if (displayListener != null) { + displayListener.register(); + } + updateDefaultDisplayRefreshRateParams(); + } + } + + /** Called when the renderer is started. */ + public void onStarted() { + started = true; + resetAdjustment(); + updateSurfacePlaybackFrameRate(/* isNewSurface= */ false); + } + + /** + * Called when the renderer changes which {@link Surface} it's rendering to renders to. + * + * @param surface The new {@link Surface}, or {@code null} if the renderer does not have one. + */ + public void onSurfaceChanged(@Nullable Surface surface) { + if (surface instanceof DummySurface) { + // We don't care about dummy surfaces for release timing, since they're not visible. + surface = null; + } + if (this.surface == surface) { + return; + } + clearSurfaceFrameRate(); + this.surface = surface; + updateSurfacePlaybackFrameRate(/* isNewSurface= */ true); + } + + /** Called when the renderer's position is reset. */ + public void onPositionReset() { + resetAdjustment(); + } + + /** + * Called when the renderer's playback speed changes. + * + * @param playbackSpeed The factor by which playback is sped up. + */ + public void onPlaybackSpeed(float playbackSpeed) { + this.playbackSpeed = playbackSpeed; + resetAdjustment(); + updateSurfacePlaybackFrameRate(/* isNewSurface= */ false); + } + + /** + * Called when the renderer's output format changes. + * + * @param formatFrameRate The format's frame rate, or {@link Format#NO_VALUE} if unknown. + */ + public void onFormatChanged(float formatFrameRate) { + this.formatFrameRate = formatFrameRate; + frameRateEstimator.reset(); + updateSurfaceMediaFrameRate(); + } + + /** + * Called by the renderer for each frame, prior to it being skipped, dropped or rendered. + * + * @param framePresentationTimeUs The frame presentation timestamp, in microseconds. + */ + public void onNextFrame(long framePresentationTimeUs) { + if (pendingLastAdjustedFrameIndex != C.INDEX_UNSET) { + lastAdjustedFrameIndex = pendingLastAdjustedFrameIndex; + lastAdjustedReleaseTimeNs = pendingLastAdjustedReleaseTimeNs; + } + frameIndex++; + frameRateEstimator.onNextFrame(framePresentationTimeUs * 1000); + updateSurfaceMediaFrameRate(); + } + + /** Called when the renderer is stopped. */ + public void onStopped() { + started = false; + clearSurfaceFrameRate(); + } + + /** Called when the renderer is disabled. */ + @TargetApi(17) // displayListener is null if Util.SDK_INT < 17. + public void onDisabled() { + if (windowManager != null) { + if (displayListener != null) { + displayListener.unregister(); + } + checkNotNull(vsyncSampler).removeObserver(); + } + } + + // Frame release time adjustment. + + /** + * Adjusts the release timestamp for the next frame. This is the frame whose presentation + * timestamp was most recently passed to {@link #onNextFrame}. + * + *

    This method may be called any number of times for each frame, including zero times (for + * skipped frames, or when rendering the first frame prior to playback starting), or more than + * once (if the caller wishes to give the helper the opportunity to refine a release time closer + * to when the frame needs to be released). + * + * @param releaseTimeNs The frame's unadjusted release time, in nanoseconds and in the same time + * base as {@link System#nanoTime()}. + * @return The adjusted frame release timestamp, in nanoseconds and in the same time base as + * {@link System#nanoTime()}. + */ + public long adjustReleaseTime(long releaseTimeNs) { + // Until we know better, the adjustment will be a no-op. + long adjustedReleaseTimeNs = releaseTimeNs; + + if (lastAdjustedFrameIndex != C.INDEX_UNSET && frameRateEstimator.isSynced()) { + long frameDurationNs = frameRateEstimator.getFrameDurationNs(); + long candidateAdjustedReleaseTimeNs = + lastAdjustedReleaseTimeNs + + (long) ((frameDurationNs * (frameIndex - lastAdjustedFrameIndex)) / playbackSpeed); + if (adjustmentAllowed(releaseTimeNs, candidateAdjustedReleaseTimeNs)) { + adjustedReleaseTimeNs = candidateAdjustedReleaseTimeNs; + } else { + resetAdjustment(); + } + } + pendingLastAdjustedFrameIndex = frameIndex; + pendingLastAdjustedReleaseTimeNs = adjustedReleaseTimeNs; + + if (vsyncSampler == null || vsyncDurationNs == C.TIME_UNSET) { + return adjustedReleaseTimeNs; + } + long sampledVsyncTimeNs = vsyncSampler.sampledVsyncTimeNs; + if (sampledVsyncTimeNs == C.TIME_UNSET) { + return adjustedReleaseTimeNs; + } + // Find the timestamp of the closest vsync. This is the vsync that we're targeting. + long snappedTimeNs = closestVsync(adjustedReleaseTimeNs, sampledVsyncTimeNs, vsyncDurationNs); + // Apply an offset so that we release before the target vsync, but after the previous one. + return snappedTimeNs - vsyncOffsetNs; + } + + private void resetAdjustment() { + frameIndex = 0; + lastAdjustedFrameIndex = C.INDEX_UNSET; + pendingLastAdjustedFrameIndex = C.INDEX_UNSET; + } + + private static boolean adjustmentAllowed( + long unadjustedReleaseTimeNs, long adjustedReleaseTimeNs) { + return Math.abs(unadjustedReleaseTimeNs - adjustedReleaseTimeNs) <= MAX_ALLOWED_ADJUSTMENT_NS; + } + + // Surface frame rate adjustment. + + /** + * Updates the media frame rate that's used to calculate the playback frame rate of the current + * {@link #surface}. If the frame rate is updated then {@link #updateSurfacePlaybackFrameRate} is + * called to update the surface. + */ + private void updateSurfaceMediaFrameRate() { + if (Util.SDK_INT < 30 || surface == null) { + return; + } + + float candidateFrameRate = + frameRateEstimator.isSynced() ? frameRateEstimator.getFrameRate() : formatFrameRate; + if (candidateFrameRate == surfaceMediaFrameRate) { + return; + } + + // The candidate is different to the current surface media frame rate. Decide whether to update + // the surface media frame rate. + boolean shouldUpdate; + if (candidateFrameRate != Format.NO_VALUE && surfaceMediaFrameRate != Format.NO_VALUE) { + boolean candidateIsHighConfidence = + frameRateEstimator.isSynced() + && frameRateEstimator.getMatchingFrameDurationSumNs() + >= MINIMUM_MATCHING_FRAME_DURATION_FOR_HIGH_CONFIDENCE_NS; + float minimumChangeForUpdate = + candidateIsHighConfidence + ? MINIMUM_MEDIA_FRAME_RATE_CHANGE_FOR_UPDATE_HIGH_CONFIDENCE + : MINIMUM_MEDIA_FRAME_RATE_CHANGE_FOR_UPDATE_LOW_CONFIDENCE; + shouldUpdate = Math.abs(candidateFrameRate - surfaceMediaFrameRate) >= minimumChangeForUpdate; + } else if (candidateFrameRate != Format.NO_VALUE) { + shouldUpdate = true; + } else { + shouldUpdate = + frameRateEstimator.getFramesWithoutSyncCount() + >= MINIMUM_FRAMES_WITHOUT_SYNC_TO_CLEAR_SURFACE_FRAME_RATE; + } + + if (shouldUpdate) { + surfaceMediaFrameRate = candidateFrameRate; + updateSurfacePlaybackFrameRate(/* isNewSurface= */ false); + } + } + + /** + * Updates the playback frame rate of the current {@link #surface} based on the playback speed, + * frame rate of the content, and whether the renderer is started. + * + * @param isNewSurface Whether the current {@link #surface} is new. + */ + private void updateSurfacePlaybackFrameRate(boolean isNewSurface) { + if (Util.SDK_INT < 30 || surface == null) { + return; + } + + float surfacePlaybackFrameRate = 0; + if (started && surfaceMediaFrameRate != Format.NO_VALUE) { + surfacePlaybackFrameRate = surfaceMediaFrameRate * playbackSpeed; + } + // We always set the frame-rate if we have a new surface, since we have no way of knowing what + // it might have been set to previously. + if (!isNewSurface && this.surfacePlaybackFrameRate == surfacePlaybackFrameRate) { + return; + } + this.surfacePlaybackFrameRate = surfacePlaybackFrameRate; + setSurfaceFrameRateV30(surface, surfacePlaybackFrameRate); + } + + /** Clears the frame-rate of the current {@link #surface}. */ + private void clearSurfaceFrameRate() { + if (Util.SDK_INT < 30 || surface == null || surfacePlaybackFrameRate == 0) { + return; + } + surfacePlaybackFrameRate = 0; + setSurfaceFrameRateV30(surface, /* frameRate= */ 0); + } + + @RequiresApi(30) + private static void setSurfaceFrameRateV30(Surface surface, float frameRate) { + int compatibility = + frameRate == 0 + ? Surface.FRAME_RATE_COMPATIBILITY_DEFAULT + : Surface.FRAME_RATE_COMPATIBILITY_FIXED_SOURCE; + try { + surface.setFrameRate(frameRate, compatibility); + } catch (IllegalStateException e) { + Log.e(TAG, "Failed to call Surface.setFrameRate", e); + } + } + + // Display refresh rate and vsync logic. + + @RequiresApi(17) + @Nullable + private DefaultDisplayListener maybeBuildDefaultDisplayListenerV17(Context context) { + DisplayManager manager = (DisplayManager) context.getSystemService(Context.DISPLAY_SERVICE); + return manager == null ? null : new DefaultDisplayListener(manager); + } + + private void updateDefaultDisplayRefreshRateParams() { + Display defaultDisplay = checkNotNull(windowManager).getDefaultDisplay(); + if (defaultDisplay != null) { + double defaultDisplayRefreshRate = defaultDisplay.getRefreshRate(); + vsyncDurationNs = (long) (C.NANOS_PER_SECOND / defaultDisplayRefreshRate); + vsyncOffsetNs = (vsyncDurationNs * VSYNC_OFFSET_PERCENTAGE) / 100; + } else { + Log.w(TAG, "Unable to query display refresh rate"); + vsyncDurationNs = C.TIME_UNSET; + vsyncOffsetNs = C.TIME_UNSET; + } + } + + private static long closestVsync(long releaseTime, long sampledVsyncTime, long vsyncDuration) { + long vsyncCount = (releaseTime - sampledVsyncTime) / vsyncDuration; + long snappedTimeNs = sampledVsyncTime + (vsyncDuration * vsyncCount); + long snappedBeforeNs; + long snappedAfterNs; + if (releaseTime <= snappedTimeNs) { + snappedBeforeNs = snappedTimeNs - vsyncDuration; + snappedAfterNs = snappedTimeNs; + } else { + snappedBeforeNs = snappedTimeNs; + snappedAfterNs = snappedTimeNs + vsyncDuration; + } + long snappedAfterDiff = snappedAfterNs - releaseTime; + long snappedBeforeDiff = releaseTime - snappedBeforeNs; + return snappedAfterDiff < snappedBeforeDiff ? snappedAfterNs : snappedBeforeNs; + } + + @RequiresApi(17) + private final class DefaultDisplayListener implements DisplayManager.DisplayListener { + + private final DisplayManager displayManager; + + public DefaultDisplayListener(DisplayManager displayManager) { + this.displayManager = displayManager; + } + + public void register() { + displayManager.registerDisplayListener(this, Util.createHandlerForCurrentLooper()); + } + + public void unregister() { + displayManager.unregisterDisplayListener(this); + } + + @Override + public void onDisplayAdded(int displayId) { + // Do nothing. + } + + @Override + public void onDisplayRemoved(int displayId) { + // Do nothing. + } + + @Override + public void onDisplayChanged(int displayId) { + if (displayId == Display.DEFAULT_DISPLAY) { + updateDefaultDisplayRefreshRateParams(); + } + } + + } + + /** + * Samples display vsync timestamps. A single instance using a single {@link Choreographer} is + * shared by all {@link VideoFrameReleaseHelper} instances. This is done to avoid a resource leak + * in the platform on API levels prior to 23. See [Internal: b/12455729]. + */ + private static final class VSyncSampler implements FrameCallback, Handler.Callback { + + public volatile long sampledVsyncTimeNs; + + private static final int CREATE_CHOREOGRAPHER = 0; + private static final int MSG_ADD_OBSERVER = 1; + private static final int MSG_REMOVE_OBSERVER = 2; + + private static final VSyncSampler INSTANCE = new VSyncSampler(); + + private final Handler handler; + private final HandlerThread choreographerOwnerThread; + private @MonotonicNonNull Choreographer choreographer; + private int observerCount; + + public static VSyncSampler getInstance() { + return INSTANCE; + } + + private VSyncSampler() { + sampledVsyncTimeNs = C.TIME_UNSET; + choreographerOwnerThread = new HandlerThread("ExoPlayer:FrameReleaseChoreographer"); + choreographerOwnerThread.start(); + handler = Util.createHandler(choreographerOwnerThread.getLooper(), /* callback= */ this); + handler.sendEmptyMessage(CREATE_CHOREOGRAPHER); + } + + /** + * Notifies the sampler that a {@link VideoFrameReleaseHelper} is observing {@link + * #sampledVsyncTimeNs}, and hence that the value should be periodically updated. + */ + public void addObserver() { + handler.sendEmptyMessage(MSG_ADD_OBSERVER); + } + + /** + * Notifies the sampler that a {@link VideoFrameReleaseHelper} is no longer observing {@link + * #sampledVsyncTimeNs}. + */ + public void removeObserver() { + handler.sendEmptyMessage(MSG_REMOVE_OBSERVER); + } + + @Override + public void doFrame(long vsyncTimeNs) { + sampledVsyncTimeNs = vsyncTimeNs; + checkNotNull(choreographer).postFrameCallbackDelayed(this, VSYNC_SAMPLE_UPDATE_PERIOD_MS); + } + + @Override + public boolean handleMessage(Message message) { + switch (message.what) { + case CREATE_CHOREOGRAPHER: { + createChoreographerInstanceInternal(); + return true; + } + case MSG_ADD_OBSERVER: { + addObserverInternal(); + return true; + } + case MSG_REMOVE_OBSERVER: { + removeObserverInternal(); + return true; + } + default: { + return false; + } + } + } + + private void createChoreographerInstanceInternal() { + choreographer = Choreographer.getInstance(); + } + + private void addObserverInternal() { + observerCount++; + if (observerCount == 1) { + checkNotNull(choreographer).postFrameCallback(this); + } + } + + private void removeObserverInternal() { + observerCount--; + if (observerCount == 0) { + checkNotNull(choreographer).removeFrameCallback(this); + sampledVsyncTimeNs = C.TIME_UNSET; + } + } + + } +} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/VideoFrameReleaseTimeHelper.java b/library/core/src/main/java/com/google/android/exoplayer2/video/VideoFrameReleaseTimeHelper.java deleted file mode 100644 index 01b296e747e..00000000000 --- a/library/core/src/main/java/com/google/android/exoplayer2/video/VideoFrameReleaseTimeHelper.java +++ /dev/null @@ -1,360 +0,0 @@ -/* - * Copyright (C) 2016 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package com.google.android.exoplayer2.video; - -import android.annotation.TargetApi; -import android.content.Context; -import android.hardware.display.DisplayManager; -import android.os.Handler; -import android.os.HandlerThread; -import android.os.Message; -import android.view.Choreographer; -import android.view.Choreographer.FrameCallback; -import android.view.Display; -import android.view.WindowManager; -import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; -import com.google.android.exoplayer2.C; -import com.google.android.exoplayer2.util.Util; - -/** - * Makes a best effort to adjust frame release timestamps for a smoother visual result. - */ -public final class VideoFrameReleaseTimeHelper { - - private static final long CHOREOGRAPHER_SAMPLE_DELAY_MILLIS = 500; - private static final long MAX_ALLOWED_DRIFT_NS = 20_000_000; - - private static final long VSYNC_OFFSET_PERCENTAGE = 80; - private static final int MIN_FRAMES_FOR_ADJUSTMENT = 6; - - @Nullable private final WindowManager windowManager; - @Nullable private final VSyncSampler vsyncSampler; - @Nullable private final DefaultDisplayListener displayListener; - - private long vsyncDurationNs; - private long vsyncOffsetNs; - - private long lastFramePresentationTimeUs; - private long adjustedLastFrameTimeNs; - private long pendingAdjustedFrameTimeNs; - - private boolean haveSync; - private long syncUnadjustedReleaseTimeNs; - private long syncFramePresentationTimeNs; - private long frameCount; - - /** - * Constructs an instance that smooths frame release timestamps but does not align them with - * the default display's vsync signal. - */ - public VideoFrameReleaseTimeHelper() { - this(null); - } - - /** - * Constructs an instance that smooths frame release timestamps and aligns them with the default - * display's vsync signal. - * - * @param context A context from which information about the default display can be retrieved. - */ - public VideoFrameReleaseTimeHelper(@Nullable Context context) { - if (context != null) { - context = context.getApplicationContext(); - windowManager = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE); - } else { - windowManager = null; - } - if (windowManager != null) { - displayListener = Util.SDK_INT >= 17 ? maybeBuildDefaultDisplayListenerV17(context) : null; - vsyncSampler = VSyncSampler.getInstance(); - } else { - displayListener = null; - vsyncSampler = null; - } - vsyncDurationNs = C.TIME_UNSET; - vsyncOffsetNs = C.TIME_UNSET; - } - - /** Enables the helper. Must be called from the playback thread. */ - @TargetApi(17) // displayListener is null if Util.SDK_INT < 17. - public void enable() { - haveSync = false; - if (windowManager != null) { - vsyncSampler.addObserver(); - if (displayListener != null) { - displayListener.register(); - } - updateDefaultDisplayRefreshRateParams(); - } - } - - /** Disables the helper. Must be called from the playback thread. */ - @TargetApi(17) // displayListener is null if Util.SDK_INT < 17. - public void disable() { - if (windowManager != null) { - if (displayListener != null) { - displayListener.unregister(); - } - vsyncSampler.removeObserver(); - } - } - - /** - * Adjusts a frame release timestamp. Must be called from the playback thread. - * - * @param framePresentationTimeUs The frame's presentation time, in microseconds. - * @param unadjustedReleaseTimeNs The frame's unadjusted release time, in nanoseconds and in - * the same time base as {@link System#nanoTime()}. - * @return The adjusted frame release timestamp, in nanoseconds and in the same time base as - * {@link System#nanoTime()}. - */ - public long adjustReleaseTime(long framePresentationTimeUs, long unadjustedReleaseTimeNs) { - long framePresentationTimeNs = framePresentationTimeUs * 1000; - - // Until we know better, the adjustment will be a no-op. - long adjustedFrameTimeNs = framePresentationTimeNs; - long adjustedReleaseTimeNs = unadjustedReleaseTimeNs; - - if (haveSync) { - // See if we've advanced to the next frame. - if (framePresentationTimeUs != lastFramePresentationTimeUs) { - frameCount++; - adjustedLastFrameTimeNs = pendingAdjustedFrameTimeNs; - } - if (frameCount >= MIN_FRAMES_FOR_ADJUSTMENT) { - // We're synced and have waited the required number of frames to apply an adjustment. - // Calculate the average frame time across all the frames we've seen since the last sync. - // This will typically give us a frame rate at a finer granularity than the frame times - // themselves (which often only have millisecond granularity). - long averageFrameDurationNs = (framePresentationTimeNs - syncFramePresentationTimeNs) - / frameCount; - // Project the adjusted frame time forward using the average. - long candidateAdjustedFrameTimeNs = adjustedLastFrameTimeNs + averageFrameDurationNs; - - if (isDriftTooLarge(candidateAdjustedFrameTimeNs, unadjustedReleaseTimeNs)) { - haveSync = false; - } else { - adjustedFrameTimeNs = candidateAdjustedFrameTimeNs; - adjustedReleaseTimeNs = syncUnadjustedReleaseTimeNs + adjustedFrameTimeNs - - syncFramePresentationTimeNs; - } - } else { - // We're synced but haven't waited the required number of frames to apply an adjustment. - // Check drift anyway. - if (isDriftTooLarge(framePresentationTimeNs, unadjustedReleaseTimeNs)) { - haveSync = false; - } - } - } - - // If we need to sync, do so now. - if (!haveSync) { - syncFramePresentationTimeNs = framePresentationTimeNs; - syncUnadjustedReleaseTimeNs = unadjustedReleaseTimeNs; - frameCount = 0; - haveSync = true; - } - - lastFramePresentationTimeUs = framePresentationTimeUs; - pendingAdjustedFrameTimeNs = adjustedFrameTimeNs; - - if (vsyncSampler == null || vsyncDurationNs == C.TIME_UNSET) { - return adjustedReleaseTimeNs; - } - long sampledVsyncTimeNs = vsyncSampler.sampledVsyncTimeNs; - if (sampledVsyncTimeNs == C.TIME_UNSET) { - return adjustedReleaseTimeNs; - } - - // Find the timestamp of the closest vsync. This is the vsync that we're targeting. - long snappedTimeNs = closestVsync(adjustedReleaseTimeNs, sampledVsyncTimeNs, vsyncDurationNs); - // Apply an offset so that we release before the target vsync, but after the previous one. - return snappedTimeNs - vsyncOffsetNs; - } - - @RequiresApi(17) - private DefaultDisplayListener maybeBuildDefaultDisplayListenerV17(Context context) { - DisplayManager manager = (DisplayManager) context.getSystemService(Context.DISPLAY_SERVICE); - return manager == null ? null : new DefaultDisplayListener(manager); - } - - private void updateDefaultDisplayRefreshRateParams() { - // Note: If we fail to update the parameters, we leave them set to their previous values. - Display defaultDisplay = windowManager.getDefaultDisplay(); - if (defaultDisplay != null) { - double defaultDisplayRefreshRate = defaultDisplay.getRefreshRate(); - vsyncDurationNs = (long) (C.NANOS_PER_SECOND / defaultDisplayRefreshRate); - vsyncOffsetNs = (vsyncDurationNs * VSYNC_OFFSET_PERCENTAGE) / 100; - } - } - - private boolean isDriftTooLarge(long frameTimeNs, long releaseTimeNs) { - long elapsedFrameTimeNs = frameTimeNs - syncFramePresentationTimeNs; - long elapsedReleaseTimeNs = releaseTimeNs - syncUnadjustedReleaseTimeNs; - return Math.abs(elapsedReleaseTimeNs - elapsedFrameTimeNs) > MAX_ALLOWED_DRIFT_NS; - } - - private static long closestVsync(long releaseTime, long sampledVsyncTime, long vsyncDuration) { - long vsyncCount = (releaseTime - sampledVsyncTime) / vsyncDuration; - long snappedTimeNs = sampledVsyncTime + (vsyncDuration * vsyncCount); - long snappedBeforeNs; - long snappedAfterNs; - if (releaseTime <= snappedTimeNs) { - snappedBeforeNs = snappedTimeNs - vsyncDuration; - snappedAfterNs = snappedTimeNs; - } else { - snappedBeforeNs = snappedTimeNs; - snappedAfterNs = snappedTimeNs + vsyncDuration; - } - long snappedAfterDiff = snappedAfterNs - releaseTime; - long snappedBeforeDiff = releaseTime - snappedBeforeNs; - return snappedAfterDiff < snappedBeforeDiff ? snappedAfterNs : snappedBeforeNs; - } - - @RequiresApi(17) - private final class DefaultDisplayListener implements DisplayManager.DisplayListener { - - private final DisplayManager displayManager; - - public DefaultDisplayListener(DisplayManager displayManager) { - this.displayManager = displayManager; - } - - public void register() { - displayManager.registerDisplayListener(this, null); - } - - public void unregister() { - displayManager.unregisterDisplayListener(this); - } - - @Override - public void onDisplayAdded(int displayId) { - // Do nothing. - } - - @Override - public void onDisplayRemoved(int displayId) { - // Do nothing. - } - - @Override - public void onDisplayChanged(int displayId) { - if (displayId == Display.DEFAULT_DISPLAY) { - updateDefaultDisplayRefreshRateParams(); - } - } - - } - - /** - * Samples display vsync timestamps. A single instance using a single {@link Choreographer} is - * shared by all {@link VideoFrameReleaseTimeHelper} instances. This is done to avoid a resource - * leak in the platform on API levels prior to 23. See [Internal: b/12455729]. - */ - private static final class VSyncSampler implements FrameCallback, Handler.Callback { - - public volatile long sampledVsyncTimeNs; - - private static final int CREATE_CHOREOGRAPHER = 0; - private static final int MSG_ADD_OBSERVER = 1; - private static final int MSG_REMOVE_OBSERVER = 2; - - private static final VSyncSampler INSTANCE = new VSyncSampler(); - - private final Handler handler; - private final HandlerThread choreographerOwnerThread; - private Choreographer choreographer; - private int observerCount; - - public static VSyncSampler getInstance() { - return INSTANCE; - } - - private VSyncSampler() { - sampledVsyncTimeNs = C.TIME_UNSET; - choreographerOwnerThread = new HandlerThread("ExoPlayer:FrameReleaseChoreographer"); - choreographerOwnerThread.start(); - handler = Util.createHandler(choreographerOwnerThread.getLooper(), /* callback= */ this); - handler.sendEmptyMessage(CREATE_CHOREOGRAPHER); - } - - /** - * Notifies the sampler that a {@link VideoFrameReleaseTimeHelper} is observing - * {@link #sampledVsyncTimeNs}, and hence that the value should be periodically updated. - */ - public void addObserver() { - handler.sendEmptyMessage(MSG_ADD_OBSERVER); - } - - /** - * Notifies the sampler that a {@link VideoFrameReleaseTimeHelper} is no longer observing - * {@link #sampledVsyncTimeNs}. - */ - public void removeObserver() { - handler.sendEmptyMessage(MSG_REMOVE_OBSERVER); - } - - @Override - public void doFrame(long vsyncTimeNs) { - sampledVsyncTimeNs = vsyncTimeNs; - choreographer.postFrameCallbackDelayed(this, CHOREOGRAPHER_SAMPLE_DELAY_MILLIS); - } - - @Override - public boolean handleMessage(Message message) { - switch (message.what) { - case CREATE_CHOREOGRAPHER: { - createChoreographerInstanceInternal(); - return true; - } - case MSG_ADD_OBSERVER: { - addObserverInternal(); - return true; - } - case MSG_REMOVE_OBSERVER: { - removeObserverInternal(); - return true; - } - default: { - return false; - } - } - } - - private void createChoreographerInstanceInternal() { - choreographer = Choreographer.getInstance(); - } - - private void addObserverInternal() { - observerCount++; - if (observerCount == 1) { - choreographer.postFrameCallback(this); - } - } - - private void removeObserverInternal() { - observerCount--; - if (observerCount == 0) { - choreographer.removeFrameCallback(this); - sampledVsyncTimeNs = C.TIME_UNSET; - } - } - - } - -} diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/VideoRendererEventListener.java b/library/core/src/main/java/com/google/android/exoplayer2/video/VideoRendererEventListener.java index 992a262dabd..aacd8ce8d49 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/video/VideoRendererEventListener.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/video/VideoRendererEventListener.java @@ -25,6 +25,7 @@ import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.Renderer; import com.google.android.exoplayer2.decoder.DecoderCounters; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; import com.google.android.exoplayer2.util.Assertions; /** @@ -52,12 +53,23 @@ default void onVideoEnabled(DecoderCounters counters) {} default void onVideoDecoderInitialized( String decoderName, long initializedTimestampMs, long initializationDurationMs) {} + /** @deprecated Use {@link #onVideoInputFormatChanged(Format, DecoderReuseEvaluation)}. */ + @Deprecated + default void onVideoInputFormatChanged(Format format) {} + /** * Called when the format of the media being consumed by the renderer changes. * * @param format The new format. + * @param decoderReuseEvaluation The result of the evaluation to determine whether an existing + * decoder instance can be reused for the new format, or {@code null} if the renderer did not + * have a decoder. */ - default void onVideoInputFormatChanged(Format format) {} + @SuppressWarnings("deprecation") + default void onVideoInputFormatChanged( + Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { + onVideoInputFormatChanged(format); + } /** * Called to report the number of frames dropped by the renderer. Dropped frames are reported @@ -120,6 +132,13 @@ default void onVideoSizeChanged( */ default void onRenderedFirstFrame(@Nullable Surface surface) {} + /** + * Called when a decoder is released. + * + * @param decoderName The decoder that was released. + */ + default void onVideoDecoderReleased(String decoderName) {} + /** * Called when the renderer is disabled. * @@ -165,10 +184,15 @@ public void decoderInitialized( } } - /** Invokes {@link VideoRendererEventListener#onVideoInputFormatChanged(Format)}. */ - public void inputFormatChanged(Format format) { + /** + * Invokes {@link VideoRendererEventListener#onVideoInputFormatChanged(Format, + * DecoderReuseEvaluation)}. + */ + public void inputFormatChanged( + Format format, @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { if (handler != null) { - handler.post(() -> castNonNull(listener).onVideoInputFormatChanged(format)); + handler.post( + () -> castNonNull(listener).onVideoInputFormatChanged(format, decoderReuseEvaluation)); } } @@ -211,6 +235,13 @@ public void renderedFirstFrame(@Nullable Surface surface) { } } + /** Invokes {@link VideoRendererEventListener#onVideoDecoderReleased(String)}. */ + public void decoderReleased(String decoderName) { + if (handler != null) { + handler.post(() -> castNonNull(listener).onVideoDecoderReleased(decoderName)); + } + } + /** Invokes {@link VideoRendererEventListener#onVideoDisabled(DecoderCounters)}. */ public void disabled(DecoderCounters counters) { counters.ensureUpdated(); diff --git a/library/core/src/main/java/com/google/android/exoplayer2/video/spherical/CameraMotionRenderer.java b/library/core/src/main/java/com/google/android/exoplayer2/video/spherical/CameraMotionRenderer.java index 75902c0f142..287c62521e5 100644 --- a/library/core/src/main/java/com/google/android/exoplayer2/video/spherical/CameraMotionRenderer.java +++ b/library/core/src/main/java/com/google/android/exoplayer2/video/spherical/CameraMotionRenderer.java @@ -59,8 +59,8 @@ public String getName() { @Capabilities public int supportsFormat(Format format) { return MimeTypes.APPLICATION_CAMERA_MOTION.equals(format.sampleMimeType) - ? RendererCapabilities.create(FORMAT_HANDLED) - : RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + ? RendererCapabilities.create(C.FORMAT_HANDLED) + : RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } @Override diff --git a/library/core/src/test/java/com/google/android/exoplayer2/DefaultLivePlaybackSpeedControlTest.java b/library/core/src/test/java/com/google/android/exoplayer2/DefaultLivePlaybackSpeedControlTest.java new file mode 100644 index 00000000000..58f6522b1df --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/DefaultLivePlaybackSpeedControlTest.java @@ -0,0 +1,794 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.MediaItem.LiveConfiguration; +import com.google.common.collect.Iterables; +import java.time.Duration; +import java.util.ArrayList; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.shadows.ShadowSystemClock; + +/** Unit test for {@link DefaultLivePlaybackSpeedControl}. */ +@RunWith(AndroidJUnit4.class) +public class DefaultLivePlaybackSpeedControlTest { + + @Test + public void getTargetLiveOffsetUs_returnsUnset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + + assertThat(defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs()).isEqualTo(C.TIME_UNSET); + } + + @Test + public void getTargetLiveOffsetUs_afterSetLiveConfiguration_returnsMediaLiveOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + assertThat(defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs()).isEqualTo(42_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterSetLiveConfigurationWithTargetGreaterThanMax_returnsMaxLiveOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 4321, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + assertThat(defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs()).isEqualTo(400_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterSetLiveConfigurationWithTargetLessThanMin_returnsMinLiveOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 3, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + assertThat(defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs()).isEqualTo(5_000); + } + + @Test + public void getTargetLiveOffsetUs_afterSetTargetLiveOffsetOverrideUs_returnsOverride() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(321_000); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + long targetLiveOffsetUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetUs).isEqualTo(321_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterSetTargetLiveOffsetOverrideUsGreaterThanMax_returnsMaxLiveOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(123_456_789); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + long targetLiveOffsetUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetUs).isEqualTo(400_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterSetTargetLiveOffsetOverrideUsLessThanMin_returnsMinLiveOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(3_141); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + long targetLiveOffsetUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetUs).isEqualTo(5_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterSetTargetLiveOffsetOverrideWithoutMediaConfiguration_returnsUnset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(123_456_789); + + long targetLiveOffsetUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetUs).isEqualTo(C.TIME_UNSET); + } + + @Test + public void + getTargetLiveOffsetUs_afterSetTargetLiveOffsetOverrideWithTimeUnset_returnsMediaLiveOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(123_456_789); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(C.TIME_UNSET); + + long targetLiveOffsetUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetUs).isEqualTo(42_000); + } + + @Test + public void getTargetLiveOffsetUs_afterNotifyRebuffer_returnsIncreasedTargetOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(3) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + long targetLiveOffsetBeforeUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + defaultLivePlaybackSpeedControl.notifyRebuffer(); + long targetLiveOffsetAfterUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetAfterUs).isGreaterThan(targetLiveOffsetBeforeUs); + assertThat(targetLiveOffsetAfterUs - targetLiveOffsetBeforeUs).isEqualTo(3_000); + } + + @Test + public void getTargetLiveOffsetUs_afterRepeatedNotifyRebuffer_returnsMaxLiveOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(3) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + List targetOffsetsUs = new ArrayList<>(); + for (int i = 0; i < 500; i++) { + targetOffsetsUs.add(defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs()); + defaultLivePlaybackSpeedControl.notifyRebuffer(); + } + + assertThat(targetOffsetsUs).isInOrder(); + assertThat(Iterables.getLast(targetOffsetsUs)).isEqualTo(400_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterNotifyRebufferWithIncrementOfZero_returnsOriginalTargetOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(0) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + defaultLivePlaybackSpeedControl.notifyRebuffer(); + long targetLiveOffsetUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetUs).isEqualTo(42_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterNotifyRebufferAndSetTargetLiveOffsetOverrideUs_returnsOverride() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(3) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + defaultLivePlaybackSpeedControl.notifyRebuffer(); + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(321_000); + long targetLiveOffsetUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetUs).isEqualTo(321_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterNotifyRebufferAndSetLiveConfigurationWithSameOffset_returnsIncreasedTargetOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(3) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + long targetLiveOffsetBeforeUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + defaultLivePlaybackSpeedControl.notifyRebuffer(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 3, + /* maxLiveOffsetMs= */ 450, + /* minPlaybackSpeed= */ 0.9f, + /* maxPlaybackSpeed= */ 1.1f)); + long targetLiveOffsetAfterUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetAfterUs).isGreaterThan(targetLiveOffsetBeforeUs); + assertThat(targetLiveOffsetAfterUs - targetLiveOffsetBeforeUs).isEqualTo(3_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterNotifyRebufferAndSetLiveConfigurationWithNewOffset_returnsNewOffset() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(3) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42, + /* minLiveOffsetMs= */ 5, + /* maxLiveOffsetMs= */ 400, + /* minPlaybackSpeed= */ 1f, + /* maxPlaybackSpeed= */ 1f)); + + defaultLivePlaybackSpeedControl.notifyRebuffer(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 39, + /* minLiveOffsetMs= */ 3, + /* maxLiveOffsetMs= */ 450, + /* minPlaybackSpeed= */ 0.9f, + /* maxPlaybackSpeed= */ 1.1f)); + long targetLiveOffsetUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetUs).isEqualTo(39_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterNotifyRebufferAndAdjustPlaybackSpeedWithLargeBufferedDuration_returnsDecreasedOffsetToIdealTarget() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(3_000) + .setMinUpdateIntervalMs(100) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42_000, + /* minLiveOffsetMs= */ 5_000, + /* maxLiveOffsetMs= */ 400_000, + /* minPlaybackSpeed= */ 0.9f, + /* maxPlaybackSpeed= */ 1.1f)); + + defaultLivePlaybackSpeedControl.notifyRebuffer(); + long targetLiveOffsetAfterRebufferUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 45_000_000, /* bufferedDurationUs= */ 9_000_000); + long targetLiveOffsetAfterOneAdjustmentUs = + defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + for (int i = 0; i < 500; i++) { + ShadowSystemClock.advanceBy(Duration.ofMillis(100)); + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 45_000_000, /* bufferedDurationUs= */ 9_000_000); + } + long targetLiveOffsetAfterManyAdjustmentsUs = + defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetAfterOneAdjustmentUs).isLessThan(targetLiveOffsetAfterRebufferUs); + assertThat(targetLiveOffsetAfterManyAdjustmentsUs) + .isLessThan(targetLiveOffsetAfterOneAdjustmentUs); + assertThat(targetLiveOffsetAfterManyAdjustmentsUs).isEqualTo(42_000_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterNotifyRebufferAndAdjustPlaybackSpeedWithSmallBufferedDuration_returnsDecreasedOffsetToSafeTarget() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(3_000) + .setMinUpdateIntervalMs(100) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42_000, + /* minLiveOffsetMs= */ 5_000, + /* maxLiveOffsetMs= */ 400_000, + /* minPlaybackSpeed= */ 0.9f, + /* maxPlaybackSpeed= */ 1.1f)); + + defaultLivePlaybackSpeedControl.notifyRebuffer(); + long targetLiveOffsetAfterRebufferUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 45_000_000, /* bufferedDurationUs= */ 1_000_000); + long targetLiveOffsetAfterOneAdjustmentUs = + defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + for (int i = 0; i < 500; i++) { + ShadowSystemClock.advanceBy(Duration.ofMillis(100)); + long noiseUs = ((i % 10) - 5L) * 1_000; + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 45_000_000, /* bufferedDurationUs= */ 1_000_000 + noiseUs); + } + long targetLiveOffsetAfterManyAdjustmentsUs = + defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetAfterOneAdjustmentUs).isLessThan(targetLiveOffsetAfterRebufferUs); + assertThat(targetLiveOffsetAfterManyAdjustmentsUs) + .isLessThan(targetLiveOffsetAfterOneAdjustmentUs); + // Should be at least be at the minimum buffered position. + assertThat(targetLiveOffsetAfterManyAdjustmentsUs).isGreaterThan(44_005_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterAdjustPlaybackSpeedWithLiveOffsetAroundCurrentTarget_returnsSafeTarget() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42_000, + /* minLiveOffsetMs= */ 5_000, + /* maxLiveOffsetMs= */ 400_000, + /* minPlaybackSpeed= */ 0.9f, + /* maxPlaybackSpeed= */ 1.1f)); + + long targetLiveOffsetBeforeUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + // Pretend to have a buffered duration at around the target duration with some artificial noise. + for (int i = 0; i < 500; i++) { + ShadowSystemClock.advanceBy(Duration.ofMillis(100)); + long noiseUs = ((i % 10) - 5L) * 1_000; + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 49_000_000, /* bufferedDurationUs= */ 7_000_000 + noiseUs); + } + ShadowSystemClock.advanceBy(Duration.ofMillis(100)); + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 49_000_000, /* bufferedDurationUs= */ 7_000_000); + long targetLiveOffsetAfterUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetBeforeUs).isEqualTo(42_000_000); + assertThat(targetLiveOffsetAfterUs).isGreaterThan(42_005_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterAdjustPlaybackSpeedAndSmoothingFactorOfZero_ignoresSafeTargetAndReturnsCurrentTarget() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setMinPossibleLiveOffsetSmoothingFactor(0f) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42_000, + /* minLiveOffsetMs= */ 5_000, + /* maxLiveOffsetMs= */ 400_000, + /* minPlaybackSpeed= */ 0.9f, + /* maxPlaybackSpeed= */ 1.1f)); + + long targetLiveOffsetBeforeUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + // Pretend to have a buffered duration at around the target duration with some artificial noise. + for (int i = 0; i < 500; i++) { + ShadowSystemClock.advanceBy(Duration.ofMillis(100)); + long noiseUs = ((i % 10) - 5L) * 1_000; + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 49_000_000, /* bufferedDurationUs= */ 7_000_000 + noiseUs); + } + ShadowSystemClock.advanceBy(Duration.ofMillis(100)); + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 49_000_000, /* bufferedDurationUs= */ 7_000_000); + long targetLiveOffsetAfterUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetBeforeUs).isEqualTo(42_000_000); + // Despite the noise indicating it's unsafe here, we still return the target offset. + assertThat(targetLiveOffsetAfterUs).isEqualTo(42_000_000); + } + + @Test + public void + getTargetLiveOffsetUs_afterAdjustPlaybackSpeedWithLiveOffsetLessThanCurrentTarget_returnsCurrentTarget() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setTargetLiveOffsetIncrementOnRebufferMs(3_000) + .setMinUpdateIntervalMs(100) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 42_000, + /* minLiveOffsetMs= */ 5_000, + /* maxLiveOffsetMs= */ 400_000, + /* minPlaybackSpeed= */ 0.9f, + /* maxPlaybackSpeed= */ 1.1f)); + + long targetLiveOffsetBeforeUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 39_000_000, /* bufferedDurationUs= */ 1_000_000); + long targetLiveOffsetAfterUs = defaultLivePlaybackSpeedControl.getTargetLiveOffsetUs(); + + assertThat(targetLiveOffsetBeforeUs).isEqualTo(42_000_000); + assertThat(targetLiveOffsetAfterUs).isEqualTo(42_000_000); + } + + @Test + public void adjustPlaybackSpeed_liveOffsetMatchesTargetOffset_returnsUnitSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_000_000, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeed).isEqualTo(1f); + } + + @Test + public void adjustPlaybackSpeed_liveOffsetWithinAcceptableErrorMargin_returnsUnitSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder() + .setMaxLiveOffsetErrorMsForUnitSpeed(5) + .build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeedJustAboveLowerErrorMargin = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_000_000 - 5_000 + 1, /* bufferedDurationUs= */ 1_000_000); + float adjustedSpeedJustBelowUpperErrorMargin = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_000_000 + 5_000 - 1, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeedJustAboveLowerErrorMargin).isEqualTo(1f); + assertThat(adjustedSpeedJustBelowUpperErrorMargin).isEqualTo(1f); + } + + @Test + public void adjustPlaybackSpeed_withLiveOffsetGreaterThanTargetOffset_returnsAdjustedSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setProportionalControlFactor(0.01f).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_500_000, /* bufferedDurationUs= */ 1_000_000); + + float expectedSpeedAccordingToDocumentation = 1f + 0.01f * (2.5f - 2f); + assertThat(adjustedSpeed).isEqualTo(expectedSpeedAccordingToDocumentation); + assertThat(adjustedSpeed).isGreaterThan(1f); + } + + @Test + public void adjustPlaybackSpeed_withLiveOffsetLowerThanTargetOffset_returnsAdjustedSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setProportionalControlFactor(0.01f).build(); + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(2_000_000); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 1_500_000, /* bufferedDurationUs= */ 1_000_000); + + float expectedSpeedAccordingToDocumentation = 1f + 0.01f * (1.5f - 2f); + assertThat(adjustedSpeed).isEqualTo(expectedSpeedAccordingToDocumentation); + assertThat(adjustedSpeed).isLessThan(1f); + } + + @Test + public void + adjustPlaybackSpeed_withLiveOffsetGreaterThanTargetOffset_clampedToFallbackMaximumSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setFallbackMaxPlaybackSpeed(1.5f).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 999_999_999_999L, /* bufferedDurationUs= */ 999_999_999_999L); + + assertThat(adjustedSpeed).isEqualTo(1.5f); + } + + @Test + public void + adjustPlaybackSpeed_withLiveOffsetLowerThanTargetOffset_clampedToFallbackMinimumSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setFallbackMinPlaybackSpeed(0.5f).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ -999_999_999_999L, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeed).isEqualTo(0.5f); + } + + @Test + public void + adjustPlaybackSpeed_andMediaProvidedMaxSpeedWithLiveOffsetGreaterThanTargetOffset_clampedToMediaMaxSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setFallbackMaxPlaybackSpeed(1.5f).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ 2f)); + + float adjustedSpeed = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 999_999_999_999L, /* bufferedDurationUs= */ 999_999_999_999L); + + assertThat(adjustedSpeed).isEqualTo(2f); + } + + @Test + public void + adjustPlaybackSpeed_andMediaProvidedMinSpeedWithLiveOffsetLowerThanTargetOffset_clampedToMediaMinSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setFallbackMinPlaybackSpeed(0.5f).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ 0.2f, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ -999_999_999_999L, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeed).isEqualTo(0.2f); + } + + @Test + public void adjustPlaybackSpeed_repeatedCallWithinMinUpdateInterval_returnsSameAdjustedSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setMinUpdateIntervalMs(123).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed1 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 1_500_000, /* bufferedDurationUs= */ 1_000_000); + ShadowSystemClock.advanceBy(Duration.ofMillis(122)); + float adjustedSpeed2 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_500_000, /* bufferedDurationUs= */ 1_000_000); + ShadowSystemClock.advanceBy(Duration.ofMillis(2)); + float adjustedSpeed3 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_500_000, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeed1).isEqualTo(adjustedSpeed2); + assertThat(adjustedSpeed3).isNotEqualTo(adjustedSpeed2); + } + + @Test + public void + adjustPlaybackSpeed_repeatedCallAfterSetLiveConfigurationWithSameOffset_returnsSameAdjustedSpeed() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setMinUpdateIntervalMs(123).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed1 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 1_500_000, /* bufferedDurationUs= */ 1_000_000); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + float adjustedSpeed2 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_500_000, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeed1).isEqualTo(adjustedSpeed2); + } + + @Test + public void + adjustPlaybackSpeed_repeatedCallAfterSetLiveConfigurationWithNewOffset_updatesSpeedAgain() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setMinUpdateIntervalMs(123).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed1 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 1_500_000, /* bufferedDurationUs= */ 1_000_000); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 1_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + float adjustedSpeed2 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_500_000, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeed1).isNotEqualTo(adjustedSpeed2); + } + + @Test + public void + adjustPlaybackSpeed_repeatedCallAfterSetTargetLiveOffsetOverrideUs_updatesSpeedAgain() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setMinUpdateIntervalMs(123).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed1 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 1_500_000, /* bufferedDurationUs= */ 1_000_000); + defaultLivePlaybackSpeedControl.setTargetLiveOffsetOverrideUs(2_000_001); + float adjustedSpeed2 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_500_000, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeed1).isNotEqualTo(adjustedSpeed2); + } + + @Test + public void adjustPlaybackSpeed_repeatedCallAfterNotifyRebuffer_updatesSpeedAgain() { + DefaultLivePlaybackSpeedControl defaultLivePlaybackSpeedControl = + new DefaultLivePlaybackSpeedControl.Builder().setMinUpdateIntervalMs(123).build(); + defaultLivePlaybackSpeedControl.setLiveConfiguration( + new LiveConfiguration( + /* targetLiveOffsetMs= */ 2_000, + /* minLiveOffsetMs= */ C.TIME_UNSET, + /* maxLiveOffsetMs= */ C.TIME_UNSET, + /* minPlaybackSpeed= */ C.RATE_UNSET, + /* maxPlaybackSpeed= */ C.RATE_UNSET)); + + float adjustedSpeed1 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 1_500_000, /* bufferedDurationUs= */ 1_000_000); + defaultLivePlaybackSpeedControl.notifyRebuffer(); + float adjustedSpeed2 = + defaultLivePlaybackSpeedControl.getAdjustedPlaybackSpeed( + /* liveOffsetUs= */ 2_500_000, /* bufferedDurationUs= */ 1_000_000); + + assertThat(adjustedSpeed1).isNotEqualTo(adjustedSpeed2); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/DefaultLoadControlTest.java b/library/core/src/test/java/com/google/android/exoplayer2/DefaultLoadControlTest.java index b00da4390a4..1cebbbd0119 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/DefaultLoadControlTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/DefaultLoadControlTest.java @@ -20,7 +20,7 @@ import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.DefaultLoadControl.Builder; import com.google.android.exoplayer2.source.TrackGroupArray; -import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.DefaultAllocator; import org.junit.Before; import org.junit.Test; @@ -174,6 +174,17 @@ public void shouldContinueLoadingWithMinBufferReached_inFastPlayback() { .isTrue(); } + @Test + public void shouldContinueLoading_withNoSelectedTracks_returnsTrue() { + loadControl = builder.build(); + loadControl.onTracksSelected(new Renderer[0], TrackGroupArray.EMPTY, new ExoTrackSelection[0]); + + assertThat( + loadControl.shouldContinueLoading( + /* playbackPositionUs= */ 0, /* bufferedDurationUs= */ 0, /* playbackSpeed= */ 1f)) + .isTrue(); + } + @Test public void shouldNotContinueLoadingWithMaxBufferReached_inFastPlayback() { build(); @@ -185,21 +196,117 @@ public void shouldNotContinueLoadingWithMaxBufferReached_inFastPlayback() { } @Test - public void startsPlayback_whenMinBufferSizeReached() { + public void shouldStartPlayback_whenMinBufferSizeReached_returnsTrue() { build(); - assertThat(loadControl.shouldStartPlayback(MIN_BUFFER_US, SPEED, /* rebuffering= */ false)) + assertThat( + loadControl.shouldStartPlayback( + MIN_BUFFER_US, + SPEED, + /* rebuffering= */ false, + /* targetLiveOffsetUs= */ C.TIME_UNSET)) .isTrue(); } @Test - public void shouldContinueLoading_withNoSelectedTracks_returnsTrue() { - loadControl = builder.build(); - loadControl.onTracksSelected(new Renderer[0], TrackGroupArray.EMPTY, new TrackSelectionArray()); + public void + shouldStartPlayback_withoutTargetLiveOffset_returnsTrueWhenBufferForPlaybackReached() { + builder.setBufferDurationsMs( + /* minBufferMs= */ 5_000, + /* maxBufferMs= */ 20_000, + /* bufferForPlaybackMs= */ 3_000, + /* bufferForPlaybackAfterRebufferMs= */ 4_000); + build(); assertThat( - loadControl.shouldContinueLoading( - /* playbackPositionUs= */ 0, /* bufferedDurationUs= */ 0, /* playbackSpeed= */ 1f)) + loadControl.shouldStartPlayback( + /* bufferedDurationUs= */ 2_999_999, + SPEED, + /* rebuffering= */ false, + /* targetLiveOffsetUs= */ C.TIME_UNSET)) + .isFalse(); + assertThat( + loadControl.shouldStartPlayback( + /* bufferedDurationUs= */ 3_000_000, + SPEED, + /* rebuffering= */ false, + /* targetLiveOffsetUs= */ C.TIME_UNSET)) + .isTrue(); + } + + @Test + public void shouldStartPlayback_withTargetLiveOffset_returnsTrueWhenHalfLiveOffsetReached() { + builder.setBufferDurationsMs( + /* minBufferMs= */ 5_000, + /* maxBufferMs= */ 20_000, + /* bufferForPlaybackMs= */ 3_000, + /* bufferForPlaybackAfterRebufferMs= */ 4_000); + build(); + + assertThat( + loadControl.shouldStartPlayback( + /* bufferedDurationUs= */ 499_999, + SPEED, + /* rebuffering= */ true, + /* targetLiveOffsetUs= */ 1_000_000)) + .isFalse(); + assertThat( + loadControl.shouldStartPlayback( + /* bufferedDurationUs= */ 500_000, + SPEED, + /* rebuffering= */ true, + /* targetLiveOffsetUs= */ 1_000_000)) + .isTrue(); + } + + @Test + public void + shouldStartPlayback_afterRebuffer_withoutTargetLiveOffset_whenBufferForPlaybackAfterRebufferReached() { + builder.setBufferDurationsMs( + /* minBufferMs= */ 5_000, + /* maxBufferMs= */ 20_000, + /* bufferForPlaybackMs= */ 3_000, + /* bufferForPlaybackAfterRebufferMs= */ 4_000); + build(); + + assertThat( + loadControl.shouldStartPlayback( + /* bufferedDurationUs= */ 3_999_999, + SPEED, + /* rebuffering= */ true, + /* targetLiveOffsetUs= */ C.TIME_UNSET)) + .isFalse(); + assertThat( + loadControl.shouldStartPlayback( + /* bufferedDurationUs= */ 4_000_000, + SPEED, + /* rebuffering= */ true, + /* targetLiveOffsetUs= */ C.TIME_UNSET)) + .isTrue(); + } + + @Test + public void shouldStartPlayback_afterRebuffer_withTargetLiveOffset_whenHalfLiveOffsetReached() { + builder.setBufferDurationsMs( + /* minBufferMs= */ 5_000, + /* maxBufferMs= */ 20_000, + /* bufferForPlaybackMs= */ 3_000, + /* bufferForPlaybackAfterRebufferMs= */ 4_000); + build(); + + assertThat( + loadControl.shouldStartPlayback( + /* bufferedDurationUs= */ 499_999, + SPEED, + /* rebuffering= */ true, + /* targetLiveOffsetUs= */ 1_000_000)) + .isFalse(); + assertThat( + loadControl.shouldStartPlayback( + /* bufferedDurationUs= */ 500_000, + SPEED, + /* rebuffering= */ true, + /* targetLiveOffsetUs= */ 1_000_000)) .isTrue(); } @@ -214,5 +321,4 @@ private void makeSureTargetBufferBytesReached() { allocator.allocate(); } } - } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/ExoPlayerTest.java b/library/core/src/test/java/com/google/android/exoplayer2/ExoPlayerTest.java index e7e35902ec8..008d8c6b537 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/ExoPlayerTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/ExoPlayerTest.java @@ -15,21 +15,24 @@ */ package com.google.android.exoplayer2; +import static com.google.android.exoplayer2.robolectric.RobolectricUtil.runMainLooperUntil; +import static com.google.android.exoplayer2.robolectric.TestPlayerRunHelper.playUntilStartOfWindow; +import static com.google.android.exoplayer2.robolectric.TestPlayerRunHelper.runUntilPlaybackState; +import static com.google.android.exoplayer2.robolectric.TestPlayerRunHelper.runUntilReceiveOffloadSchedulingEnabledNewState; +import static com.google.android.exoplayer2.robolectric.TestPlayerRunHelper.runUntilSleepingForOffload; +import static com.google.android.exoplayer2.robolectric.TestPlayerRunHelper.runUntilTimelineChanged; import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.oneByteSample; -import static com.google.android.exoplayer2.testutil.TestExoPlayer.playUntilStartOfWindow; -import static com.google.android.exoplayer2.testutil.TestExoPlayer.runUntilPlaybackState; -import static com.google.android.exoplayer2.testutil.TestExoPlayer.runUntilReceiveOffloadSchedulingEnabledNewState; -import static com.google.android.exoplayer2.testutil.TestExoPlayer.runUntilTimelineChanged; -import static com.google.android.exoplayer2.testutil.TestUtil.runMainLooperUntil; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertArrayEquals; import static org.junit.Assert.assertThrows; import static org.junit.Assert.fail; import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyBoolean; import static org.mockito.ArgumentMatchers.anyInt; import static org.mockito.ArgumentMatchers.argThat; import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.atLeastOnce; import static org.mockito.Mockito.inOrder; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.never; @@ -44,7 +47,6 @@ import android.net.Uri; import android.os.Looper; import android.view.Surface; -import android.view.ViewGroup; import androidx.annotation.Nullable; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; @@ -55,23 +57,23 @@ import com.google.android.exoplayer2.audio.AudioAttributes; import com.google.android.exoplayer2.drm.DrmSessionEventListener; import com.google.android.exoplayer2.drm.DrmSessionManager; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.id3.BinaryFrame; +import com.google.android.exoplayer2.metadata.id3.TextInformationFrame; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; import com.google.android.exoplayer2.source.ClippingMediaSource; import com.google.android.exoplayer2.source.CompositeMediaSource; import com.google.android.exoplayer2.source.ConcatenatingMediaSource; -import com.google.android.exoplayer2.source.DefaultMediaSourceFactory; import com.google.android.exoplayer2.source.LoopingMediaSource; import com.google.android.exoplayer2.source.MaskingMediaSource; import com.google.android.exoplayer2.source.MediaPeriod; import com.google.android.exoplayer2.source.MediaSource; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.MediaSourceEventListener; -import com.google.android.exoplayer2.source.SampleStream; import com.google.android.exoplayer2.source.SilenceMediaSource; import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.source.ads.AdPlaybackState; -import com.google.android.exoplayer2.source.ads.AdsLoader; -import com.google.android.exoplayer2.source.ads.AdsMediaSource; import com.google.android.exoplayer2.testutil.Action; import com.google.android.exoplayer2.testutil.ActionSchedule; import com.google.android.exoplayer2.testutil.ActionSchedule.PlayerRunnable; @@ -87,25 +89,26 @@ import com.google.android.exoplayer2.testutil.FakeMediaSource; import com.google.android.exoplayer2.testutil.FakeRenderer; import com.google.android.exoplayer2.testutil.FakeSampleStream; +import com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem; import com.google.android.exoplayer2.testutil.FakeShuffleOrder; import com.google.android.exoplayer2.testutil.FakeTimeline; import com.google.android.exoplayer2.testutil.FakeTimeline.TimelineWindowDefinition; import com.google.android.exoplayer2.testutil.FakeTrackSelection; import com.google.android.exoplayer2.testutil.FakeTrackSelector; import com.google.android.exoplayer2.testutil.NoUidTimeline; -import com.google.android.exoplayer2.testutil.TestExoPlayer; +import com.google.android.exoplayer2.testutil.TestExoPlayerBuilder; import com.google.android.exoplayer2.trackselection.DefaultTrackSelector; -import com.google.android.exoplayer2.trackselection.TrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; import com.google.android.exoplayer2.upstream.Allocation; import com.google.android.exoplayer2.upstream.Allocator; -import com.google.android.exoplayer2.upstream.DataSpec; import com.google.android.exoplayer2.upstream.Loader; import com.google.android.exoplayer2.upstream.TransferListener; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Clock; +import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.collect.ImmutableList; -import com.google.common.collect.Lists; +import com.google.common.collect.Iterables; +import com.google.common.collect.Range; import java.io.IOException; import java.util.ArrayList; import java.util.Arrays; @@ -113,13 +116,13 @@ import java.util.HashSet; import java.util.List; import java.util.concurrent.CountDownLatch; -import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicReference; import java.util.stream.Collectors; import org.junit.Before; +import org.junit.Ignore; import org.junit.Test; import org.junit.runner.RunWith; import org.mockito.ArgumentCaptor; @@ -127,6 +130,7 @@ import org.mockito.InOrder; import org.mockito.Mockito; import org.robolectric.shadows.ShadowAudioManager; +import org.robolectric.shadows.ShadowLooper; /** Unit test for {@link ExoPlayer}. */ @RunWith(AndroidJUnit4.class) @@ -163,7 +167,7 @@ public void playEmptyTimeline() throws Exception { new MaskingMediaSource.PlaceholderTimeline(FakeMediaSource.FAKE_MEDIA_ITEM); FakeRenderer renderer = new FakeRenderer(C.TRACK_TYPE_UNKNOWN); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(renderer).build(); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(renderer).build(); EventListener mockEventListener = mock(EventListener.class); player.addListener(mockEventListener); @@ -191,9 +195,9 @@ public void playEmptyTimeline() throws Exception { /** Tests playback of a source that exposes a single period. */ @Test public void playSinglePeriodTimeline() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); FakeRenderer renderer = new FakeRenderer(C.TRACK_TYPE_VIDEO); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(renderer).build(); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(renderer).build(); EventListener mockEventListener = mock(EventListener.class); player.addListener(mockEventListener); @@ -227,7 +231,7 @@ public void playSinglePeriodTimeline() throws Exception { public void playMultiPeriodTimeline() throws Exception { Timeline timeline = new FakeTimeline(/* windowCount= */ 3); FakeRenderer renderer = new FakeRenderer(C.TRACK_TYPE_VIDEO); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(renderer).build(); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(renderer).build(); EventListener mockEventListener = mock(EventListener.class); player.addListener(mockEventListener); @@ -265,7 +269,7 @@ public void playShortDurationPeriods() throws Exception { Timeline timeline = new FakeTimeline(new TimelineWindowDefinition(/* periodCount= */ 100, /* id= */ 0)); FakeRenderer renderer = new FakeRenderer(C.TRACK_TYPE_VIDEO); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(renderer).build(); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(renderer).build(); EventListener mockEventListener = mock(EventListener.class); player.addListener(mockEventListener); @@ -347,7 +351,7 @@ public boolean isEnded() { } }; SimpleExoPlayer player = - new TestExoPlayer.Builder(context).setRenderers(videoRenderer, audioRenderer).build(); + new TestExoPlayerBuilder(context).setRenderers(videoRenderer, audioRenderer).build(); EventListener mockEventListener = mock(EventListener.class); player.addListener(mockEventListener); @@ -386,8 +390,7 @@ public void resettingMediaSourcesGivesFreshSourceInfo() throws Exception { MediaSource firstSource = new FakeMediaSource(firstTimeline, ExoPlayerTestRunner.VIDEO_FORMAT); AtomicBoolean secondSourcePrepared = new AtomicBoolean(); MediaSource secondSource = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.VIDEO_FORMAT) { + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT) { @Override public synchronized void prepareSourceInternal( @Nullable TransferListener mediaTransferListener) { @@ -395,9 +398,9 @@ public synchronized void prepareSourceInternal( secondSourcePrepared.set(true); } }; - Timeline thirdTimeline = new FakeTimeline(/* windowCount= */ 1); + Timeline thirdTimeline = new FakeTimeline(); MediaSource thirdSource = new FakeMediaSource(thirdTimeline, ExoPlayerTestRunner.VIDEO_FORMAT); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(renderer).build(); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(renderer).build(); EventListener mockEventListener = mock(EventListener.class); player.addListener(mockEventListener); @@ -445,7 +448,7 @@ public synchronized void prepareSourceInternal( public void repeatModeChanges() throws Exception { Timeline timeline = new FakeTimeline(/* windowCount= */ 3); FakeRenderer renderer = new FakeRenderer(C.TRACK_TYPE_VIDEO); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(renderer).build(); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(renderer).build(); AnalyticsListener mockAnalyticsListener = mock(AnalyticsListener.class); player.addAnalyticsListener(mockAnalyticsListener); @@ -485,7 +488,7 @@ public void repeatModeChanges() throws Exception { @Test public void shuffleModeEnabledChanges() throws Exception { - Timeline fakeTimeline = new FakeTimeline(/* windowCount= */ 1); + Timeline fakeTimeline = new FakeTimeline(); MediaSource[] fakeMediaSources = { new FakeMediaSource(fakeTimeline, ExoPlayerTestRunner.VIDEO_FORMAT), new FakeMediaSource(fakeTimeline, ExoPlayerTestRunner.VIDEO_FORMAT), @@ -648,7 +651,7 @@ public void seekDiscontinuityWithAdjustment() throws Exception { FakeMediaSource mediaSource = new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -659,6 +662,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( FakeMediaPeriod mediaPeriod = new FakeMediaPeriod( trackGroupArray, + allocator, TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, mediaSourceEventDispatcher, drmSessionManager, @@ -692,7 +696,7 @@ public void internalDiscontinuityAtNewPosition() throws Exception { FakeMediaSource mediaSource = new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -703,6 +707,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( FakeMediaPeriod mediaPeriod = new FakeMediaPeriod( trackGroupArray, + allocator, TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, mediaSourceEventDispatcher); mediaPeriod.setDiscontinuityPositionUs(10); @@ -720,11 +725,11 @@ protected FakeMediaPeriod createFakeMediaPeriod( @Test public void internalDiscontinuityAtInitialPosition() throws Exception { - FakeTimeline timeline = new FakeTimeline(/* windowCount= */ 1); + FakeTimeline timeline = new FakeTimeline(); FakeMediaSource mediaSource = new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -735,6 +740,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( FakeMediaPeriod mediaPeriod = new FakeMediaPeriod( trackGroupArray, + allocator, TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, mediaSourceEventDispatcher); // Set a discontinuity at the position this period is supposed to start at anyway. @@ -755,10 +761,9 @@ protected FakeMediaPeriod createFakeMediaPeriod( @Test public void allActivatedTrackSelectionAreReleasedForSinglePeriod() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); MediaSource mediaSource = new FakeMediaSource( - timeline, ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); FakeRenderer videoRenderer = new FakeRenderer(C.TRACK_TYPE_VIDEO); FakeRenderer audioRenderer = new FakeRenderer(C.TRACK_TYPE_AUDIO); FakeTrackSelector trackSelector = new FakeTrackSelector(); @@ -815,10 +820,9 @@ public void allActivatedTrackSelectionAreReleasedForMultiPeriods() throws Except @Test public void allActivatedTrackSelectionAreReleasedWhenTrackSelectionsAreRemade() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); MediaSource mediaSource = new FakeMediaSource( - timeline, ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); FakeRenderer videoRenderer = new FakeRenderer(C.TRACK_TYPE_VIDEO); FakeRenderer audioRenderer = new FakeRenderer(C.TRACK_TYPE_AUDIO); final FakeTrackSelector trackSelector = new FakeTrackSelector(); @@ -854,10 +858,9 @@ public void allActivatedTrackSelectionAreReleasedWhenTrackSelectionsAreRemade() @Test public void allActivatedTrackSelectionAreReleasedWhenTrackSelectionsAreReused() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); MediaSource mediaSource = new FakeMediaSource( - timeline, ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); FakeRenderer videoRenderer = new FakeRenderer(C.TRACK_TYPE_VIDEO); FakeRenderer audioRenderer = new FakeRenderer(C.TRACK_TYPE_AUDIO); final FakeTrackSelector trackSelector = @@ -971,10 +974,9 @@ public void setPlaybackSpeedBeforePreparationCompletesSucceeds() throws Exceptio final CountDownLatch createPeriodCalledCountDownLatch = new CountDownLatch(1); final FakeMediaPeriod[] fakeMediaPeriodHolder = new FakeMediaPeriod[1]; MediaSource mediaSource = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.VIDEO_FORMAT) { + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -986,6 +988,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( fakeMediaPeriodHolder[0] = new FakeMediaPeriod( trackGroupArray, + allocator, TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, mediaSourceEventDispatcher, drmSessionManager, @@ -1024,11 +1027,10 @@ protected FakeMediaPeriod createFakeMediaPeriod( public void seekBeforePreparationCompletes_seeksToCorrectPosition() throws Exception { CountDownLatch createPeriodCalledCountDownLatch = new CountDownLatch(1); FakeMediaPeriod[] fakeMediaPeriodHolder = new FakeMediaPeriod[1]; - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); FakeMediaSource mediaSource = new FakeMediaSource(/* timeline= */ null, ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -1040,6 +1042,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( fakeMediaPeriodHolder[0] = new FakeMediaPeriod( trackGroupArray, + allocator, TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, mediaSourceEventDispatcher, drmSessionManager, @@ -1056,7 +1059,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( .waitForPlaybackState(Player.STATE_BUFFERING) // Ensure we use the MaskingMediaPeriod by delaying the initial timeline update. .delay(1) - .executeRunnable(() -> mediaSource.setNewSourceInfo(timeline)) + .executeRunnable(() -> mediaSource.setNewSourceInfo(new FakeTimeline())) .waitForTimelineChanged() // Block until createPeriod has been called on the fake media source. .executeRunnable( @@ -1094,13 +1097,12 @@ public void run(SimpleExoPlayer player) { @Test public void stop_withoutReset_doesNotResetPosition_correctMasking() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); int[] currentWindowIndex = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; long[] currentPosition = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; long[] bufferedPosition = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; long[] totalBufferedDuration = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; final FakeMediaSource mediaSource = - new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .pause() @@ -1165,7 +1167,7 @@ public void run(SimpleExoPlayer player) { @Test public void stop_withoutReset_releasesMediaSource() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); final FakeMediaSource mediaSource = new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT); ActionSchedule actionSchedule = @@ -1187,13 +1189,12 @@ public void stop_withoutReset_releasesMediaSource() throws Exception { @Test public void stop_withReset_doesResetPosition_correctMasking() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); int[] currentWindowIndex = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; long[] currentPosition = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; long[] bufferedPosition = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; long[] totalBufferedDuration = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; final FakeMediaSource mediaSource = - new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .pause() @@ -1259,7 +1260,7 @@ public void run(SimpleExoPlayer player) { @Test public void stop_withReset_releasesMediaSource() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); final FakeMediaSource mediaSource = new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT); ActionSchedule actionSchedule = @@ -1281,13 +1282,12 @@ public void stop_withReset_releasesMediaSource() throws Exception { @Test public void release_correctMasking() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); int[] currentWindowIndex = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; long[] currentPosition = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; long[] bufferedPosition = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; long[] totalBufferedDuration = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; final FakeMediaSource mediaSource = - new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .pause() @@ -1346,7 +1346,7 @@ public void run(SimpleExoPlayer player) { @Test public void settingNewStartPositionPossibleAfterStopWithReset() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 2); MediaSource secondSource = new FakeMediaSource(secondTimeline, ExoPlayerTestRunner.VIDEO_FORMAT); @@ -1572,7 +1572,6 @@ public void run(SimpleExoPlayer player) { @Test public void stopDuringPreparationOverwritesPreparation() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .waitForPlaybackState(Player.STATE_BUFFERING) @@ -1581,7 +1580,7 @@ public void stopDuringPreparationOverwritesPreparation() throws Exception { .build(); ExoPlayerTestRunner testRunner = new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) + .setTimeline(new FakeTimeline()) .setActionSchedule(actionSchedule) .build() .start() @@ -1595,7 +1594,7 @@ public void stopDuringPreparationOverwritesPreparation() throws Exception { @Test public void stopAndSeekAfterStopDoesNotResetTimeline() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .waitForPlaybackState(Player.STATE_READY) @@ -1621,7 +1620,7 @@ public void stopAndSeekAfterStopDoesNotResetTimeline() throws Exception { @Test public void reprepareAfterPlaybackError() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .waitForPlaybackState(Player.STATE_READY) @@ -1649,7 +1648,7 @@ public void reprepareAfterPlaybackError() throws Exception { @Test public void seekAndReprepareAfterPlaybackError() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); final long[] positionHolder = new long[2]; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -1736,8 +1735,6 @@ public void run(SimpleExoPlayer player) { @Test public void restartAfterEmptyTimelineWithShuffleModeEnabledUsesCorrectFirstPeriod() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); - FakeMediaSource mediaSource = new FakeMediaSource(timeline); ConcatenatingMediaSource concatenatingMediaSource = new ConcatenatingMediaSource(/* isAtomic= */ false, new FakeShuffleOrder(0)); AtomicInteger windowIndexAfterAddingSources = new AtomicInteger(); @@ -1751,7 +1748,7 @@ public void restartAfterEmptyTimelineWithShuffleModeEnabledUsesCorrectFirstPerio .executeRunnable( () -> concatenatingMediaSource.addMediaSources( - Arrays.asList(mediaSource, mediaSource))) + ImmutableList.of(new FakeMediaSource(), new FakeMediaSource()))) .waitForTimelineChanged() .executeRunnable( new PlayerRunnable() { @@ -1903,7 +1900,6 @@ public void run(SimpleExoPlayer player) { @Test public void playbackErrorAndReprepareWithPositionResetKeepsWindowSequenceNumber() throws Exception { - FakeMediaSource mediaSource = new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .pause() @@ -1937,7 +1933,6 @@ public void onPlayWhenReadyChanged( }; ExoPlayerTestRunner testRunner = new ExoPlayerTestRunner.Builder(context) - .setMediaSources(mediaSource) .setActionSchedule(actionSchedule) .setAnalyticsListener(listener) .build(); @@ -1952,7 +1947,7 @@ public void onPlayWhenReadyChanged( @Test public void playbackErrorTwiceStillKeepsTimeline() throws Exception { - final Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + final Timeline timeline = new FakeTimeline(); final FakeMediaSource mediaSource2 = new FakeMediaSource(timeline); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -1988,7 +1983,6 @@ public void playbackErrorTwiceStillKeepsTimeline() throws Exception { @Test public void sendMessagesDuringPreparation() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -1998,7 +1992,6 @@ public void sendMessagesDuringPreparation() throws Exception { .play() .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2008,7 +2001,7 @@ public void sendMessagesDuringPreparation() throws Exception { @Test public void sendMessagesAfterPreparation() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -2029,7 +2022,6 @@ public void sendMessagesAfterPreparation() throws Exception { @Test public void multipleSendMessages() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); PositionGrabbingMessageTarget target50 = new PositionGrabbingMessageTarget(); PositionGrabbingMessageTarget target80 = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = @@ -2041,7 +2033,6 @@ public void multipleSendMessages() throws Exception { .play() .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2053,7 +2044,6 @@ public void multipleSendMessages() throws Exception { @Test public void sendMessagesFromStartPositionOnlyOnce() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); AtomicInteger counter = new AtomicInteger(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -2069,7 +2059,6 @@ public void sendMessagesFromStartPositionOnlyOnce() throws Exception { .play() .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2080,7 +2069,6 @@ public void sendMessagesFromStartPositionOnlyOnce() throws Exception { @Test public void multipleSendMessagesAtSameTime() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); PositionGrabbingMessageTarget target1 = new PositionGrabbingMessageTarget(); PositionGrabbingMessageTarget target2 = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = @@ -2092,7 +2080,6 @@ public void multipleSendMessagesAtSameTime() throws Exception { .play() .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2185,7 +2172,6 @@ public void sendMessagesAtStartAndEndOfPeriod() throws Exception { @Test public void sendMessagesSeekOnDeliveryTimeDuringPreparation() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -2194,7 +2180,6 @@ public void sendMessagesSeekOnDeliveryTimeDuringPreparation() throws Exception { .seek(/* positionMs= */ 50) .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2204,7 +2189,7 @@ public void sendMessagesSeekOnDeliveryTimeDuringPreparation() throws Exception { @Test public void sendMessagesSeekOnDeliveryTimeAfterPreparation() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -2225,7 +2210,6 @@ public void sendMessagesSeekOnDeliveryTimeAfterPreparation() throws Exception { @Test public void sendMessagesSeekAfterDeliveryTimeDuringPreparation() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -2236,7 +2220,6 @@ public void sendMessagesSeekAfterDeliveryTimeDuringPreparation() throws Exceptio .play() .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2246,7 +2229,7 @@ public void sendMessagesSeekAfterDeliveryTimeDuringPreparation() throws Exceptio @Test public void sendMessagesSeekAfterDeliveryTimeAfterPreparation() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -2268,7 +2251,6 @@ public void sendMessagesSeekAfterDeliveryTimeAfterPreparation() throws Exception @Test public void sendMessagesRepeatDoesNotRepost() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -2281,7 +2263,6 @@ public void sendMessagesRepeatDoesNotRepost() throws Exception { .setRepeatMode(Player.REPEAT_MODE_OFF) .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2292,7 +2273,6 @@ public void sendMessagesRepeatDoesNotRepost() throws Exception { @Test public void sendMessagesRepeatWithoutDeletingDoesRepost() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -2310,7 +2290,6 @@ public void sendMessagesRepeatWithoutDeletingDoesRepost() throws Exception { .play() .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2431,7 +2410,7 @@ public void sendMessagesMoveWindowIndex() throws Exception { @Test public void sendMessagesNonLinearPeriodOrder() throws Exception { - Timeline fakeTimeline = new FakeTimeline(/* windowCount= */ 1); + Timeline fakeTimeline = new FakeTimeline(); MediaSource[] fakeMediaSources = { new FakeMediaSource(fakeTimeline, ExoPlayerTestRunner.VIDEO_FORMAT), new FakeMediaSource(fakeTimeline, ExoPlayerTestRunner.VIDEO_FORMAT), @@ -2466,7 +2445,6 @@ public void sendMessagesNonLinearPeriodOrder() throws Exception { @Test public void cancelMessageBeforeDelivery() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); final PositionGrabbingMessageTarget target = new PositionGrabbingMessageTarget(); final AtomicReference message = new AtomicReference<>(); ActionSchedule actionSchedule = @@ -2487,7 +2465,6 @@ public void run(SimpleExoPlayer player) { .play() .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2498,7 +2475,6 @@ public void run(SimpleExoPlayer player) { @Test public void cancelRepeatedMessageAfterDelivery() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); final CountingMessageTarget target = new CountingMessageTarget(); final AtomicReference message = new AtomicReference<>(); ActionSchedule actionSchedule = @@ -2525,7 +2501,6 @@ public void run(SimpleExoPlayer player) { .play() .build(); new ExoPlayerTestRunner.Builder(context) - .setTimeline(timeline) .setActionSchedule(actionSchedule) .build() .start() @@ -2537,9 +2512,8 @@ public void run(SimpleExoPlayer player) { @Test public void sendMessages_withMediaRemoval_triggersCorrectMessagesAndDoesNotThrow() throws Exception { - ExoPlayer player = new TestExoPlayer.Builder(context).build(); - MediaSource mediaSource = new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)); - player.addMediaSources(Arrays.asList(mediaSource, mediaSource)); + ExoPlayer player = new TestExoPlayerBuilder(context).build(); + player.addMediaSources(ImmutableList.of(new FakeMediaSource(), new FakeMediaSource())); player .createMessage((messageType, payload) -> {}) .setPosition(/* windowIndex= */ 0, /* positionMs= */ 0) @@ -2708,6 +2682,64 @@ public void timelineUpdateWithNewMidrollAdCuePoint_dropsPrebufferedPeriod() thro assertThat(mediaSource.getCreatedMediaPeriods().get(3).adGroupIndex).isEqualTo(C.INDEX_UNSET); } + @Test + public void seekPastBufferingMidroll_playsAdAndThenContentFromSeekPosition() throws Exception { + long adGroupWindowTimeMs = 1_000; + long seekPositionMs = 95_000; + long contentDurationMs = 100_000; + AdPlaybackState adPlaybackState = + FakeTimeline.createAdPlaybackState( + /* adsPerAdGroup= */ 1, + /* adGroupTimesUs...= */ TimelineWindowDefinition + .DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US + + C.msToUs(adGroupWindowTimeMs)); + Timeline timeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* durationUs= */ C.msToUs(contentDurationMs), + adPlaybackState)); + AtomicBoolean hasCreatedAdMediaPeriod = new AtomicBoolean(); + FakeMediaSource mediaSource = + new FakeMediaSource(timeline) { + @Override + public MediaPeriod createPeriod( + MediaPeriodId id, Allocator allocator, long startPositionUs) { + if (id.adGroupIndex == 0) { + hasCreatedAdMediaPeriod.set(true); + } + return super.createPeriod(id, allocator, startPositionUs); + } + }; + SimpleExoPlayer player = new TestExoPlayerBuilder(context).build(); + player.setMediaSource(mediaSource); + // Throw on the playback thread if the player position reaches a value that is just less than + // seek position. This ensures that playback stops and the assertion on the player position + // below fails, even if a long time passes between detecting the discontinuity and asserting. + player + .createMessage( + (messageType, payload) -> { + throw new IllegalStateException(); + }) + .setPosition(seekPositionMs - 1) + .send(); + player.pause(); + player.prepare(); + + // Block until the midroll has started buffering, then seek after the midroll before playing. + runMainLooperUntil(hasCreatedAdMediaPeriod::get); + player.seekTo(seekPositionMs); + player.play(); + + // When the ad finishes, the player position should be at or after the requested seek position. + TestPlayerRunHelper.runUntilPositionDiscontinuity( + player, Player.DISCONTINUITY_REASON_AD_INSERTION); + assertThat(player.getCurrentPosition()).isAtLeast(seekPositionMs); + } + @Test public void repeatedSeeksToUnpreparedPeriodInSameWindowKeepsWindowSequenceNumber() throws Exception { @@ -2756,7 +2788,7 @@ public void repeatedSeeksToUnpreparedPeriodInSameWindowKeepsWindowSequenceNumber @Test public void invalidSeekFallsBackToSubsequentPeriodOfTheRemovedPeriod() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); CountDownLatch sourceReleasedCountDownLatch = new CountDownLatch(/* count= */ 1); MediaSource mediaSourceToRemove = new FakeMediaSource(timeline) { @@ -2987,9 +3019,7 @@ public void clippedLoopedPeriodsArePlayedFully() throws Exception { long expectedDurationUs = 700_000; MediaSource mediaSource = new ClippingMediaSource( - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - startPositionUs, - startPositionUs + expectedDurationUs); + new FakeMediaSource(), startPositionUs, startPositionUs + expectedDurationUs); Clock clock = new AutoAdvancingFakeClock(); AtomicReference playerReference = new AtomicReference<>(); AtomicLong positionAtDiscontinuityMs = new AtomicLong(C.TIME_UNSET); @@ -3052,7 +3082,7 @@ public void updateTrackSelectorThenSeekToUnpreparedPeriod_returnsEmptyTrackGroup new FakeTimeline( new TimelineWindowDefinition( /* isSeekable= */ true, /* isDynamic= */ false, /* durationUs= */ C.TIME_UNSET)); - Timeline timelineSetDuration = new FakeTimeline(/* windowCount= */ 1); + Timeline timelineSetDuration = new FakeTimeline(); MediaSource mediaSource = new ConcatenatingMediaSource( new FakeMediaSource(timelineUnsetDuration, ExoPlayerTestRunner.VIDEO_FORMAT), @@ -3126,7 +3156,6 @@ public void removingLoopingLastPeriodFromPlaylistDoesNotThrow() throws Exception @Test public void seekToUnpreparedWindowWithNonZeroOffsetInConcatenationStartsAtCorrectPosition() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); FakeMediaSource mediaSource = new FakeMediaSource(/* timeline= */ null); MediaSource clippedMediaSource = new ClippingMediaSource( @@ -3143,7 +3172,7 @@ public void seekToUnpreparedWindowWithNonZeroOffsetInConcatenationStartsAtCorrec .seek(/* positionMs= */ 10) .waitForPendingPlayerCommands() // Finish preparation. - .executeRunnable(() -> mediaSource.setNewSourceInfo(timeline)) + .executeRunnable(() -> mediaSource.setNewSourceInfo(new FakeTimeline())) .waitForTimelineChanged() .waitForPlaybackState(Player.STATE_READY) .executeRunnable( @@ -3172,7 +3201,7 @@ public void seekToUnpreparedWindowWithMultiplePeriodsInConcatenationStartsAtCorr Timeline timeline = new FakeTimeline( new TimelineWindowDefinition( - /* periodCount =*/ 2, + /* periodCount= */ 2, /* id= */ new Object(), /* isSeekable= */ true, /* isDynamic= */ false, @@ -3614,7 +3643,7 @@ public void run(SimpleExoPlayer player) { @SuppressWarnings("deprecation") @Test public void seekTo_windowIndexIsReset_deprecated() throws Exception { - FakeTimeline fakeTimeline = new FakeTimeline(/* windowCount= */ 1); + FakeTimeline fakeTimeline = new FakeTimeline(); FakeMediaSource mediaSource = new FakeMediaSource(fakeTimeline); LoopingMediaSource loopingMediaSource = new LoopingMediaSource(mediaSource, 2); final int[] windowIndex = {C.INDEX_UNSET}; @@ -3668,7 +3697,7 @@ public void run(SimpleExoPlayer player) { @Test public void seekTo_windowIndexIsReset() throws Exception { - FakeTimeline fakeTimeline = new FakeTimeline(/* windowCount= */ 1); + FakeTimeline fakeTimeline = new FakeTimeline(); FakeMediaSource mediaSource = new FakeMediaSource(fakeTimeline); LoopingMediaSource loopingMediaSource = new LoopingMediaSource(mediaSource, 2); final int[] windowIndex = {C.INDEX_UNSET}; @@ -3830,8 +3859,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 9200)); assertThat(windowIndex[0]).isEqualTo(0); @@ -3859,8 +3888,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 0)); assertThat(windowIndex[0]).isEqualTo(2); @@ -3893,8 +3922,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + new FakeMediaSource(), + new FakeMediaSource()); assertThat(windowIndex[0]).isEqualTo(1); assertThat(positionMs[0]).isEqualTo(1000); @@ -3929,7 +3958,7 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(1); @@ -3964,7 +3993,7 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(1); @@ -3997,8 +4026,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(1); @@ -4034,7 +4063,7 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(0); @@ -4067,8 +4096,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(0); @@ -4101,8 +4130,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 0)); @@ -4136,7 +4165,7 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(0); @@ -4171,8 +4200,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(0); @@ -4206,8 +4235,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(0); @@ -4240,8 +4269,8 @@ public void run(SimpleExoPlayer player) { positionMs, bufferedPositions, totalBufferedDuration, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), + new FakeMediaSource(), + new FakeMediaSource(), createPartiallyBufferedMediaSource(/* maxBufferedPositionMs= */ 4000)); assertThat(windowIndex[0]).isEqualTo(0); @@ -4317,7 +4346,7 @@ private static FakeMediaSource createPartiallyBufferedMediaSource(long maxBuffer AdPlaybackState.NONE)); return new FakeMediaSource(fakeTimeline, ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -4325,17 +4354,19 @@ protected FakeMediaPeriod createFakeMediaPeriod( DrmSessionManager drmSessionManager, DrmSessionEventListener.EventDispatcher drmEventDispatcher, @Nullable TransferListener transferListener) { - FakeMediaPeriod fakeMediaPeriod = - new FakeMediaPeriod( - trackGroupArray, - FakeMediaPeriod.TrackDataFactory.singleSampleWithTimeUs(/* sampleTimeUs= */ 0), - mediaSourceEventDispatcher, - drmSessionManager, - drmEventDispatcher, - /* deferOnPrepared= */ false); - fakeMediaPeriod.setBufferedPositionUs( - windowOffsetInFirstPeriodUs + C.msToUs(maxBufferedPositionMs)); - return fakeMediaPeriod; + return new FakeMediaPeriod( + trackGroupArray, + allocator, + /* trackDataFactory= */ (format, mediaPeriodId) -> + ImmutableList.of( + oneByteSample(windowOffsetInFirstPeriodUs, C.BUFFER_FLAG_KEY_FRAME), + oneByteSample( + windowOffsetInFirstPeriodUs + C.msToUs(maxBufferedPositionMs), + C.BUFFER_FLAG_KEY_FRAME)), + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + /* deferOnPrepared= */ false); } }; } @@ -4344,7 +4375,8 @@ protected FakeMediaPeriod createFakeMediaPeriod( public void addMediaSource_whilePlayingAd_correctMasking() throws Exception { long contentDurationMs = 10_000; long adDurationMs = 100_000; - AdPlaybackState adPlaybackState = new AdPlaybackState(/* adGroupTimesUs...= */ 0); + AdPlaybackState adPlaybackState = + new AdPlaybackState(/* adsId= */ new Object(), /* adGroupTimesUs...= */ 0); adPlaybackState = adPlaybackState.withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1); adPlaybackState = adPlaybackState.withAdUri(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, Uri.EMPTY); @@ -4368,19 +4400,15 @@ public void addMediaSource_whilePlayingAd_correctMasking() throws Exception { boolean[] isPlayingAd = new boolean[3]; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) - .waitForPlaybackState(Player.STATE_READY) - .waitForIsLoading(true) - .waitForIsLoading(false) + .pause() .waitForIsLoading(true) .waitForIsLoading(false) - .pause() + .waitForPlaybackState(Player.STATE_READY) .executeRunnable( new PlayerRunnable() { @Override public void run(SimpleExoPlayer player) { - player.addMediaSource( - /* index= */ 1, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + player.addMediaSource(/* index= */ 1, new FakeMediaSource()); windowIndex[0] = player.getCurrentWindowIndex(); isPlayingAd[0] = player.isPlayingAd(); positionMs[0] = player.getCurrentPosition(); @@ -4406,8 +4434,7 @@ public void run(SimpleExoPlayer player) { new PlayerRunnable() { @Override public void run(SimpleExoPlayer player) { - player.addMediaSource( - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + player.addMediaSource(new FakeMediaSource()); windowIndex[2] = player.getCurrentWindowIndex(); isPlayingAd[2] = player.isPlayingAd(); positionMs[2] = player.getCurrentPosition(); @@ -4419,8 +4446,7 @@ public void run(SimpleExoPlayer player) { .build(); new ExoPlayerTestRunner.Builder(context) - .setMediaSources( - adsMediaSource, new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))) + .setMediaSources(adsMediaSource, new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start() @@ -4431,26 +4457,27 @@ adsMediaSource, new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))) assertThat(isPlayingAd[0]).isTrue(); assertThat(positionMs[0]).isAtMost(adDurationMs); assertThat(bufferedPositionMs[0]).isEqualTo(adDurationMs); - assertThat(totalBufferedDurationMs[0]).isEqualTo(adDurationMs - positionMs[0]); + assertThat(totalBufferedDurationMs[0]).isAtLeast(adDurationMs - positionMs[0]); assertThat(windowIndex[1]).isEqualTo(0); assertThat(isPlayingAd[1]).isTrue(); assertThat(positionMs[1]).isAtMost(adDurationMs); assertThat(bufferedPositionMs[1]).isEqualTo(adDurationMs); - assertThat(totalBufferedDurationMs[1]).isEqualTo(adDurationMs - positionMs[1]); + assertThat(totalBufferedDurationMs[1]).isAtLeast(adDurationMs - positionMs[1]); assertThat(windowIndex[2]).isEqualTo(0); assertThat(isPlayingAd[2]).isFalse(); assertThat(positionMs[2]).isGreaterThan(8000); assertThat(bufferedPositionMs[2]).isEqualTo(contentDurationMs); - assertThat(totalBufferedDurationMs[2]).isEqualTo(contentDurationMs - positionMs[2]); + assertThat(totalBufferedDurationMs[2]).isAtLeast(contentDurationMs - positionMs[2]); } @Test public void seekTo_whilePlayingAd_correctMasking() throws Exception { long contentDurationMs = 10_000; long adDurationMs = 4_000; - AdPlaybackState adPlaybackState = new AdPlaybackState(/* adGroupTimesUs...= */ 0); + AdPlaybackState adPlaybackState = + new AdPlaybackState(/* adsId= */ new Object(), /* adGroupTimesUs...= */ 0); adPlaybackState = adPlaybackState.withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1); adPlaybackState = adPlaybackState.withAdUri(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, Uri.EMPTY); @@ -4526,6 +4553,42 @@ public void run(SimpleExoPlayer player) { assertThat(totalBufferedDurationMs[1]).isEqualTo(adDurationMs); } + // https://github.com/google/ExoPlayer/issues/8349 + @Test + public void seekTo_whilePlayingAd_doesntBlockFutureUpdates() throws Exception { + long contentDurationMs = 10_000; + long adDurationMs = 4_000; + AdPlaybackState adPlaybackState = + new AdPlaybackState(/* adsId= */ new Object(), /* adGroupTimesUs...= */ 0) + .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) + .withAdUri(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, Uri.EMPTY); + long[][] durationsUs = new long[1][]; + durationsUs[0] = new long[] {C.msToUs(adDurationMs)}; + adPlaybackState = adPlaybackState.withAdDurationsUs(durationsUs); + Timeline adTimeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* durationUs= */ C.msToUs(contentDurationMs), + adPlaybackState)); + FakeMediaSource adsMediaSource = new FakeMediaSource(adTimeline); + + SimpleExoPlayer player = new TestExoPlayerBuilder(context).build(); + player.setMediaSource(adsMediaSource); + player.pause(); + player.prepare(); + runUntilPlaybackState(player, Player.STATE_READY); + + player.seekTo(0, 8000); + player.play(); + + // This times out if playback info updates after the seek are blocked. + runUntilPlaybackState(player, Player.STATE_ENDED); + } + @Test public void becomingNoisyIgnoredIfBecomingNoisyHandlingIsDisabled() throws Exception { CountDownLatch becomingNoisyHandlingDisabled = new CountDownLatch(1); @@ -4601,7 +4664,10 @@ public boolean shouldContinueLoading( @Override public boolean shouldStartPlayback( - long bufferedDurationUs, float playbackSpeed, boolean rebuffering) { + long bufferedDurationUs, + float playbackSpeed, + boolean rebuffering, + long targetLiveOffsetUs) { return true; } }; @@ -4612,7 +4678,7 @@ public boolean shouldStartPlayback( /* chunkDurationUs= */ 500_000, /* bitratePercentStdDev= */ 10.0); MediaSource chunkedMediaSource = new FakeAdaptiveMediaSource( - new FakeTimeline(/* windowCount= */ 1), + new FakeTimeline(), new TrackGroupArray(new TrackGroup(ExoPlayerTestRunner.VIDEO_FORMAT)), new FakeChunkSource.Factory(dataSetFactory, new FakeDataSource.Factory())); @@ -4645,15 +4711,17 @@ public boolean shouldContinueLoading( @Override public boolean shouldStartPlayback( - long bufferedDurationUs, float playbackSpeed, boolean rebuffering) { + long bufferedDurationUs, + float playbackSpeed, + boolean rebuffering, + long targetLiveOffsetUs) { return true; } }; MediaSource mediaSourceWithLoadInProgress = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.VIDEO_FORMAT) { + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -4663,6 +4731,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( @Nullable TransferListener transferListener) { return new FakeMediaPeriod( trackGroupArray, + allocator, TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, mediaSourceEventDispatcher) { @Override @@ -4693,7 +4762,7 @@ public boolean isReady() { }; ExoPlayer player = - new TestExoPlayer.Builder(context) + new TestExoPlayerBuilder(context) .setRenderers(rendererWaitingForData) .setLoadControl(loadControlWithMaxBufferUs) .build(); @@ -4702,8 +4771,8 @@ public boolean isReady() { // Wait until the MediaSource is prepared, i.e. returned its timeline, and at least one // iteration of doSomeWork after this was run. - TestExoPlayer.runUntilTimelineChanged(player); - TestExoPlayer.runUntilPendingCommandsAreFullyHandled(player); + TestPlayerRunHelper.runUntilTimelineChanged(player); + TestPlayerRunHelper.runUntilPendingCommandsAreFullyHandled(player); assertThat(player.getPlayerError()).isNull(); } @@ -4720,7 +4789,10 @@ public boolean shouldContinueLoading( @Override public boolean shouldStartPlayback( - long bufferedDurationUs, float playbackSpeed, boolean rebuffering) { + long bufferedDurationUs, + float playbackSpeed, + boolean rebuffering, + long targetLiveOffsetUs) { return false; } }; @@ -4731,7 +4803,7 @@ public boolean shouldStartPlayback( /* chunkDurationUs= */ 500_000, /* bitratePercentStdDev= */ 10.0); MediaSource chunkedMediaSource = new FakeAdaptiveMediaSource( - new FakeTimeline(/* windowCount= */ 1), + new FakeTimeline(), new TrackGroupArray(new TrackGroup(ExoPlayerTestRunner.VIDEO_FORMAT)), new FakeChunkSource.Factory(dataSetFactory, new FakeDataSource.Factory())); @@ -4744,6 +4816,53 @@ public boolean shouldStartPlayback( .blockUntilEnded(TIMEOUT_MS); } + @Test + public void shortAdFollowedByUnpreparedAd_playbackDoesNotGetStuck() throws Exception { + AdPlaybackState adPlaybackState = + FakeTimeline.createAdPlaybackState(/* adsPerAdGroup= */ 2, /* adGroupTimesUs...= */ 0); + long shortAdDurationMs = 1_000; + adPlaybackState = + adPlaybackState.withAdDurationsUs(new long[][] {{shortAdDurationMs, shortAdDurationMs}}); + Timeline timeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* durationUs= */ C.msToUs(10000), + adPlaybackState)); + // Simulate the second ad not being prepared. + FakeMediaSource mediaSource = + new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT) { + @Override + protected MediaPeriod createMediaPeriod( + MediaPeriodId id, + TrackGroupArray trackGroupArray, + Allocator allocator, + MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + @Nullable TransferListener transferListener) { + return new FakeMediaPeriod( + trackGroupArray, + allocator, + FakeMediaPeriod.TrackDataFactory.singleSampleWithTimeUs(0), + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + /* deferOnPrepared= */ id.adIndexInAdGroup == 1); + } + }; + SimpleExoPlayer player = new TestExoPlayerBuilder(context).build(); + player.setMediaSource(mediaSource); + player.prepare(); + player.play(); + + // The player is not stuck in the buffering state. + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + } + @Test public void moveMediaItem() throws Exception { TimelineWindowDefinition firstWindowDefinition = @@ -4766,8 +4885,8 @@ public void moveMediaItem() throws Exception { MediaSource mediaSource2 = new FakeMediaSource(timeline2); Timeline expectedPlaceholderTimeline = new FakeTimeline( - TimelineWindowDefinition.createDummy(/* tag= */ 1), - TimelineWindowDefinition.createDummy(/* tag= */ 2)); + TimelineWindowDefinition.createPlaceholder(/* tag= */ 1), + TimelineWindowDefinition.createPlaceholder(/* tag= */ 2)); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .waitForTimelineChanged( @@ -4839,9 +4958,9 @@ public void removeMediaItem() throws Exception { Timeline expectedPlaceholderTimeline = new FakeTimeline( - TimelineWindowDefinition.createDummy(/* tag= */ 1), - TimelineWindowDefinition.createDummy(/* tag= */ 2), - TimelineWindowDefinition.createDummy(/* tag= */ 3)); + TimelineWindowDefinition.createPlaceholder(/* tag= */ 1), + TimelineWindowDefinition.createPlaceholder(/* tag= */ 2), + TimelineWindowDefinition.createPlaceholder(/* tag= */ 3)); Timeline expectedRealTimeline = new FakeTimeline(firstWindowDefinition, secondWindowDefinition, thirdWindowDefinition); Timeline expectedRealTimelineAfterRemove = @@ -4899,9 +5018,9 @@ public void removeMediaItems() throws Exception { Timeline expectedPlaceholderTimeline = new FakeTimeline( - TimelineWindowDefinition.createDummy(/* tag= */ 1), - TimelineWindowDefinition.createDummy(/* tag= */ 2), - TimelineWindowDefinition.createDummy(/* tag= */ 3)); + TimelineWindowDefinition.createPlaceholder(/* tag= */ 1), + TimelineWindowDefinition.createPlaceholder(/* tag= */ 2), + TimelineWindowDefinition.createPlaceholder(/* tag= */ 3)); Timeline expectedRealTimeline = new FakeTimeline(firstWindowDefinition, secondWindowDefinition, thirdWindowDefinition); Timeline expectedRealTimelineAfterRemove = new FakeTimeline(firstWindowDefinition); @@ -4915,7 +5034,7 @@ public void removeMediaItems() throws Exception { @Test public void clearMediaItems() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .waitForTimelineChanged(timeline, Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE) @@ -4943,7 +5062,7 @@ public void clearMediaItems() throws Exception { @Test public void multipleModificationWithRecursiveListenerInvocations() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaSource mediaSource = new FakeMediaSource(timeline); Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 2); MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); @@ -4979,10 +5098,6 @@ public void multipleModificationWithRecursiveListenerInvocations() throws Except @Test public void modifyPlaylistUnprepared_remainsInIdle_needsPrepareForBuffering() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); int[] playbackStates = new int[4]; int[] timelineWindowCounts = new int[4]; int[] maskingPlaybackState = {C.INDEX_UNSET}; @@ -4999,13 +5114,13 @@ public void modifyPlaylistUnprepared_remainsInIdle_needsPrepareForBuffering() th new PlayerRunnable() { @Override public void run(SimpleExoPlayer player) { - player.setMediaSource(firstMediaSource, /* startPositionMs= */ 1000); + player.setMediaSource(new FakeMediaSource(), /* startPositionMs= */ 1000); maskingPlaybackState[0] = player.getPlaybackState(); } }) .executeRunnable( new PlaybackStateCollector(/* index= */ 2, playbackStates, timelineWindowCounts)) - .addMediaSources(secondMediaSource) + .addMediaSources(new FakeMediaSource()) .executeRunnable( new PlaybackStateCollector(/* index= */ 3, playbackStates, timelineWindowCounts)) .seek(/* windowIndex= */ 1, /* positionMs= */ 2000) @@ -5017,7 +5132,7 @@ public void run(SimpleExoPlayer player) { .build(); ExoPlayerTestRunner exoPlayerTestRunner = new ExoPlayerTestRunner.Builder(context) - .setMediaSources(firstMediaSource) + .setMediaSources(new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start(/* doPrepare= */ false) @@ -5040,8 +5155,8 @@ public void run(SimpleExoPlayer player) { Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE /* source update after prepare */); Timeline expectedSecondPlaceholderTimeline = new FakeTimeline( - TimelineWindowDefinition.createDummy(/* tag= */ 0), - TimelineWindowDefinition.createDummy(/* tag= */ 0)); + TimelineWindowDefinition.createPlaceholder(/* tag= */ 0), + TimelineWindowDefinition.createPlaceholder(/* tag= */ 0)); Timeline expectedSecondRealTimeline = new FakeTimeline( new TimelineWindowDefinition( @@ -5067,7 +5182,7 @@ public void run(SimpleExoPlayer player) { @Test public void modifyPlaylistPrepared_remainsInEnded_needsSeekForBuffering() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); FakeMediaSource secondMediaSource = new FakeMediaSource(timeline); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -5126,7 +5241,7 @@ public void modifyPlaylistPrepared_remainsInEnded_needsSeekForBuffering() throws @Test public void stopWithNoReset_modifyingPlaylistRemainsInIdleState_needsPrepareForBuffering() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); FakeMediaSource secondMediaSource = new FakeMediaSource(timeline); int[] playbackStateHolder = new int[3]; int[] windowCountHolder = new int[3]; @@ -5209,7 +5324,7 @@ public void run(SimpleExoPlayer player) { @Test public void prepareWhenAlreadyPreparedIsANoop() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG).waitForPlaybackState(Player.STATE_READY).prepare().build(); ExoPlayerTestRunner exoPlayerTestRunner = @@ -5550,125 +5665,8 @@ public void run(SimpleExoPlayer player) { assertThat(positionAfterSetShuffleOrder.get()).isAtLeast(5000); } - @Test - public void setMediaSources_secondAdMediaSource_throws() throws Exception { - AdsMediaSource adsMediaSource = - new AdsMediaSource( - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - /* adTagDataSpec= */ new DataSpec(Uri.EMPTY), - new DefaultMediaSourceFactory(context), - new FakeAdsLoader(), - new FakeAdViewProvider()); - Exception[] exception = {null}; - ActionSchedule actionSchedule = - new ActionSchedule.Builder(TAG) - .executeRunnable( - new PlayerRunnable() { - @Override - public void run(SimpleExoPlayer player) { - try { - player.setMediaSource(adsMediaSource); - player.addMediaSource(adsMediaSource); - } catch (Exception e) { - exception[0] = e; - } - player.prepare(); - } - }) - .build(); - - new ExoPlayerTestRunner.Builder(context) - .setActionSchedule(actionSchedule) - .build() - .start(/* doPrepare= */ false) - .blockUntilActionScheduleFinished(TIMEOUT_MS) - .blockUntilEnded(TIMEOUT_MS); - - assertThat(exception[0]).isInstanceOf(IllegalStateException.class); - } - - @Test - public void setMediaSources_multipleMediaSourcesWithAd_throws() throws Exception { - MediaSource mediaSource = new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)); - AdsMediaSource adsMediaSource = - new AdsMediaSource( - mediaSource, - /* adTagDataSpec= */ new DataSpec(Uri.EMPTY), - new DefaultMediaSourceFactory(context), - new FakeAdsLoader(), - new FakeAdViewProvider()); - final Exception[] exception = {null}; - ActionSchedule actionSchedule = - new ActionSchedule.Builder(TAG) - .executeRunnable( - new PlayerRunnable() { - @Override - public void run(SimpleExoPlayer player) { - try { - List sources = new ArrayList<>(); - sources.add(mediaSource); - sources.add(adsMediaSource); - player.setMediaSources(sources); - } catch (Exception e) { - exception[0] = e; - } - player.prepare(); - } - }) - .build(); - - new ExoPlayerTestRunner.Builder(context) - .setActionSchedule(actionSchedule) - .build() - .start(/* doPrepare= */ false) - .blockUntilActionScheduleFinished(TIMEOUT_MS) - .blockUntilEnded(TIMEOUT_MS); - - assertThat(exception[0]).isInstanceOf(IllegalArgumentException.class); - } - - @Test - public void setMediaSources_addingMediaSourcesWithAdToNonEmptyPlaylist_throws() throws Exception { - MediaSource mediaSource = new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)); - AdsMediaSource adsMediaSource = - new AdsMediaSource( - mediaSource, - /* adTagDataSpec= */ new DataSpec(Uri.EMPTY), - new DefaultMediaSourceFactory(context), - new FakeAdsLoader(), - new FakeAdViewProvider()); - final Exception[] exception = {null}; - ActionSchedule actionSchedule = - new ActionSchedule.Builder(TAG) - .waitForPlaybackState(Player.STATE_READY) - .executeRunnable( - new PlayerRunnable() { - @Override - public void run(SimpleExoPlayer player) { - try { - player.addMediaSource(adsMediaSource); - } catch (Exception e) { - exception[0] = e; - } - } - }) - .build(); - - new ExoPlayerTestRunner.Builder(context) - .setMediaSources(mediaSource) - .setActionSchedule(actionSchedule) - .build() - .start() - .blockUntilActionScheduleFinished(TIMEOUT_MS) - .blockUntilEnded(TIMEOUT_MS); - - assertThat(exception[0]).isInstanceOf(IllegalArgumentException.class); - } - @Test public void setMediaSources_empty_whenEmpty_correctMaskingWindowIndex() throws Exception { - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -5677,9 +5675,8 @@ public void setMediaSources_empty_whenEmpty_correctMaskingWindowIndex() throws E @Override public void run(SimpleExoPlayer player) { currentWindowIndices[0] = player.getCurrentWindowIndex(); - List listOfTwo = new ArrayList<>(); - listOfTwo.add(secondMediaSource); - listOfTwo.add(secondMediaSource); + List listOfTwo = + ImmutableList.of(new FakeMediaSource(), new FakeMediaSource()); player.addMediaSources(/* index= */ 0, listOfTwo); currentWindowIndices[1] = player.getCurrentWindowIndex(); } @@ -5719,7 +5716,7 @@ public void run(SimpleExoPlayer player) { currentWindowIndices[0] = player.getCurrentWindowIndex(); currentPositions[0] = player.getCurrentPosition(); List listOfTwo = - Lists.newArrayList( + ImmutableList.of( MediaItem.fromUri(Uri.EMPTY), MediaItem.fromUri(Uri.EMPTY)); player.setMediaItems(listOfTwo, /* resetPosition= */ true); currentWindowIndices[1] = player.getCurrentWindowIndex(); @@ -5743,8 +5740,6 @@ public void run(SimpleExoPlayer player) { @Test public void setMediaSources_empty_whenEmpty_validInitialSeek_correctMaskingWindowIndex() throws Exception { - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -5756,9 +5751,8 @@ public void setMediaSources_empty_whenEmpty_validInitialSeek_correctMaskingWindo @Override public void run(SimpleExoPlayer player) { currentWindowIndices[0] = player.getCurrentWindowIndex(); - List listOfTwo = new ArrayList<>(); - listOfTwo.add(secondMediaSource); - listOfTwo.add(secondMediaSource); + List listOfTwo = + ImmutableList.of(new FakeMediaSource(), new FakeMediaSource()); player.addMediaSources(/* index= */ 0, listOfTwo); currentWindowIndices[1] = player.getCurrentWindowIndex(); } @@ -5787,8 +5781,6 @@ public void run(SimpleExoPlayer player) { @Test public void setMediaSources_empty_whenEmpty_invalidInitialSeek_correctMaskingWindowIndex() throws Exception { - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -5800,9 +5792,8 @@ public void setMediaSources_empty_whenEmpty_invalidInitialSeek_correctMaskingWin @Override public void run(SimpleExoPlayer player) { currentWindowIndices[0] = player.getCurrentWindowIndex(); - List listOfTwo = new ArrayList<>(); - listOfTwo.add(secondMediaSource); - listOfTwo.add(secondMediaSource); + List listOfTwo = + ImmutableList.of(new FakeMediaSource(), new FakeMediaSource()); player.addMediaSources(/* index= */ 0, listOfTwo); currentWindowIndices[1] = player.getCurrentWindowIndex(); } @@ -5830,10 +5821,6 @@ public void run(SimpleExoPlayer player) { @Test public void setMediaSources_whenEmpty_correctMaskingWindowIndex() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -5843,7 +5830,7 @@ public void setMediaSources_whenEmpty_correctMaskingWindowIndex() throws Excepti @Override public void run(SimpleExoPlayer player) { // Increase current window index. - player.addMediaSource(/* index= */ 0, secondMediaSource); + player.addMediaSource(/* index= */ 0, new FakeMediaSource()); currentWindowIndices[0] = player.getCurrentWindowIndex(); } }) @@ -5852,7 +5839,7 @@ public void run(SimpleExoPlayer player) { @Override public void run(SimpleExoPlayer player) { // Current window index is unchanged. - player.addMediaSource(/* index= */ 2, secondMediaSource); + player.addMediaSource(/* index= */ 2, new FakeMediaSource()); currentWindowIndices[1] = player.getCurrentWindowIndex(); } }) @@ -5860,8 +5847,7 @@ public void run(SimpleExoPlayer player) { new PlayerRunnable() { @Override public void run(SimpleExoPlayer player) { - MediaSource mediaSource = - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)); + MediaSource mediaSource = new FakeMediaSource(); ConcatenatingMediaSource concatenatingMediaSource = new ConcatenatingMediaSource(mediaSource, mediaSource, mediaSource); // Increase current window with multi window source. @@ -5882,7 +5868,7 @@ public void run(SimpleExoPlayer player) { }) .build(); new ExoPlayerTestRunner.Builder(context) - .setMediaSources(firstMediaSource) + .setMediaSources(new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start() @@ -5946,10 +5932,6 @@ public void run(SimpleExoPlayer player) { @Test public void setMediaSources_whenEmpty_invalidInitialSeek_correctMasking() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1, new Object()); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; final long[] currentPositions = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; final long[] bufferedPositions = {C.TIME_UNSET, C.TIME_UNSET, C.TIME_UNSET}; @@ -5966,7 +5948,7 @@ public void run(SimpleExoPlayer player) { currentPositions[0] = player.getCurrentPosition(); bufferedPositions[0] = player.getBufferedPosition(); // Increase current window index. - player.addMediaSource(/* index= */ 0, secondMediaSource); + player.addMediaSource(/* index= */ 0, new FakeMediaSource()); currentWindowIndices[1] = player.getCurrentWindowIndex(); currentPositions[1] = player.getCurrentPosition(); bufferedPositions[1] = player.getBufferedPosition(); @@ -5987,7 +5969,7 @@ public void run(SimpleExoPlayer player) { .build(); new ExoPlayerTestRunner.Builder(context) .initialSeek(/* windowIndex= */ 1, 2000) - .setMediaSources(firstMediaSource) + .setMediaSources(new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start(/* doPrepare= */ false) @@ -6000,10 +5982,6 @@ public void run(SimpleExoPlayer player) { @Test public void setMediaSources_correctMaskingWindowIndex() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1, new Object()); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -6014,7 +5992,7 @@ public void setMediaSources_correctMaskingWindowIndex() throws Exception { public void run(SimpleExoPlayer player) { currentWindowIndices[0] = player.getCurrentWindowIndex(); // Increase current window index. - player.addMediaSource(/* index= */ 0, secondMediaSource); + player.addMediaSource(/* index= */ 0, new FakeMediaSource()); currentWindowIndices[1] = player.getCurrentWindowIndex(); } }) @@ -6028,7 +6006,7 @@ public void run(SimpleExoPlayer player) { }) .build(); new ExoPlayerTestRunner.Builder(context) - .setMediaSources(firstMediaSource) + .setMediaSources(new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start() @@ -6039,10 +6017,6 @@ public void run(SimpleExoPlayer player) { @Test public void setMediaSources_whenIdle_correctMaskingPlaybackState() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1, 1L); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1, 2L); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] maskingPlaybackStates = new int[4]; Arrays.fill(maskingPlaybackStates, C.INDEX_UNSET); ActionSchedule actionSchedule = @@ -6061,7 +6035,7 @@ public void run(SimpleExoPlayer player) { @Override public void run(SimpleExoPlayer player) { // Set media item with an implicit seek to the current position. - player.setMediaSource(firstMediaSource); + player.setMediaSource(new FakeMediaSource()); maskingPlaybackStates[1] = player.getPlaybackState(); } }) @@ -6070,7 +6044,8 @@ public void run(SimpleExoPlayer player) { @Override public void run(SimpleExoPlayer player) { // Set media item with an explicit seek. - player.setMediaSource(secondMediaSource, /* startPositionMs= */ C.TIME_UNSET); + player.setMediaSource( + new FakeMediaSource(), /* startPositionMs= */ C.TIME_UNSET); maskingPlaybackStates[2] = player.getPlaybackState(); } }) @@ -6157,8 +6132,7 @@ public void setMediaSources_whenIdle_noSeek_correctMaskingPlaybackState() throws @Override public void run(SimpleExoPlayer player) { // Set media item with no seek. - player.setMediaSource( - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + player.setMediaSource(new FakeMediaSource()); maskingPlaybackStates[0] = player.getPlaybackState(); } }) @@ -6197,7 +6171,7 @@ public void run(SimpleExoPlayer player) { maskingPlaybackStates[0] = player.getPlaybackState(); } }) - .setMediaSources(new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))) + .setMediaSources(new FakeMediaSource()) .prepare() .waitForPlaybackState(Player.STATE_READY) .build(); @@ -6355,9 +6329,7 @@ public void setMediaSources_whenEnded_noSeek_correctMaskingPlaybackState() throw @Override public void run(SimpleExoPlayer player) { // Set media item with no seek (keep current position). - player.setMediaSource( - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)), - /* resetPosition= */ false); + player.setMediaSource(new FakeMediaSource(), /* resetPosition= */ false); maskingPlaybackStates[0] = player.getPlaybackState(); } }) @@ -6598,9 +6570,7 @@ public void addMediaSources_whenEmptyInitialSeek_correctPeriodMasking() throws E new PlayerRunnable() { @Override public void run(SimpleExoPlayer player) { - player.addMediaSource( - /* index= */ 0, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + player.addMediaSource(/* index= */ 0, new FakeMediaSource()); positions[0] = player.getCurrentPosition(); positions[1] = player.getBufferedPosition(); } @@ -6646,9 +6616,7 @@ public void run(SimpleExoPlayer player) { /* index= */ 0, new FakeMediaSource(new FakeTimeline(/* windowCount= */ 2))); currentWindowIndices[2] = player.getCurrentWindowIndex(); - player.addMediaSource( - /* index= */ 0, - new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + player.addMediaSource(/* index= */ 0, new FakeMediaSource()); currentWindowIndices[3] = player.getCurrentWindowIndex(); // With a non-empty timeline, we mask the periodId in the playback info. currentPositions[1] = player.getCurrentPosition(); @@ -6689,8 +6657,6 @@ public void run(SimpleExoPlayer player) { public void testAddMediaSources_skipSettingMediaItems_invalidInitialSeek_correctMaskingWindowIndex() throws Exception { - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -6702,7 +6668,7 @@ public void run(SimpleExoPlayer player) { @Override public void run(SimpleExoPlayer player) { currentWindowIndices[0] = player.getCurrentWindowIndex(); - player.addMediaSource(secondMediaSource); + player.addMediaSource(new FakeMediaSource()); currentWindowIndices[1] = player.getCurrentWindowIndex(); } }) @@ -6729,7 +6695,7 @@ public void run(SimpleExoPlayer player) { @Test public void moveMediaItems_correctMaskingWindowIndex() throws Exception { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaSource firstMediaSource = new FakeMediaSource(timeline); MediaSource secondMediaSource = new FakeMediaSource(timeline); MediaSource thirdMediaSource = new FakeMediaSource(timeline); @@ -6810,10 +6776,6 @@ public void run(SimpleExoPlayer player) { @Test public void moveMediaItems_unprepared_correctMaskingWindowIndex() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -6839,7 +6801,7 @@ public void run(SimpleExoPlayer player) { }) .build(); new ExoPlayerTestRunner.Builder(context) - .setMediaSources(firstMediaSource, secondMediaSource) + .setMediaSources(new FakeMediaSource(), new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start(/* doPrepare= */ false) @@ -6850,10 +6812,6 @@ public void run(SimpleExoPlayer player) { @Test public void removeMediaItems_correctMaskingWindowIndex() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) @@ -6871,7 +6829,7 @@ public void run(SimpleExoPlayer player) { .build(); new ExoPlayerTestRunner.Builder(context) .initialSeek(/* windowIndex= */ 1, /* positionMs= */ C.TIME_UNSET) - .setMediaSources(firstMediaSource, secondMediaSource) + .setMediaSources(new FakeMediaSource(), new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start() @@ -6882,10 +6840,6 @@ public void run(SimpleExoPlayer player) { @Test public void removeMediaItems_currentItemRemoved_correctMasking() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET}; final long[] currentPositions = {C.TIME_UNSET, C.TIME_UNSET}; final long[] bufferedPositions = {C.TIME_UNSET, C.TIME_UNSET}; @@ -6909,7 +6863,7 @@ public void run(SimpleExoPlayer player) { .build(); new ExoPlayerTestRunner.Builder(context) .initialSeek(/* windowIndex= */ 1, /* positionMs= */ 5000) - .setMediaSources(firstMediaSource, secondMediaSource, firstMediaSource) + .setMediaSources(new FakeMediaSource(), new FakeMediaSource(), new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start() @@ -7056,10 +7010,6 @@ public void run(SimpleExoPlayer player) { @Test public void removeMediaItems_removeTailWithCurrentWindow_whenIdle_finishesPlayback() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1, new Object()); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .seek(/* windowIndex= */ 1, /* positionMs= */ C.TIME_UNSET) @@ -7070,7 +7020,7 @@ public void removeMediaItems_removeTailWithCurrentWindow_whenIdle_finishesPlayba .build(); ExoPlayerTestRunner exoPlayerTestRunner = new ExoPlayerTestRunner.Builder(context) - .setMediaSources(firstMediaSource, secondMediaSource) + .setMediaSources(new FakeMediaSource(), new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start(/* doPrepare= */ false) @@ -7082,10 +7032,6 @@ public void removeMediaItems_removeTailWithCurrentWindow_whenIdle_finishesPlayba @Test public void clearMediaItems_correctMasking() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET}; final int[] maskingPlaybackState = {C.INDEX_UNSET}; final long[] currentPosition = {C.TIME_UNSET, C.TIME_UNSET}; @@ -7112,7 +7058,7 @@ public void run(SimpleExoPlayer player) { .build(); new ExoPlayerTestRunner.Builder(context) .initialSeek(/* windowIndex= */ 1, /* positionMs= */ C.TIME_UNSET) - .setMediaSources(firstMediaSource, secondMediaSource) + .setMediaSources(new FakeMediaSource(), new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start() @@ -7129,10 +7075,6 @@ public void run(SimpleExoPlayer player) { @Test public void clearMediaItems_unprepared_correctMaskingWindowIndex_notEnded() throws Exception { - Timeline firstTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource firstMediaSource = new FakeMediaSource(firstTimeline); - Timeline secondTimeline = new FakeTimeline(/* windowCount= */ 1); - MediaSource secondMediaSource = new FakeMediaSource(secondTimeline); final int[] currentWindowIndices = {C.INDEX_UNSET, C.INDEX_UNSET}; final int[] currentStates = {C.INDEX_UNSET, C.INDEX_UNSET, C.INDEX_UNSET}; ActionSchedule actionSchedule = @@ -7163,7 +7105,7 @@ public void run(SimpleExoPlayer player) { .build(); new ExoPlayerTestRunner.Builder(context) .initialSeek(/* windowIndex= */ 1, /* positionMs= */ C.TIME_UNSET) - .setMediaSources(firstMediaSource, secondMediaSource) + .setMediaSources(new FakeMediaSource(), new FakeMediaSource()) .setActionSchedule(actionSchedule) .build() .start(/* doPrepare= */ false) @@ -7178,12 +7120,9 @@ public void run(SimpleExoPlayer player) { public void errorThrownDuringPlaylistUpdate_keepsConsistentPlayerState() { FakeMediaSource source1 = new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), - ExoPlayerTestRunner.VIDEO_FORMAT, - ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); FakeMediaSource source2 = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT); AtomicInteger audioRendererEnableCount = new AtomicInteger(0); FakeRenderer videoRenderer = new FakeRenderer(C.TRACK_TYPE_VIDEO); FakeRenderer audioRenderer = @@ -7225,8 +7164,6 @@ public void onPlayerError( // Wait until fully buffered so that the new renderer can be enabled immediately. .waitForIsLoading(true) .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) .removeMediaItem(0) .build(); ExoPlayerTestRunner testRunner = @@ -7379,10 +7316,9 @@ public void run(SimpleExoPlayer player) { // Return no end of stream signal to prevent playback from ending. FakeMediaPeriod.TrackDataFactory trackDataWithoutEos = (format, periodId) -> ImmutableList.of(); MediaSource continuouslyAllocatingMediaSource = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.VIDEO_FORMAT) { + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -7392,6 +7328,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( @Nullable TransferListener transferListener) { return new FakeMediaPeriod( trackGroupArray, + allocator, trackDataWithoutEos, mediaSourceEventDispatcher, drmSessionManager, @@ -7456,11 +7393,18 @@ public void load() throws IOException { @Override public void cancelLoad() {} }; + // Create 3 samples without end of stream signal to test that all 3 samples are + // still played before the sample stream exception is thrown. + FakeSampleStreamItem sample = + oneByteSample( + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, + C.BUFFER_FLAG_KEY_FRAME); + FakeMediaPeriod.TrackDataFactory threeSamplesWithoutEos = + (format, mediaPeriodId) -> ImmutableList.of(sample, sample, sample); MediaSource largeBufferAllocatingMediaSource = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.VIDEO_FORMAT) { + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -7470,7 +7414,8 @@ protected FakeMediaPeriod createFakeMediaPeriod( @Nullable TransferListener transferListener) { return new FakeMediaPeriod( trackGroupArray, - TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US, + allocator, + threeSamplesWithoutEos, mediaSourceEventDispatcher, drmSessionManager, drmEventDispatcher, @@ -7479,30 +7424,29 @@ protected FakeMediaPeriod createFakeMediaPeriod( @Override public boolean continueLoading(long positionUs) { - loader.startLoading( - loadable, new FakeLoaderCallback(), /* defaultMinRetryCount= */ 1); + super.continueLoading(positionUs); + if (!loader.isLoading()) { + loader.startLoading( + loadable, new FakeLoaderCallback(), /* defaultMinRetryCount= */ 1); + } return true; } @Override - protected SampleStream createSampleStream( - long positionUs, - TrackSelection selection, - MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + protected FakeSampleStream createSampleStream( + Allocator allocator, + @Nullable MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, DrmSessionManager drmSessionManager, - DrmSessionEventListener.EventDispatcher drmEventDispatcher) { - // Create 3 samples without end of stream signal to test that all 3 samples are - // still played before the exception is thrown. + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + Format initialFormat, + List fakeSampleStreamItems) { return new FakeSampleStream( + allocator, mediaSourceEventDispatcher, drmSessionManager, drmEventDispatcher, - selection.getSelectedFormat(), - ImmutableList.of( - oneByteSample(positionUs), - oneByteSample(positionUs), - oneByteSample(positionUs))) { - + initialFormat, + fakeSampleStreamItems) { @Override public void maybeThrowError() throws IOException { loader.maybeThrowError(); @@ -7664,32 +7608,34 @@ protected boolean shouldProcessBuffer(long bufferTimeUs, long playbackPositionUs /* defaultPositionUs= */ 4_567_890, /* windowOffsetInFirstPeriodUs= */ 1_234_567, AdPlaybackState.NONE)); - ExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(renderer).build(); + ExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(renderer).build(); long firstSampleTimeUs = 4_567_890 + 1_234_567; FakeMediaSource firstMediaSource = new FakeMediaSource( /* timeline= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, (unusedFormat, unusedMediaPeriodId) -> - ImmutableList.of(oneByteSample(firstSampleTimeUs), END_OF_STREAM_ITEM), + ImmutableList.of( + oneByteSample(firstSampleTimeUs, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM), ExoPlayerTestRunner.VIDEO_FORMAT); FakeMediaSource secondMediaSource = new FakeMediaSource( timelineWithOffsets, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, (unusedFormat, unusedMediaPeriodId) -> - ImmutableList.of(oneByteSample(firstSampleTimeUs), END_OF_STREAM_ITEM), + ImmutableList.of( + oneByteSample(firstSampleTimeUs, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM), ExoPlayerTestRunner.VIDEO_FORMAT); player.setMediaSources(ImmutableList.of(firstMediaSource, secondMediaSource)); // Start playback and wait until player is idly waiting for an update of the first source. player.prepare(); player.play(); - TestExoPlayer.runUntilPendingCommandsAreFullyHandled(player); + TestPlayerRunHelper.runUntilPendingCommandsAreFullyHandled(player); // Update media with a non-zero default start position and window offset. firstMediaSource.setNewSourceInfo(timelineWithOffsets); // Wait until player transitions to second source (which also has non-zero offsets). - TestExoPlayer.runUntilPositionDiscontinuity( + TestPlayerRunHelper.runUntilPositionDiscontinuity( player, Player.DISCONTINUITY_REASON_PERIOD_TRANSITION); assertThat(player.getCurrentWindowIndex()).isEqualTo(1); player.release(); @@ -7711,11 +7657,52 @@ protected boolean shouldProcessBuffer(long bufferTimeUs, long playbackPositionUs @Test public void mediaItemOfSources_correctInTimelineWindows() throws Exception { - SilenceMediaSource.Factory factory = - new SilenceMediaSource.Factory().setDurationUs(C.msToUs(100_000)); + TimelineWindowDefinition window1 = + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 1, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* isLive= */ false, + /* isPlaceholder= */ false, + /* durationUs = */ 100_000, + /* defaultPositionUs = */ 0, + /* windowOffsetInFirstPeriodUs= */ 0, + AdPlaybackState.NONE, + MediaItem.fromUri("http://foo.bar/fake1")); + FakeMediaSource fakeMediaSource1 = new FakeMediaSource(new FakeTimeline(window1)); + TimelineWindowDefinition window2 = + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 2, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* isLive= */ false, + /* isPlaceholder= */ false, + /* durationUs = */ 100_000, + /* defaultPositionUs = */ 0, + /* windowOffsetInFirstPeriodUs= */ 0, + AdPlaybackState.NONE, + MediaItem.fromUri("http://foo.bar/fake2")); + FakeMediaSource fakeMediaSource2 = new FakeMediaSource(new FakeTimeline(window2)); + TimelineWindowDefinition window3 = + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 3, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* isLive= */ false, + /* isPlaceholder= */ false, + /* durationUs = */ 100_000, + /* defaultPositionUs = */ 0, + /* windowOffsetInFirstPeriodUs= */ 0, + AdPlaybackState.NONE, + MediaItem.fromUri("http://foo.bar/fake3")); + FakeMediaSource fakeMediaSource3 = new FakeMediaSource(new FakeTimeline(window3)); final Player[] playerHolder = {null}; ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) + .pause() .executeRunnable( new PlayerRunnable() { @Override @@ -7725,10 +7712,10 @@ public void run(SimpleExoPlayer player) { }) .waitForPlaybackState(Player.STATE_READY) .seek(/* positionMs= */ 0) - .waitForPlaybackState(Player.STATE_ENDED) + .play() .build(); List currentMediaItems = new ArrayList<>(); - List initialMediaItems = new ArrayList<>(); + List mediaItemsInTimeline = new ArrayList<>(); Player.EventListener eventListener = new Player.EventListener() { @Override @@ -7738,31 +7725,34 @@ public void onTimelineChanged(Timeline timeline, int reason) { } Window window = new Window(); for (int i = 0; i < timeline.getWindowCount(); i++) { - initialMediaItems.add(timeline.getWindow(i, window).mediaItem); + mediaItemsInTimeline.add(timeline.getWindow(i, window).mediaItem); } } @Override public void onPositionDiscontinuity(int reason) { - currentMediaItems.add(playerHolder[0].getCurrentMediaItem()); + if (reason == Player.DISCONTINUITY_REASON_SEEK + || reason == Player.DISCONTINUITY_REASON_PERIOD_TRANSITION) { + currentMediaItems.add(playerHolder[0].getCurrentMediaItem()); + } } }; new ExoPlayerTestRunner.Builder(context) .setEventListener(eventListener) .setActionSchedule(actionSchedule) - .setMediaSources( - factory.setTag("1").createMediaSource(), - factory.setTag("2").createMediaSource(), - factory.setTag("3").createMediaSource()) + .setMediaSources(fakeMediaSource1, fakeMediaSource2, fakeMediaSource3) .build() .start() .blockUntilActionScheduleFinished(TIMEOUT_MS) .blockUntilEnded(TIMEOUT_MS); - assertThat(currentMediaItems.get(0).playbackProperties.tag).isEqualTo("1"); - assertThat(currentMediaItems.get(1).playbackProperties.tag).isEqualTo("2"); - assertThat(currentMediaItems.get(2).playbackProperties.tag).isEqualTo("3"); - assertThat(initialMediaItems).containsExactlyElementsIn(currentMediaItems); + assertThat(currentMediaItems.get(0).playbackProperties.uri.toString()) + .isEqualTo("http://foo.bar/fake1"); + assertThat(currentMediaItems.get(1).playbackProperties.uri.toString()) + .isEqualTo("http://foo.bar/fake2"); + assertThat(currentMediaItems.get(2).playbackProperties.uri.toString()) + .isEqualTo("http://foo.bar/fake3"); + assertThat(mediaItemsInTimeline).containsExactlyElementsIn(currentMediaItems); } @Test @@ -7841,6 +7831,7 @@ public void clearMediaItem_notifiesMediaItemTransition() throws Exception { SilenceMediaSource mediaSource2 = factory.setTag("2").createMediaSource(); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) + .pause() .waitForPlaybackState(Player.STATE_READY) .playUntilPosition(/* windowIndex= */ 1, /* positionMs= */ 2000) .clearMediaItems() @@ -7922,10 +7913,36 @@ public void seekTo_sameWindow_doesNotNotifyMediaItemTransition() throws Exceptio @Test public void repeat_notifiesMediaItemTransition() throws Exception { - SilenceMediaSource.Factory factory = - new SilenceMediaSource.Factory().setDurationUs(C.msToUs(100_000)); - SilenceMediaSource mediaSource1 = factory.setTag("1").createMediaSource(); - SilenceMediaSource mediaSource2 = factory.setTag("2").createMediaSource(); + MediaItem mediaItem1 = MediaItem.fromUri("http://foo.bar/fake1"); + TimelineWindowDefinition window1 = + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 1, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* isLive= */ false, + /* isPlaceholder= */ false, + /* durationUs = */ 100_000, + /* defaultPositionUs = */ 0, + /* windowOffsetInFirstPeriodUs= */ 0, + AdPlaybackState.NONE, + mediaItem1); + MediaItem mediaItem2 = MediaItem.fromUri("http://foo.bar/fake2"); + TimelineWindowDefinition window2 = + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 2, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* isLive= */ false, + /* isPlaceholder= */ false, + /* durationUs = */ 100_000, + /* defaultPositionUs = */ 0, + /* windowOffsetInFirstPeriodUs= */ 0, + AdPlaybackState.NONE, + mediaItem2); + FakeMediaSource mediaSource1 = new FakeMediaSource(new FakeTimeline(window1)); + FakeMediaSource mediaSource2 = new FakeMediaSource(new FakeTimeline(window2)); ActionSchedule actionSchedule = new ActionSchedule.Builder(TAG) .pause() @@ -7937,9 +7954,9 @@ public void run(SimpleExoPlayer player) { player.setRepeatMode(Player.REPEAT_MODE_ONE); } }) - .play() - .waitForPositionDiscontinuity() - .waitForPositionDiscontinuity() + .playUntilPosition(/* windowIndex= */ 0, /* positionMs= */ 90) + .playUntilPosition(/* windowIndex= */ 0, /* positionMs= */ 80) + .playUntilPosition(/* windowIndex= */ 0, /* positionMs= */ 70) .executeRunnable( new PlayerRunnable() { @Override @@ -7947,6 +7964,7 @@ public void run(SimpleExoPlayer player) { player.setRepeatMode(Player.REPEAT_MODE_OFF); } }) + .play() .build(); ExoPlayerTestRunner exoPlayerTestRunner = @@ -8092,8 +8110,8 @@ public void timelineRefresh_withModifiedMediaItem_doesNotNotifyMediaItemTransiti public void mediaSourceMaybeThrowSourceInfoRefreshError_isNotThrownUntilPlaybackReachedFailingItem() throws Exception { - ExoPlayer player = new TestExoPlayer.Builder(context).build(); - player.addMediaSource(new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + ExoPlayer player = new TestExoPlayerBuilder(context).build(); + player.addMediaSource(new FakeMediaSource()); player.addMediaSource( new FakeMediaSource(/* timeline= */ null) { @Override @@ -8104,7 +8122,7 @@ public void maybeThrowSourceInfoRefreshError() throws IOException { player.prepare(); player.play(); - ExoPlaybackException error = TestExoPlayer.runUntilError(player); + ExoPlaybackException error = TestPlayerRunHelper.runUntilError(player); Object period1Uid = player @@ -8118,13 +8136,13 @@ public void maybeThrowSourceInfoRefreshError() throws IOException { @Test public void mediaPeriodMaybeThrowPrepareError_isNotThrownUntilPlaybackReachedFailingItem() throws Exception { - ExoPlayer player = new TestExoPlayer.Builder(context).build(); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + ExoPlayer player = new TestExoPlayerBuilder(context).build(); + Timeline timeline = new FakeTimeline(); player.addMediaSource(new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT)); player.addMediaSource( new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -8134,9 +8152,10 @@ protected FakeMediaPeriod createFakeMediaPeriod( @Nullable TransferListener transferListener) { return new FakeMediaPeriod( trackGroupArray, + allocator, /* singleSampleTimeUs= */ 0, mediaSourceEventDispatcher, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, drmEventDispatcher, /* deferOnPrepared= */ true) { @Override @@ -8149,7 +8168,7 @@ public void maybeThrowPrepareError() throws IOException { player.prepare(); player.play(); - ExoPlaybackException error = TestExoPlayer.runUntilError(player); + ExoPlaybackException error = TestPlayerRunHelper.runUntilError(player); Object period1Uid = player @@ -8163,13 +8182,13 @@ public void maybeThrowPrepareError() throws IOException { @Test public void sampleStreamMaybeThrowError_isNotThrownUntilPlaybackReachedFailingItem() throws Exception { - ExoPlayer player = new TestExoPlayer.Builder(context).build(); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + ExoPlayer player = new TestExoPlayerBuilder(context).build(); + Timeline timeline = new FakeTimeline(); player.addMediaSource(new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT)); player.addMediaSource( new FakeMediaSource(timeline, ExoPlayerTestRunner.VIDEO_FORMAT) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -8178,20 +8197,28 @@ protected FakeMediaPeriod createFakeMediaPeriod( DrmSessionEventListener.EventDispatcher drmEventDispatcher, @Nullable TransferListener transferListener) { return new FakeMediaPeriod( - trackGroupArray, /* singleSampleTimeUs= */ 0, mediaSourceEventDispatcher) { + trackGroupArray, + allocator, + /* trackDataFactory= */ (format, mediaPeriodId) -> ImmutableList.of(), + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + /* deferOnPrepared= */ false) { @Override - protected SampleStream createSampleStream( - long positionUs, - TrackSelection selection, - MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + protected FakeSampleStream createSampleStream( + Allocator allocator, + @Nullable MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, DrmSessionManager drmSessionManager, - DrmSessionEventListener.EventDispatcher drmEventDispatcher) { + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + Format initialFormat, + List fakeSampleStreamItems) { return new FakeSampleStream( + allocator, mediaSourceEventDispatcher, - DrmSessionManager.DUMMY, + drmSessionManager, drmEventDispatcher, - selection.getSelectedFormat(), - /* fakeSampleStreamItems= */ ImmutableList.of()) { + initialFormat, + fakeSampleStreamItems) { @Override public void maybeThrowError() throws IOException { throw new IOException(); @@ -8204,7 +8231,7 @@ public void maybeThrowError() throws IOException { player.prepare(); player.play(); - ExoPlaybackException error = TestExoPlayer.runUntilError(player); + ExoPlaybackException error = TestPlayerRunHelper.runUntilError(player); Object period1Uid = player @@ -8218,11 +8245,9 @@ public void maybeThrowError() throws IOException { @Test public void rendererError_isReportedWithReadingMediaPeriodId() throws Exception { FakeMediaSource source0 = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.VIDEO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT); FakeMediaSource source1 = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT); RenderersFactory renderersFactory = (eventHandler, videoListener, audioListener, textOutput, metadataOutput) -> new Renderer[] { @@ -8239,12 +8264,12 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) } }; ExoPlayer player = - new TestExoPlayer.Builder(context).setRenderersFactory(renderersFactory).build(); + new TestExoPlayerBuilder(context).setRenderersFactory(renderersFactory).build(); player.setMediaSources(ImmutableList.of(source0, source1)); player.prepare(); player.play(); - ExoPlaybackException error = TestExoPlayer.runUntilError(player); + ExoPlaybackException error = TestPlayerRunHelper.runUntilError(player); Object period1Uid = player @@ -8258,7 +8283,7 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) @Test public void enableOffloadSchedulingWhileIdle_isToggled_isReported() throws Exception { - SimpleExoPlayer player = new TestExoPlayer.Builder(context).build(); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).build(); player.experimentalSetOffloadSchedulingEnabled(true); assertThat(runUntilReceiveOffloadSchedulingEnabledNewState(player)).isTrue(); @@ -8270,8 +8295,8 @@ public void enableOffloadSchedulingWhileIdle_isToggled_isReported() throws Excep @Test public void enableOffloadSchedulingWhilePlaying_isToggled_isReported() throws Exception { FakeSleepRenderer sleepRenderer = new FakeSleepRenderer(C.TRACK_TYPE_AUDIO).sleepOnNextRender(); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(sleepRenderer).build(); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(sleepRenderer).build(); + Timeline timeline = new FakeTimeline(); player.setMediaSource(new FakeMediaSource(timeline, ExoPlayerTestRunner.AUDIO_FORMAT)); player.prepare(); player.play(); @@ -8283,29 +8308,30 @@ public void enableOffloadSchedulingWhilePlaying_isToggled_isReported() throws Ex assertThat(runUntilReceiveOffloadSchedulingEnabledNewState(player)).isFalse(); } + @Ignore // See [internal: b/170387438] @Test public void enableOffloadSchedulingWhileSleepingForOffload_isDisabled_isReported() throws Exception { FakeSleepRenderer sleepRenderer = new FakeSleepRenderer(C.TRACK_TYPE_AUDIO).sleepOnNextRender(); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(sleepRenderer).build(); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(sleepRenderer).build(); + Timeline timeline = new FakeTimeline(); player.setMediaSource(new FakeMediaSource(timeline, ExoPlayerTestRunner.AUDIO_FORMAT)); player.experimentalSetOffloadSchedulingEnabled(true); - runUntilReceiveOffloadSchedulingEnabledNewState(player); player.prepare(); player.play(); - runMainLooperUntil(sleepRenderer::isSleeping); + runUntilSleepingForOffload(player, /* expectedSleepForOffload= */ true); player.experimentalSetOffloadSchedulingEnabled(false); assertThat(runUntilReceiveOffloadSchedulingEnabledNewState(player)).isFalse(); } + @Ignore // See [internal: b/170387438] @Test public void enableOffloadScheduling_isEnable_playerSleeps() throws Exception { FakeSleepRenderer sleepRenderer = new FakeSleepRenderer(C.TRACK_TYPE_AUDIO); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(sleepRenderer).build(); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(sleepRenderer).build(); + Timeline timeline = new FakeTimeline(); player.setMediaSource(new FakeMediaSource(timeline, ExoPlayerTestRunner.AUDIO_FORMAT)); player.experimentalSetOffloadSchedulingEnabled(true); player.prepare(); @@ -8313,50 +8339,627 @@ public void enableOffloadScheduling_isEnable_playerSleeps() throws Exception { sleepRenderer.sleepOnNextRender(); - runMainLooperUntil(sleepRenderer::isSleeping); - // TODO(b/163303129): There is currently no way to check that the player is sleeping for - // offload, for now use a timeout to check that the renderer is never woken up. - final int renderTimeoutMs = 500; - assertThrows( - TimeoutException.class, - () -> - runMainLooperUntil(() -> !sleepRenderer.isSleeping(), renderTimeoutMs, Clock.DEFAULT)); + runUntilSleepingForOffload(player, /* expectedSleepForOffload= */ true); + assertThat(player.experimentalIsSleepingForOffload()).isTrue(); } + @Ignore // See [internal: b/170387438] @Test public void experimentalEnableOffloadSchedulingWhileSleepingForOffload_isDisabled_renderingResumes() throws Exception { FakeSleepRenderer sleepRenderer = new FakeSleepRenderer(C.TRACK_TYPE_AUDIO).sleepOnNextRender(); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(sleepRenderer).build(); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(sleepRenderer).build(); + Timeline timeline = new FakeTimeline(); player.setMediaSource(new FakeMediaSource(timeline, ExoPlayerTestRunner.AUDIO_FORMAT)); player.experimentalSetOffloadSchedulingEnabled(true); player.prepare(); player.play(); - runMainLooperUntil(sleepRenderer::isSleeping); + runUntilSleepingForOffload(player, /* expectedSleepForOffload= */ true); player.experimentalSetOffloadSchedulingEnabled(false); // Force the player to exit offload sleep - runMainLooperUntil(() -> !sleepRenderer.isSleeping()); + runUntilSleepingForOffload(player, /* expectedSleepForOffload= */ false); + assertThat(player.experimentalIsSleepingForOffload()).isFalse(); runUntilPlaybackState(player, Player.STATE_ENDED); } + @Ignore // See [internal: b/170387438] @Test public void wakeupListenerWhileSleepingForOffload_isWokenUp_renderingResumes() throws Exception { FakeSleepRenderer sleepRenderer = new FakeSleepRenderer(C.TRACK_TYPE_AUDIO).sleepOnNextRender(); - SimpleExoPlayer player = new TestExoPlayer.Builder(context).setRenderers(sleepRenderer).build(); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).setRenderers(sleepRenderer).build(); + Timeline timeline = new FakeTimeline(); player.setMediaSource(new FakeMediaSource(timeline, ExoPlayerTestRunner.AUDIO_FORMAT)); player.experimentalSetOffloadSchedulingEnabled(true); player.prepare(); player.play(); - runMainLooperUntil(sleepRenderer::isSleeping); + runUntilSleepingForOffload(player, /* expectedSleepForOffload= */ true); sleepRenderer.wakeup(); - runMainLooperUntil(() -> !sleepRenderer.isSleeping()); + runUntilSleepingForOffload(player, /* expectedSleepForOffload= */ false); + assertThat(player.experimentalIsSleepingForOffload()).isFalse(); + runUntilPlaybackState(player, Player.STATE_ENDED); + } + + @Test + public void staticMetadata_callbackIsCalledCorrectlyAndMatchesGetter() throws Exception { + Format videoFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setMetadata( + new Metadata( + new TextInformationFrame( + /* id= */ "TT2", + /* description= */ "Video", + /* value= */ "Video track name"))) + .build(); + Format audioFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.AUDIO_AAC) + .setMetadata( + new Metadata( + new TextInformationFrame( + /* id= */ "TT2", + /* description= */ "Audio", + /* value= */ "Audio track name"))) + .build(); + EventListener eventListener = mock(EventListener.class); + Timeline fakeTimeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* isSeekable= */ true, /* isDynamic= */ false, /* durationUs= */ 100000)); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).build(); + + player.setMediaSource(new FakeMediaSource(fakeTimeline, videoFormat, audioFormat)); + player.addListener(eventListener); + player.prepare(); + player.play(); + runUntilPlaybackState(player, Player.STATE_ENDED); + List metadata = player.getCurrentStaticMetadata(); + player.release(); + + assertThat(metadata).containsExactly(videoFormat.metadata, audioFormat.metadata).inOrder(); + verify(eventListener) + .onStaticMetadataChanged(ImmutableList.of(videoFormat.metadata, audioFormat.metadata)); + } + + @Test + public void staticMetadata_callbackIsNotCalledWhenMetadataEmptyAndGetterReturnsEmptyList() + throws Exception { + Format videoFormat = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(); + Format audioFormat = new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build(); + EventListener eventListener = mock(EventListener.class); + Timeline fakeTimeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* isSeekable= */ true, /* isDynamic= */ false, /* durationUs= */ 100000)); + SimpleExoPlayer player = new TestExoPlayerBuilder(context).build(); + + player.setMediaSource(new FakeMediaSource(fakeTimeline, videoFormat, audioFormat)); + player.addListener(eventListener); + player.prepare(); + player.play(); runUntilPlaybackState(player, Player.STATE_ENDED); + List metadata = player.getCurrentStaticMetadata(); + player.release(); + + assertThat(metadata).isEmpty(); + verify(eventListener, never()).onStaticMetadataChanged(any()); + } + + @Test + public void targetLiveOffsetInMedia_adjustsLiveOffsetToTargetOffset() throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 20_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline timeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(9_000).build())); + Player.EventListener mockListener = mock(Player.EventListener.class); + player.addListener(mockListener); + player.pause(); + player.setMediaSource(new FakeMediaSource(timeline)); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + long liveOffsetAtStart = player.getCurrentLiveOffset(); + // Verify test setup (now = 20 seconds in live window, default start position = 8 seconds). + assertThat(liveOffsetAtStart).isIn(Range.closed(11_900L, 12_100L)); + + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 0, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Assert that player adjusted live offset to the media value. + assertThat(liveOffsetAtEnd).isIn(Range.closed(8_900L, 9_100L)); + // Assert that none of these playback speed changes were reported. + verify(mockListener, never()).onPlaybackParametersChanged(any()); + } + + @Test + public void targetLiveOffsetInMedia_withInitialSeek_adjustsLiveOffsetToInitialSeek() + throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 20_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline timeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(9_000).build())); + player.pause(); + + player.seekTo(18_000); + player.setMediaSource(new FakeMediaSource(timeline), /* resetPosition= */ false); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + long liveOffsetAtStart = player.getCurrentLiveOffset(); + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 0, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Target should have been permanently adjusted to 2 seconds. + // (initial now = 20 seconds in live window, initial seek to 18 seconds) + assertThat(liveOffsetAtStart).isIn(Range.closed(1_900L, 2_100L)); + assertThat(liveOffsetAtEnd).isIn(Range.closed(1_900L, 2_100L)); + } + + @Test + public void targetLiveOffsetInMedia_withUserSeek_adjustsLiveOffsetToSeek() throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 20_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline timeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(9_000).build())); + player.pause(); + player.setMediaSource(new FakeMediaSource(timeline)); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + long liveOffsetAtStart = player.getCurrentLiveOffset(); + // Verify test setup (now = 20 seconds in live window, default start position = 8 seconds). + assertThat(liveOffsetAtStart).isIn(Range.closed(11_900L, 12_100L)); + + // Seek to a live offset of 2 seconds. + player.seekTo(18_000); + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 0, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Assert the live offset adjustment was permanent. + assertThat(liveOffsetAtEnd).isIn(Range.closed(1_900L, 2_100L)); + } + + @Test + public void targetLiveOffsetInMedia_withTimelineUpdate_adjustsLiveOffsetToLatestTimeline() + throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 20_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline initialTimeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(9_000).build())); + Timeline updatedTimeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs + 50_000), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(4_000).build())); + FakeMediaSource fakeMediaSource = new FakeMediaSource(initialTimeline); + player.pause(); + player.setMediaSource(fakeMediaSource); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + long liveOffsetAtStart = player.getCurrentLiveOffset(); + // Verify test setup (now = 20 seconds in live window, default start position = 8 seconds). + assertThat(liveOffsetAtStart).isIn(Range.closed(11_900L, 12_100L)); + + // Play a bit and update configuration. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 0, /* positionMs= */ 55_000); + fakeMediaSource.setNewSourceInfo(updatedTimeline); + + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 0, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Assert that adjustment uses target offset from the updated timeline. + assertThat(liveOffsetAtEnd).isIn(Range.closed(3_900L, 4_100L)); + } + + @Test + public void targetLiveOffsetInMedia_withSetPlaybackParameters_usesPlaybackParameterSpeed() + throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 20_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline timeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 20 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(9_000).build())); + Player.EventListener mockListener = mock(Player.EventListener.class); + player.addListener(mockListener); + player.pause(); + player.setMediaSource(new FakeMediaSource(timeline)); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + long liveOffsetAtStart = player.getCurrentLiveOffset(); + // Verify test setup (now = 20 seconds in live window, default start position = 20 seconds). + assertThat(liveOffsetAtStart).isIn(Range.closed(-100L, 100L)); + + player.setPlaybackParameters(new PlaybackParameters(/* speed */ 2.0f)); + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 0, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Assert that the player didn't adjust the live offset to the media value (9 seconds) and + // instead played the media with double speed (resulting in a negative live offset). + assertThat(liveOffsetAtEnd).isLessThan(0); + // Assert that user-set speed was reported + verify(mockListener).onPlaybackParametersChanged(new PlaybackParameters(2.0f)); + } + + @Test + public void + targetLiveOffsetInMedia_afterAutomaticPeriodTransition_adjustsLiveOffsetToTargetOffset() + throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 10_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline nonLiveTimeline = new FakeTimeline(); + Timeline liveTimeline = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(9_000).build())); + player.pause(); + player.addMediaSource(new FakeMediaSource(nonLiveTimeline)); + player.addMediaSource(new FakeMediaSource(liveTimeline)); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 1, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Assert that player adjusted live offset to the media value. + assertThat(liveOffsetAtEnd).isIn(Range.closed(8_900L, 9_100L)); + } + + @Test + public void + targetLiveOffsetInMedia_afterSeekToDefaultPositionInOtherStream_adjustsLiveOffsetToMediaOffset() + throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 20_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline liveTimeline1 = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(9_000).build())); + Timeline liveTimeline2 = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(4_000).build())); + player.pause(); + player.addMediaSource(new FakeMediaSource(liveTimeline1)); + player.addMediaSource(new FakeMediaSource(liveTimeline2)); + // Ensure we override the target live offset to a seek position in the first live stream. + player.seekTo(10_000); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + + // Seek to default position in second stream. + player.next(); + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 1, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Assert that player adjusted live offset to the media value. + assertThat(liveOffsetAtEnd).isIn(Range.closed(3_900L, 4_100L)); + } + + @Test + public void + targetLiveOffsetInMedia_afterSeekToSpecificPositionInOtherStream_adjustsLiveOffsetToSeekPosition() + throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 20_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline liveTimeline1 = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(9_000).build())); + Timeline liveTimeline2 = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(4_000).build())); + player.pause(); + player.addMediaSource(new FakeMediaSource(liveTimeline1)); + player.addMediaSource(new FakeMediaSource(liveTimeline2)); + // Ensure we override the target live offset to a seek position in the first live stream. + player.seekTo(10_000); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + + // Seek to specific position in second stream (at 2 seconds live offset). + player.seekTo(/* windowIndex= */ 1, /* positionMs= */ 18_000); + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 1, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Assert that player adjusted live offset to the seek. + assertThat(liveOffsetAtEnd).isIn(Range.closed(1_900L, 2_100L)); + } + + @Test + public void noTargetLiveOffsetInMedia_doesNotAdjustLiveOffset() throws Exception { + long windowStartUnixTimeMs = 987_654_321_000L; + long nowUnixTimeMs = windowStartUnixTimeMs + 20_000; + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(new AutoAdvancingFakeClock(/* initialTimeMs= */ nowUnixTimeMs)) + .build(); + Timeline liveTimelineWithoutTargetLiveOffset = + new FakeTimeline( + new TimelineWindowDefinition( + /* periodCount= */ 1, + /* id= */ 0, + /* isSeekable= */ true, + /* isDynamic= */ true, + /* isLive= */ true, + /* isPlaceholder= */ false, + /* durationUs= */ 1000 * C.MICROS_PER_SECOND, + /* defaultPositionUs= */ 8 * C.MICROS_PER_SECOND, + /* windowOffsetInFirstPeriodUs= */ C.msToUs(windowStartUnixTimeMs), + AdPlaybackState.NONE, + new MediaItem.Builder().setUri(Uri.EMPTY).build())); + player.pause(); + player.setMediaSource(new FakeMediaSource(liveTimelineWithoutTargetLiveOffset)); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + long liveOffsetAtStart = player.getCurrentLiveOffset(); + // Verify test setup (now = 20 seconds in live window, default start position = 8 seconds). + assertThat(liveOffsetAtStart).isIn(Range.closed(11_900L, 12_100L)); + + // Play until close to the end of the available live window. + TestPlayerRunHelper.playUntilPosition(player, /* windowIndex= */ 0, /* positionMs= */ 999_000); + long liveOffsetAtEnd = player.getCurrentLiveOffset(); + player.release(); + + // Assert that live offset is still the same (i.e. unadjusted). + assertThat(liveOffsetAtEnd).isIn(Range.closed(11_900L, 12_100L)); + } + + @Test + public void onEvents_correspondToListenerCalls() throws Exception { + ExoPlayer player = new TestExoPlayerBuilder(context).build(); + EventListener listener = mock(EventListener.class); + player.addListener(listener); + Format formatWithStaticMetadata = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setMetadata(new Metadata(new BinaryFrame(/* id= */ "", /* data= */ new byte[0]))) + .build(); + + // Set multiple values together. + player.setMediaSource(new FakeMediaSource(new FakeTimeline(), formatWithStaticMetadata)); + player.seekTo(2_000); + player.setPlaybackParameters(new PlaybackParameters(/* speed= */ 2.0f)); + ShadowLooper.runMainLooperToNextTask(); + + verify(listener).onTimelineChanged(any(), anyInt()); + verify(listener).onMediaItemTransition(any(), anyInt()); + verify(listener).onPositionDiscontinuity(anyInt()); + verify(listener).onPlaybackParametersChanged(any()); + ArgumentCaptor eventCaptor = ArgumentCaptor.forClass(Player.Events.class); + verify(listener).onEvents(eq(player), eventCaptor.capture()); + Player.Events events = eventCaptor.getValue(); + assertThat(events.contains(Player.EVENT_TIMELINE_CHANGED)).isTrue(); + assertThat(events.contains(Player.EVENT_MEDIA_ITEM_TRANSITION)).isTrue(); + assertThat(events.contains(Player.EVENT_POSITION_DISCONTINUITY)).isTrue(); + assertThat(events.contains(Player.EVENT_PLAYBACK_PARAMETERS_CHANGED)).isTrue(); + + // Set values recursively. + player.addListener( + new EventListener() { + @Override + public void onRepeatModeChanged(int repeatMode) { + player.setShuffleModeEnabled(true); + } + }); + player.setRepeatMode(Player.REPEAT_MODE_ONE); + ShadowLooper.runMainLooperToNextTask(); + + verify(listener).onRepeatModeChanged(anyInt()); + verify(listener).onShuffleModeEnabledChanged(anyBoolean()); + verify(listener, times(2)).onEvents(eq(player), eventCaptor.capture()); + events = Iterables.getLast(eventCaptor.getAllValues()); + assertThat(events.contains(Player.EVENT_REPEAT_MODE_CHANGED)).isTrue(); + assertThat(events.contains(Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED)).isTrue(); + + // Ensure all other events are called (even though we can't control how exactly they are + // combined together in onEvents calls). + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + player.play(); + player.setMediaItem(MediaItem.fromUri("http://this-will-throw-an-exception.mp4")); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_IDLE); + ShadowLooper.runMainLooperToNextTask(); + player.release(); + + // Verify that all callbacks have been called at least once. + verify(listener, atLeastOnce()).onTimelineChanged(any(), anyInt()); + verify(listener, atLeastOnce()).onMediaItemTransition(any(), anyInt()); + verify(listener, atLeastOnce()).onPositionDiscontinuity(anyInt()); + verify(listener, atLeastOnce()).onPlaybackParametersChanged(any()); + verify(listener, atLeastOnce()).onRepeatModeChanged(anyInt()); + verify(listener, atLeastOnce()).onShuffleModeEnabledChanged(anyBoolean()); + verify(listener, atLeastOnce()).onPlaybackStateChanged(anyInt()); + verify(listener, atLeastOnce()).onIsLoadingChanged(anyBoolean()); + verify(listener, atLeastOnce()).onTracksChanged(any(), any()); + verify(listener, atLeastOnce()).onStaticMetadataChanged(any()); + verify(listener, atLeastOnce()).onPlayWhenReadyChanged(anyBoolean(), anyInt()); + verify(listener, atLeastOnce()).onIsPlayingChanged(anyBoolean()); + verify(listener, atLeastOnce()).onPlayerError(any()); + + // Verify all the same events have been recorded with onEvents. + verify(listener, atLeastOnce()).onEvents(eq(player), eventCaptor.capture()); + List allEvents = eventCaptor.getAllValues(); + assertThat(containsEvent(allEvents, Player.EVENT_TIMELINE_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_MEDIA_ITEM_TRANSITION)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_POSITION_DISCONTINUITY)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_PLAYBACK_PARAMETERS_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_REPEAT_MODE_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_PLAYBACK_STATE_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_IS_LOADING_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_TRACKS_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_STATIC_METADATA_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_PLAY_WHEN_READY_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_IS_PLAYING_CHANGED)).isTrue(); + assertThat(containsEvent(allEvents, Player.EVENT_PLAYER_ERROR)).isTrue(); } // Internal methods. @@ -8386,19 +8989,27 @@ private static void deliverBroadcast(Intent intent) { shadowOf(Looper.getMainLooper()).idle(); } + private static boolean containsEvent( + List eventsList, @Player.EventFlags int event) { + for (Player.Events events : eventsList) { + if (events.contains(event)) { + return true; + } + } + return false; + } + // Internal classes. - /* {@link FakeRenderer} that can sleep and be woken-up. */ + /** {@link FakeRenderer} that can sleep and be woken-up. */ private static class FakeSleepRenderer extends FakeRenderer { private static final long WAKEUP_DEADLINE_MS = 60 * C.MICROS_PER_SECOND; private final AtomicBoolean sleepOnNextRender; - private final AtomicBoolean isSleeping; private final AtomicReference wakeupListenerReceiver; public FakeSleepRenderer(int trackType) { super(trackType); sleepOnNextRender = new AtomicBoolean(false); - isSleeping = new AtomicBoolean(false); wakeupListenerReceiver = new AtomicReference<>(); } @@ -8414,14 +9025,6 @@ public FakeSleepRenderer sleepOnNextRender() { return this; } - /** - * Returns whether {@link Renderer.WakeupListener#onSleep(long)} was called on the last {@link - * #render(long, long)} - */ - public boolean isSleeping() { - return isSleeping.get(); - } - @Override public void handleMessage(int what, @Nullable Object object) throws ExoPlaybackException { if (what == MSG_SET_WAKEUP_LISTENER) { @@ -8436,11 +9039,6 @@ public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackEx super.render(positionUs, elapsedRealtimeUs); if (sleepOnNextRender.compareAndSet(/* expect= */ true, /* update= */ false)) { wakeupListenerReceiver.get().onSleep(WAKEUP_DEADLINE_MS); - // TODO(b/163303129): Use an actual message from the player instead of guessing that the - // player will always sleep for offload after calling `onSleep`. - isSleeping.set(true); - } else { - isSleeping.set(false); } } } @@ -8542,55 +9140,10 @@ public Loader.LoadErrorAction onLoadError( } } - private static class FakeAdsLoader implements AdsLoader { - - @Override - public void setPlayer(@Nullable Player player) {} - - @Override - public void release() {} - - @Override - public void setSupportedContentTypes(int... contentTypes) {} - - @Override - public void setAdTagDataSpec(DataSpec adTagDataSpec) {} - - @Override - public void start(AdsLoader.EventListener eventListener, AdViewProvider adViewProvider) {} - - @Override - public void stop() {} - - @Override - public void handlePrepareComplete(int adGroupIndex, int adIndexInAdGroup) {} - - @Override - public void handlePrepareError(int adGroupIndex, int adIndexInAdGroup, IOException exception) {} - } - - private static class FakeAdViewProvider implements AdsLoader.AdViewProvider { - - @Override - public ViewGroup getAdViewGroup() { - return null; - } - - @Override - public ImmutableList getAdOverlayInfos() { - return ImmutableList.of(); - } - } - /** * Returns an argument matcher for {@link Timeline} instances that ignores period and window uids. */ private static ArgumentMatcher noUid(Timeline timeline) { - return new ArgumentMatcher() { - @Override - public boolean matches(Timeline argument) { - return new NoUidTimeline(timeline).equals(new NoUidTimeline(argument)); - } - }; + return argument -> new NoUidTimeline(timeline).equals(new NoUidTimeline(argument)); } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/MediaPeriodQueueTest.java b/library/core/src/test/java/com/google/android/exoplayer2/MediaPeriodQueueTest.java index 0eaab52ff4c..e3067d8e25a 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/MediaPeriodQueueTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/MediaPeriodQueueTest.java @@ -31,7 +31,7 @@ import com.google.android.exoplayer2.testutil.FakeShuffleOrder; import com.google.android.exoplayer2.testutil.FakeTimeline; import com.google.android.exoplayer2.testutil.FakeTimeline.TimelineWindowDefinition; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelector; import com.google.android.exoplayer2.trackselection.TrackSelectorResult; import com.google.android.exoplayer2.upstream.Allocator; @@ -54,7 +54,7 @@ public final class MediaPeriodQueueTest { CONTENT_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); private static final Uri AD_URI = Uri.EMPTY; @@ -406,7 +406,8 @@ public void getNextMediaPeriodInfo_inMultiPeriodWindow_returnsCorrectMediaPeriod private void setupAdTimeline(long... adGroupTimesUs) { adPlaybackState = - new AdPlaybackState(adGroupTimesUs).withContentDurationUs(CONTENT_DURATION_US); + new AdPlaybackState(/* adsId= */ new Object(), adGroupTimesUs) + .withContentDurationUs(CONTENT_DURATION_US); SinglePeriodAdTimeline adTimeline = new SinglePeriodAdTimeline(CONTENT_TIMELINE, adPlaybackState); setupTimeline(adTimeline); @@ -434,6 +435,7 @@ private void setupTimeline(Timeline timeline) { /* isLoading= */ false, /* trackGroups= */ null, /* trackSelectorResult= */ null, + /* staticMetadata= */ ImmutableList.of(), /* loadingMediaPeriodId= */ null, /* playWhenReady= */ false, Player.PLAYBACK_SUPPRESSION_REASON_NONE, @@ -441,7 +443,8 @@ private void setupTimeline(Timeline timeline) { /* bufferedPositionUs= */ 0, /* totalBufferedDurationUs= */ 0, /* positionUs= */ 0, - /* offloadSchedulingEnabled= */ false); + /* offloadSchedulingEnabled= */ false, + /* sleepingForOffload= */ false); } private void advance() { @@ -467,7 +470,7 @@ private void enqueueNext() { mediaSourceList, getNextMediaPeriodInfo(), new TrackSelectorResult( - new RendererConfiguration[0], new TrackSelection[0], /* info= */ null)); + new RendererConfiguration[0], new ExoTrackSelection[0], /* info= */ null)); } private MediaPeriodInfo getNextMediaPeriodInfo() { @@ -498,7 +501,8 @@ private void setAdGroupFailedToLoad(int adGroupIndex) { private void updateAdPlaybackStateAndTimeline(long... adGroupTimesUs) { adPlaybackState = - new AdPlaybackState(adGroupTimesUs).withContentDurationUs(CONTENT_DURATION_US); + new AdPlaybackState(/* adsId= */ new Object(), adGroupTimesUs) + .withContentDurationUs(CONTENT_DURATION_US); updateTimeline(); } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/MediaSourceListTest.java b/library/core/src/test/java/com/google/android/exoplayer2/MediaSourceListTest.java index b3ff5e5c551..ea40519a3cb 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/MediaSourceListTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/MediaSourceListTest.java @@ -30,7 +30,6 @@ import com.google.android.exoplayer2.source.ShuffleOrder; import com.google.android.exoplayer2.testutil.FakeMediaSource; import com.google.android.exoplayer2.testutil.FakeShuffleOrder; -import com.google.android.exoplayer2.testutil.FakeTimeline; import com.google.android.exoplayer2.util.Util; import java.util.ArrayList; import java.util.Collections; @@ -528,11 +527,11 @@ private static void assertFirstWindowInChildIndices( } private static List createFakeHolders() { - MediaSource fakeMediaSource = new FakeMediaSource(new FakeTimeline(1)); List holders = new ArrayList<>(); for (int i = 0; i < MEDIA_SOURCE_LIST_SIZE; i++) { holders.add( - new MediaSourceList.MediaSourceHolder(fakeMediaSource, /* useLazyPreparation= */ true)); + new MediaSourceList.MediaSourceHolder( + new FakeMediaSource(), /* useLazyPreparation= */ true)); } return holders; } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/MetadataRetrieverTest.java b/library/core/src/test/java/com/google/android/exoplayer2/MetadataRetrieverTest.java index 09b546e89e6..53f6c24f10d 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/MetadataRetrieverTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/MetadataRetrieverTest.java @@ -17,35 +17,53 @@ package com.google.android.exoplayer2; import static com.google.android.exoplayer2.MetadataRetriever.retrieveMetadata; +import static com.google.android.exoplayer2.metadata.mp4.MdtaMetadataEntry.KEY_ANDROID_CAPTURE_FPS; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; import android.content.Context; import android.net.Uri; -import android.os.SystemClock; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.metadata.mp4.MdtaMetadataEntry; +import com.google.android.exoplayer2.metadata.mp4.MotionPhotoMetadata; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData; +import com.google.android.exoplayer2.metadata.mp4.SmtaMetadataEntry; import com.google.android.exoplayer2.source.TrackGroupArray; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.util.concurrent.ListenableFuture; +import java.util.ArrayList; +import java.util.List; import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeUnit; +import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; -import org.robolectric.annotation.LooperMode; /** Tests for {@link MetadataRetriever}. */ @RunWith(AndroidJUnit4.class) -@LooperMode(LooperMode.Mode.PAUSED) public class MetadataRetrieverTest { + private static final long TEST_TIMEOUT_SEC = 10; + + private Context context; + private AutoAdvancingFakeClock clock; + + @Before + public void setUp() throws Exception { + context = ApplicationProvider.getApplicationContext(); + clock = new AutoAdvancingFakeClock(); + } + @Test - public void retrieveMetadata_singleMediaItem() throws Exception { - Context context = ApplicationProvider.getApplicationContext(); + public void retrieveMetadata_singleMediaItem_outputsExpectedMetadata() throws Exception { MediaItem mediaItem = MediaItem.fromUri(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); - ListenableFuture trackGroupsFuture = retrieveMetadata(context, mediaItem); - TrackGroupArray trackGroups = waitAndGetTrackGroups(trackGroupsFuture); + ListenableFuture trackGroupsFuture = + retrieveMetadata(context, mediaItem, clock); + TrackGroupArray trackGroups = trackGroupsFuture.get(TEST_TIMEOUT_SEC, TimeUnit.SECONDS); assertThat(trackGroups.length).isEqualTo(2); // Video group. @@ -57,17 +75,18 @@ public void retrieveMetadata_singleMediaItem() throws Exception { } @Test - public void retrieveMetadata_multipleMediaItems() throws Exception { - Context context = ApplicationProvider.getApplicationContext(); + public void retrieveMetadata_multipleMediaItems_outputsExpectedMetadata() throws Exception { MediaItem mediaItem1 = MediaItem.fromUri(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); MediaItem mediaItem2 = MediaItem.fromUri(Uri.parse("asset://android_asset/media/mp3/bear-id3.mp3")); - ListenableFuture trackGroupsFuture1 = retrieveMetadata(context, mediaItem1); - ListenableFuture trackGroupsFuture2 = retrieveMetadata(context, mediaItem2); - TrackGroupArray trackGroups1 = waitAndGetTrackGroups(trackGroupsFuture1); - TrackGroupArray trackGroups2 = waitAndGetTrackGroups(trackGroupsFuture2); + ListenableFuture trackGroupsFuture1 = + retrieveMetadata(context, mediaItem1, clock); + ListenableFuture trackGroupsFuture2 = + retrieveMetadata(context, mediaItem2, clock); + TrackGroupArray trackGroups1 = trackGroupsFuture1.get(TEST_TIMEOUT_SEC, TimeUnit.SECONDS); + TrackGroupArray trackGroups2 = trackGroupsFuture2.get(TEST_TIMEOUT_SEC, TimeUnit.SECONDS); // First track group. assertThat(trackGroups1.length).isEqualTo(2); @@ -86,24 +105,88 @@ public void retrieveMetadata_multipleMediaItems() throws Exception { } @Test - public void retrieveMetadata_throwsErrorIfCannotLoad() { - Context context = ApplicationProvider.getApplicationContext(); + public void retrieveMetadata_heicMotionPhoto_outputsExpectedMetadata() throws Exception { MediaItem mediaItem = - MediaItem.fromUri(Uri.parse("asset://android_asset/media/does_not_exist")); + MediaItem.fromUri(Uri.parse("asset://android_asset/media/mp4/sample_MP.heic")); + MotionPhotoMetadata expectedMotionPhotoMetadata = + new MotionPhotoMetadata( + /* photoStartPosition= */ 0, + /* photoSize= */ 28_853, + /* photoPresentationTimestampUs= */ C.TIME_UNSET, + /* videoStartPosition= */ 28_869, + /* videoSize= */ 28_803); + + ListenableFuture trackGroupsFuture = + retrieveMetadata(context, mediaItem, clock); + TrackGroupArray trackGroups = trackGroupsFuture.get(TEST_TIMEOUT_SEC, TimeUnit.SECONDS); + + assertThat(trackGroups.length).isEqualTo(1); + assertThat(trackGroups.get(0).length).isEqualTo(1); + assertThat(trackGroups.get(0).getFormat(0).metadata.length()).isEqualTo(1); + assertThat(trackGroups.get(0).getFormat(0).metadata.get(0)) + .isEqualTo(expectedMotionPhotoMetadata); + } + + @Test + public void retrieveMetadata_heicStillPhoto_outputsEmptyMetadata() throws Exception { + MediaItem mediaItem = + MediaItem.fromUri(Uri.parse("asset://android_asset/media/mp4/sample_still_photo.heic")); - ListenableFuture trackGroupsFuture = retrieveMetadata(context, mediaItem); + ListenableFuture trackGroupsFuture = + retrieveMetadata(context, mediaItem, clock); + TrackGroupArray trackGroups = trackGroupsFuture.get(TEST_TIMEOUT_SEC, TimeUnit.SECONDS); - assertThrows(ExecutionException.class, () -> waitAndGetTrackGroups(trackGroupsFuture)); + assertThat(trackGroups.length).isEqualTo(1); + assertThat(trackGroups.get(0).length).isEqualTo(1); + assertThat(trackGroups.get(0).getFormat(0).metadata).isNull(); + } + + @Test + public void retrieveMetadata_sefSlowMotion_outputsExpectedMetadata() throws Exception { + MediaItem mediaItem = + MediaItem.fromUri(Uri.parse("asset://android_asset/media/mp4/sample_sef_slow_motion.mp4")); + SmtaMetadataEntry expectedSmtaEntry = + new SmtaMetadataEntry(/* captureFrameRate= */ 240, /* svcTemporalLayerCount= */ 4); + List segments = new ArrayList<>(); + segments.add( + new SlowMotionData.Segment( + /* startTimeMs= */ 88, /* endTimeMs= */ 879, /* speedDivisor= */ 2)); + segments.add( + new SlowMotionData.Segment( + /* startTimeMs= */ 1255, /* endTimeMs= */ 1970, /* speedDivisor= */ 8)); + SlowMotionData expectedSlowMotionData = new SlowMotionData(segments); + MdtaMetadataEntry expectedMdtaEntry = + new MdtaMetadataEntry( + KEY_ANDROID_CAPTURE_FPS, + /* value= */ new byte[] {67, 112, 0, 0}, + /* localeIndicator= */ 0, + /* typeIndicator= */ 23); + + ListenableFuture trackGroupsFuture = + retrieveMetadata(context, mediaItem, clock); + TrackGroupArray trackGroups = trackGroupsFuture.get(TEST_TIMEOUT_SEC, TimeUnit.SECONDS); + + assertThat(trackGroups.length).isEqualTo(2); // Video and audio + // Audio + assertThat(trackGroups.get(0).getFormat(0).metadata.length()).isEqualTo(2); + assertThat(trackGroups.get(0).getFormat(0).metadata.get(0)).isEqualTo(expectedSmtaEntry); + assertThat(trackGroups.get(0).getFormat(0).metadata.get(1)).isEqualTo(expectedSlowMotionData); + // Video + assertThat(trackGroups.get(1).getFormat(0).metadata.length()).isEqualTo(3); + assertThat(trackGroups.get(1).getFormat(0).metadata.get(0)).isEqualTo(expectedMdtaEntry); + assertThat(trackGroups.get(1).getFormat(0).metadata.get(1)).isEqualTo(expectedSmtaEntry); + assertThat(trackGroups.get(1).getFormat(0).metadata.get(2)).isEqualTo(expectedSlowMotionData); } - private static TrackGroupArray waitAndGetTrackGroups( - ListenableFuture trackGroupsFuture) - throws InterruptedException, ExecutionException { - while (!trackGroupsFuture.isDone()) { - // Simulate advancing SystemClock so that delayed messages sent to handlers are received. - SystemClock.setCurrentTimeMillis(SystemClock.uptimeMillis() + 100); - Thread.sleep(/* millis= */ 100); - } - return trackGroupsFuture.get(); + @Test + public void retrieveMetadata_invalidMediaItem_throwsError() { + MediaItem mediaItem = + MediaItem.fromUri(Uri.parse("asset://android_asset/media/does_not_exist")); + + ListenableFuture trackGroupsFuture = + retrieveMetadata(context, mediaItem, clock); + + assertThrows( + ExecutionException.class, () -> trackGroupsFuture.get(TEST_TIMEOUT_SEC, TimeUnit.SECONDS)); } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/PlayerMessageTest.java b/library/core/src/test/java/com/google/android/exoplayer2/PlayerMessageTest.java index 490cc520fe7..41579f073c0 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/PlayerMessageTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/PlayerMessageTest.java @@ -17,12 +17,11 @@ import static com.google.common.truth.Truth.assertThat; import static java.util.concurrent.TimeUnit.SECONDS; -import static org.junit.Assert.fail; +import static org.junit.Assert.assertThrows; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import static org.mockito.MockitoAnnotations.initMocks; -import android.os.Handler; import android.os.HandlerThread; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.util.Clock; @@ -55,9 +54,14 @@ public void setUp() { PlayerMessage.Target target = (messageType, payload) -> {}; handlerThread = new HandlerThread("TestHandler"); handlerThread.start(); - Handler handler = new Handler(handlerThread.getLooper()); message = - new PlayerMessage(sender, target, Timeline.EMPTY, /* defaultWindowIndex= */ 0, handler); + new PlayerMessage( + sender, + target, + Timeline.EMPTY, + /* defaultWindowIndex= */ 0, + clock, + handlerThread.getLooper()); } @After @@ -66,31 +70,26 @@ public void tearDown() { } @Test - public void experimentalBlockUntilDelivered_timesOut() throws Exception { + public void blockUntilDelivered_timesOut() throws Exception { when(clock.elapsedRealtime()).thenReturn(0L).thenReturn(TIMEOUT_MS * 2); - try { - message.send().experimentalBlockUntilDelivered(TIMEOUT_MS, clock); - fail(); - } catch (TimeoutException expected) { - } + assertThrows(TimeoutException.class, () -> message.send().blockUntilDelivered(TIMEOUT_MS)); - // Ensure experimentalBlockUntilDelivered() entered the blocking loop + // Ensure blockUntilDelivered() entered the blocking loop. verify(clock, Mockito.times(2)).elapsedRealtime(); } @Test - public void experimentalBlockUntilDelivered_onAlreadyProcessed_succeeds() throws Exception { + public void blockUntilDelivered_onAlreadyProcessed_succeeds() throws Exception { when(clock.elapsedRealtime()).thenReturn(0L); message.send().markAsProcessed(/* isDelivered= */ true); - assertThat(message.experimentalBlockUntilDelivered(TIMEOUT_MS, clock)).isTrue(); + assertThat(message.blockUntilDelivered(TIMEOUT_MS)).isTrue(); } @Test - public void experimentalBlockUntilDelivered_markAsProcessedWhileBlocked_succeeds() - throws Exception { + public void blockUntilDelivered_markAsProcessedWhileBlocked_succeeds() throws Exception { message.send(); // Use a separate Thread to mark the message as processed. @@ -114,8 +113,8 @@ public void experimentalBlockUntilDelivered_markAsProcessedWhileBlocked_succeeds }); try { - assertThat(message.experimentalBlockUntilDelivered(TIMEOUT_MS, clock)).isTrue(); - // Ensure experimentalBlockUntilDelivered() entered the blocking loop. + assertThat(message.blockUntilDelivered(TIMEOUT_MS)).isTrue(); + // Ensure blockUntilDelivered() entered the blocking loop. verify(clock, Mockito.atLeast(2)).elapsedRealtime(); future.get(1, SECONDS); } finally { diff --git a/library/core/src/test/java/com/google/android/exoplayer2/SimpleExoPlayerTest.java b/library/core/src/test/java/com/google/android/exoplayer2/SimpleExoPlayerTest.java index e3a625a3ceb..a11961c301d 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/SimpleExoPlayerTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/SimpleExoPlayerTest.java @@ -15,14 +15,25 @@ */ package com.google.android.exoplayer2; +import static com.google.android.exoplayer2.robolectric.TestPlayerRunHelper.runUntilPlaybackState; import static com.google.common.truth.Truth.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.analytics.AnalyticsListener; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.ExoPlayerTestRunner; +import com.google.android.exoplayer2.testutil.FakeMediaSource; +import com.google.android.exoplayer2.testutil.FakeTimeline; +import com.google.android.exoplayer2.testutil.FakeVideoRenderer; import java.util.concurrent.atomic.AtomicReference; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.annotation.Config; +import org.robolectric.shadows.ShadowLooper; /** Unit test for {@link SimpleExoPlayer}. */ @RunWith(AndroidJUnit4.class) @@ -43,4 +54,29 @@ public void builder_inBackgroundThread_doesNotThrow() throws Exception { assertThat(builderThrow.get()).isNull(); } + + @Test + public void release_triggersAllPendingEventsInAnalyticsListeners() throws Exception { + SimpleExoPlayer player = + new SimpleExoPlayer.Builder( + ApplicationProvider.getApplicationContext(), + (handler, videoListener, audioListener, textOutput, metadataOutput) -> + new Renderer[] {new FakeVideoRenderer(handler, videoListener)}) + .setClock(new AutoAdvancingFakeClock()) + .build(); + AnalyticsListener listener = mock(AnalyticsListener.class); + player.addAnalyticsListener(listener); + // Do something that requires clean-up callbacks like decoder disabling. + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT)); + player.prepare(); + player.play(); + runUntilPlaybackState(player, Player.STATE_READY); + + player.release(); + ShadowLooper.runMainLooperToNextTask(); + + verify(listener).onVideoDisabled(any(), any()); + verify(listener).onPlayerReleased(any()); + } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/TimelineTest.java b/library/core/src/test/java/com/google/android/exoplayer2/TimelineTest.java index 65b01193544..fe3edf41772 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/TimelineTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/TimelineTest.java @@ -19,6 +19,7 @@ import androidx.annotation.Nullable; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.MediaItem.LiveConfiguration; import com.google.android.exoplayer2.testutil.FakeTimeline; import com.google.android.exoplayer2.testutil.FakeTimeline.TimelineWindowDefinition; import com.google.android.exoplayer2.testutil.TimelineAsserts; @@ -93,7 +94,7 @@ public void windowEquals() { assertThat(window).isNotEqualTo(otherWindow); otherWindow = new Timeline.Window(); - otherWindow.isLive = true; + otherWindow.liveConfiguration = LiveConfiguration.UNSET; assertThat(window).isNotEqualTo(otherWindow); otherWindow = new Timeline.Window(); @@ -131,7 +132,7 @@ public void windowEquals() { window.elapsedRealtimeEpochOffsetMs, window.isSeekable, window.isDynamic, - window.isLive, + window.liveConfiguration, window.defaultPositionUs, window.durationUs, window.firstPeriodIndex, diff --git a/library/core/src/test/java/com/google/android/exoplayer2/analytics/AnalyticsCollectorTest.java b/library/core/src/test/java/com/google/android/exoplayer2/analytics/AnalyticsCollectorTest.java index 1238831cbca..ad807c40799 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/analytics/AnalyticsCollectorTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/analytics/AnalyticsCollectorTest.java @@ -15,10 +15,53 @@ */ package com.google.android.exoplayer2.analytics; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_AUDIO_DECODER_INITIALIZED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_AUDIO_DISABLED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_AUDIO_ENABLED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_AUDIO_INPUT_FORMAT_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_AUDIO_POSITION_ADVANCING; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_DOWNSTREAM_FORMAT_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_DRM_KEYS_LOADED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_DRM_SESSION_ACQUIRED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_DRM_SESSION_MANAGER_ERROR; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_DRM_SESSION_RELEASED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_DROPPED_VIDEO_FRAMES; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_IS_LOADING_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_IS_PLAYING_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_LOAD_COMPLETED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_LOAD_ERROR; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_LOAD_STARTED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_MEDIA_ITEM_TRANSITION; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_PLAYBACK_PARAMETERS_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_PLAYBACK_STATE_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_PLAYER_ERROR; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_PLAY_WHEN_READY_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_POSITION_DISCONTINUITY; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_RENDERED_FIRST_FRAME; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_TIMELINE_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_TRACKS_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_VIDEO_DECODER_INITIALIZED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_VIDEO_DISABLED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_VIDEO_ENABLED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_VIDEO_FRAME_PROCESSING_OFFSET; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_VIDEO_INPUT_FORMAT_CHANGED; +import static com.google.android.exoplayer2.analytics.AnalyticsListener.EVENT_VIDEO_SIZE_CHANGED; import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.oneByteSample; import static com.google.common.truth.Truth.assertThat; - +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyBoolean; +import static org.mockito.ArgumentMatchers.anyFloat; +import static org.mockito.ArgumentMatchers.anyInt; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.atLeastOnce; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.spy; +import static org.mockito.Mockito.verify; + +import android.os.Looper; +import android.util.SparseArray; import android.view.Surface; import androidx.annotation.Nullable; import androidx.test.core.app.ApplicationProvider; @@ -26,6 +69,7 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.PlaybackParameters; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Renderer; @@ -41,6 +85,7 @@ import com.google.android.exoplayer2.drm.MediaDrmCallback; import com.google.android.exoplayer2.drm.MediaDrmCallbackException; import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; import com.google.android.exoplayer2.source.ConcatenatingMediaSource; import com.google.android.exoplayer2.source.LoadEventInfo; import com.google.android.exoplayer2.source.MediaLoadData; @@ -58,8 +103,11 @@ import com.google.android.exoplayer2.testutil.FakeTimeline; import com.google.android.exoplayer2.testutil.FakeTimeline.TimelineWindowDefinition; import com.google.android.exoplayer2.testutil.FakeVideoRenderer; +import com.google.android.exoplayer2.testutil.TestExoPlayerBuilder; import com.google.android.exoplayer2.testutil.TestUtil; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.util.Clock; +import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.Util; import com.google.common.collect.ImmutableList; import java.io.IOException; @@ -71,6 +119,10 @@ import java.util.concurrent.atomic.AtomicReference; import org.junit.Test; import org.junit.runner.RunWith; +import org.mockito.ArgumentCaptor; +import org.mockito.InOrder; +import org.mockito.Mockito; +import org.robolectric.shadows.ShadowLooper; /** Integration test for {@link AnalyticsCollector}. */ @RunWith(AndroidJUnit4.class) @@ -78,51 +130,14 @@ public final class AnalyticsCollectorTest { private static final String TAG = "AnalyticsCollectorTest"; - private static final int EVENT_PLAYER_STATE_CHANGED = 0; - private static final int EVENT_TIMELINE_CHANGED = 1; - private static final int EVENT_POSITION_DISCONTINUITY = 2; - private static final int EVENT_SEEK_STARTED = 3; - private static final int EVENT_SEEK_PROCESSED = 4; - private static final int EVENT_PLAYBACK_PARAMETERS_CHANGED = 5; - private static final int EVENT_REPEAT_MODE_CHANGED = 6; - private static final int EVENT_SHUFFLE_MODE_CHANGED = 7; - private static final int EVENT_LOADING_CHANGED = 8; - private static final int EVENT_PLAYER_ERROR = 9; - private static final int EVENT_TRACKS_CHANGED = 10; - private static final int EVENT_LOAD_STARTED = 11; - private static final int EVENT_LOAD_COMPLETED = 12; - private static final int EVENT_LOAD_CANCELED = 13; - private static final int EVENT_LOAD_ERROR = 14; - private static final int EVENT_DOWNSTREAM_FORMAT_CHANGED = 15; - private static final int EVENT_UPSTREAM_DISCARDED = 16; - private static final int EVENT_BANDWIDTH_ESTIMATE = 17; - private static final int EVENT_SURFACE_SIZE_CHANGED = 18; - private static final int EVENT_METADATA = 19; - private static final int EVENT_DECODER_ENABLED = 20; - private static final int EVENT_DECODER_INIT = 21; - private static final int EVENT_DECODER_FORMAT_CHANGED = 22; - private static final int EVENT_DECODER_DISABLED = 23; - private static final int EVENT_AUDIO_ENABLED = 24; - private static final int EVENT_AUDIO_DECODER_INIT = 25; - private static final int EVENT_AUDIO_INPUT_FORMAT_CHANGED = 26; - private static final int EVENT_AUDIO_DISABLED = 27; - private static final int EVENT_AUDIO_SESSION_ID = 28; - private static final int EVENT_AUDIO_POSITION_ADVANCING = 29; - private static final int EVENT_AUDIO_UNDERRUN = 30; - private static final int EVENT_VIDEO_ENABLED = 31; - private static final int EVENT_VIDEO_DECODER_INIT = 32; - private static final int EVENT_VIDEO_INPUT_FORMAT_CHANGED = 33; - private static final int EVENT_DROPPED_FRAMES = 34; - private static final int EVENT_VIDEO_DISABLED = 35; - private static final int EVENT_RENDERED_FIRST_FRAME = 36; - private static final int EVENT_VIDEO_FRAME_PROCESSING_OFFSET = 37; - private static final int EVENT_VIDEO_SIZE_CHANGED = 38; - private static final int EVENT_DRM_KEYS_LOADED = 39; - private static final int EVENT_DRM_ERROR = 40; - private static final int EVENT_DRM_KEYS_RESTORED = 41; - private static final int EVENT_DRM_KEYS_REMOVED = 42; - private static final int EVENT_DRM_SESSION_ACQUIRED = 43; - private static final int EVENT_DRM_SESSION_RELEASED = 44; + // Deprecated event constants. + private static final long EVENT_PLAYER_STATE_CHANGED = 1L << 63; + private static final long EVENT_SEEK_STARTED = 1L << 62; + private static final long EVENT_SEEK_PROCESSED = 1L << 61; + private static final long EVENT_DECODER_ENABLED = 1L << 60; + private static final long EVENT_DECODER_INIT = 1L << 59; + private static final long EVENT_DECODER_FORMAT_CHANGED = 1L << 58; + private static final long EVENT_DECODER_DISABLED = 1L << 57; private static final UUID DRM_SCHEME_UUID = UUID.nameUUIDFromBytes(TestUtil.createByteArray(7, 8, 9)); @@ -143,7 +158,7 @@ public final class AnalyticsCollectorTest { ExoPlayerTestRunner.VIDEO_FORMAT.buildUpon().setDrmInitData(DRM_DATA_1).build(); private static final int TIMEOUT_MS = 10_000; - private static final Timeline SINGLE_PERIOD_TIMELINE = new FakeTimeline(/* windowCount= */ 1); + private static final Timeline SINGLE_PERIOD_TIMELINE = new FakeTimeline(); private static final EventWindowAndPeriodId WINDOW_0 = new EventWindowAndPeriodId(/* windowIndex= */ 0, /* mediaPeriodId= */ null); private static final EventWindowAndPeriodId WINDOW_1 = @@ -202,7 +217,7 @@ public void singlePeriod() throws Exception { assertThat(listener.getEvents(EVENT_TIMELINE_CHANGED)) .containsExactly(WINDOW_0 /* PLAYLIST_CHANGED */, WINDOW_0 /* SOURCE_UPDATE */) .inOrder(); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) .containsExactly(period0 /* started */, period0 /* stopped */) .inOrder(); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)).containsExactly(period0); @@ -225,14 +240,13 @@ public void singlePeriod() throws Exception { .containsExactly(period0 /* audio */, period0 /* video */) .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_ENABLED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INIT)).containsExactly(period0); + assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INITIALIZED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_AUDIO_INPUT_FORMAT_CHANGED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_AUDIO_SESSION_ID)).containsExactly(period0); assertThat(listener.getEvents(EVENT_AUDIO_POSITION_ADVANCING)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)).containsExactly(period0); + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)).containsExactly(period0); + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_RENDERED_FIRST_FRAME)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_FRAME_PROCESSING_OFFSET)).containsExactly(period0); @@ -265,8 +279,8 @@ public void automaticPeriodTransition() throws Exception { .containsExactly(WINDOW_0 /* PLAYLIST_CHANGED */, period0 /* SOURCE_UPDATE */) .inOrder(); assertThat(listener.getEvents(EVENT_POSITION_DISCONTINUITY)).containsExactly(period1); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) - .containsExactly(period0, period0, period0, period0) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) + .containsExactly(period0, period0) .inOrder(); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)) .containsExactly(period0, period1) @@ -301,22 +315,21 @@ public void automaticPeriodTransition() throws Exception { period0 /* audio */, period0 /* video */, period1 /* audio */, period1 /* video */) .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_ENABLED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INITIALIZED)) .containsExactly(period0, period1) .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_INPUT_FORMAT_CHANGED)) .containsExactly(period0, period1) .inOrder(); - assertThat(listener.getEvents(EVENT_AUDIO_SESSION_ID)).containsExactly(period0); assertThat(listener.getEvents(EVENT_AUDIO_POSITION_ADVANCING)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)) .containsExactly(period0, period1) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)) .containsExactly(period0, period1) .inOrder(); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)).containsExactly(period1); + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)).containsExactly(period1); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)) .containsExactly(period0, period1) .inOrder(); @@ -347,8 +360,8 @@ public void periodTransitionWithRendererChange() throws Exception { .containsExactly(WINDOW_0 /* PLAYLIST_CHANGED */, period0 /* SOURCE_UPDATE */) .inOrder(); assertThat(listener.getEvents(EVENT_POSITION_DISCONTINUITY)).containsExactly(period1); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) - .containsExactly(period0, period0, period0, period0) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) + .containsExactly(period0, period0) .inOrder(); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)) .containsExactly(period0, period1) @@ -381,15 +394,14 @@ public void periodTransitionWithRendererChange() throws Exception { .inOrder(); assertThat(listener.getEvents(EVENT_DECODER_DISABLED)).containsExactly(period0 /* video */); assertThat(listener.getEvents(EVENT_AUDIO_ENABLED)).containsExactly(period1); - assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INIT)).containsExactly(period1); + assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INITIALIZED)).containsExactly(period1); assertThat(listener.getEvents(EVENT_AUDIO_INPUT_FORMAT_CHANGED)).containsExactly(period1); - assertThat(listener.getEvents(EVENT_AUDIO_SESSION_ID)).containsExactly(period1); assertThat(listener.getEvents(EVENT_AUDIO_POSITION_ADVANCING)).containsExactly(period1); assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)).containsExactly(period0); + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_DISABLED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)).containsExactly(period0); + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_RENDERED_FIRST_FRAME)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_FRAME_PROCESSING_OFFSET)).containsExactly(period0); @@ -411,8 +423,6 @@ public void seekToOtherPeriod() throws Exception { // Wait until second period has fully loaded to assert loading events without flakiness. .waitForIsLoading(true) .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) .seek(/* windowIndex= */ 1, /* positionMs= */ 0) .play() .build(); @@ -436,9 +446,9 @@ public void seekToOtherPeriod() throws Exception { assertThat(listener.getEvents(EVENT_POSITION_DISCONTINUITY)).containsExactly(period1); assertThat(listener.getEvents(EVENT_SEEK_STARTED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_SEEK_PROCESSED)).containsExactly(period1); - List loadingEvents = listener.getEvents(EVENT_LOADING_CHANGED); - assertThat(loadingEvents).hasSize(4); - assertThat(loadingEvents).containsAtLeast(period0, period0).inOrder(); + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) + .containsExactly(period0, period0) + .inOrder(); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)) .containsExactly(period0, period1, period1) .inOrder(); @@ -472,21 +482,18 @@ public void seekToOtherPeriod() throws Exception { .containsExactly(period0 /* video */, period0 /* audio */) .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_ENABLED)).containsExactly(period0, period1).inOrder(); - assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INITIALIZED)) .containsExactly(period0, period1) .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_INPUT_FORMAT_CHANGED)) .containsExactly(period0, period1) .inOrder(); - assertThat(listener.getEvents(EVENT_AUDIO_SESSION_ID)) - .containsExactly(period0, period1) - .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_POSITION_ADVANCING)) .containsExactly(period0, period1) .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_DISABLED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)).containsExactly(period0); + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_DISABLED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)).containsExactly(period0); @@ -537,8 +544,8 @@ public void seekBackAfterReadingAhead() throws Exception { .inOrder(); assertThat(listener.getEvents(EVENT_SEEK_STARTED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_SEEK_PROCESSED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) - .containsExactly(period0, period0, period0, period0, period0, period0) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) + .containsExactly(period0, period0, period0, period0) .inOrder(); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)) .containsExactly(period0, period1Seq2) @@ -575,28 +582,25 @@ public void seekBackAfterReadingAhead() throws Exception { assertThat(listener.getEvents(EVENT_AUDIO_ENABLED)) .containsExactly(period1, period1Seq2) .inOrder(); - assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_AUDIO_DECODER_INITIALIZED)) .containsExactly(period1Seq1, period1Seq2) .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_INPUT_FORMAT_CHANGED)) .containsExactly(period1Seq1, period1Seq2) .inOrder(); - assertThat(listener.getEvents(EVENT_AUDIO_SESSION_ID)) - .containsExactly(period1Seq1, period1Seq2) - .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_POSITION_ADVANCING)) .containsExactly(period1Seq1, period1Seq2) .inOrder(); assertThat(listener.getEvents(EVENT_AUDIO_DISABLED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)).containsExactly(period0, period0); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)) .containsExactly(period0, period1Seq1, period1Seq2) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)) .containsExactly(period0, period1Seq1, period1Seq2) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_DISABLED)).containsExactly(period0); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)) + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)) .containsExactly(period0, period1Seq2) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)) @@ -655,7 +659,7 @@ public void prepareNewSource() throws Exception { WINDOW_0 /* SOURCE_UPDATE */, WINDOW_0 /* PLAYLIST_CHANGE */, WINDOW_0 /* SOURCE_UPDATE */); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) .containsExactly(period0Seq0, period0Seq0, period0Seq1, period0Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)) @@ -692,14 +696,14 @@ public void prepareNewSource() throws Exception { assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)) .containsExactly(period0Seq0, period0Seq1) .inOrder(); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)) .containsExactly(period0Seq0, period0Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)) .containsExactly(period0Seq0, period0Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_DISABLED)).containsExactly(period0Seq0); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)).containsExactly(period0Seq1); + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)).containsExactly(period0Seq1); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)) .containsExactly(period0Seq0, period0Seq1) .inOrder(); @@ -748,7 +752,7 @@ public void reprepareAfterError() throws Exception { assertThat(listener.getEvents(EVENT_POSITION_DISCONTINUITY)).containsExactly(period0Seq0); assertThat(listener.getEvents(EVENT_SEEK_STARTED)).containsExactly(period0Seq0); assertThat(listener.getEvents(EVENT_SEEK_PROCESSED)).containsExactly(period0Seq0); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) .containsExactly(period0Seq0, period0Seq0, period0Seq0, period0Seq0); assertThat(listener.getEvents(EVENT_PLAYER_ERROR)).containsExactly(period0Seq0); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)).containsExactly(period0Seq0, period0Seq0); @@ -774,12 +778,12 @@ public void reprepareAfterError() throws Exception { .containsExactly(period0Seq0, period0Seq0); assertThat(listener.getEvents(EVENT_DECODER_DISABLED)).containsExactly(period0Seq0); assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)).containsExactly(period0Seq0, period0Seq0); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)) .containsExactly(period0Seq0, period0Seq0); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)) .containsExactly(period0Seq0, period0Seq0); assertThat(listener.getEvents(EVENT_VIDEO_DISABLED)).containsExactly(period0Seq0); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)).containsExactly(period0Seq0); + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)).containsExactly(period0Seq0); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)) .containsExactly(period0Seq0, period0Seq0); assertThat(listener.getEvents(EVENT_RENDERED_FIRST_FRAME)) @@ -833,9 +837,8 @@ public void dynamicTimelineChange() throws Exception { window0Period1Seq0 /* SOURCE_UPDATE (concatenated timeline replaces placeholder) */, period1Seq0 /* SOURCE_UPDATE (child sources in concatenating source moved) */) .inOrder(); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) - .containsExactly( - window0Period1Seq0, window0Period1Seq0, window0Period1Seq0, window0Period1Seq0); + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) + .containsExactly(window0Period1Seq0, window0Period1Seq0, period1Seq0, period1Seq0); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)).containsExactly(window0Period1Seq0); assertThat(listener.getEvents(EVENT_LOAD_STARTED)) .containsExactly( @@ -861,14 +864,14 @@ public void dynamicTimelineChange() throws Exception { assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)) .containsExactly(window0Period1Seq0, window0Period1Seq0) .inOrder(); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)) .containsExactly(window0Period1Seq0, window1Period0Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)) .containsExactly(window0Period1Seq0, window1Period0Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_DISABLED)).containsExactly(window0Period1Seq0); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)) + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)) .containsExactly(window0Period1Seq0, period1Seq0) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)) @@ -931,7 +934,7 @@ public void playlistOperations() throws Exception { period0Seq0 /* SOURCE_UPDATE (second item) */, period0Seq1 /* PLAYLIST_CHANGED (remove) */) .inOrder(); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) .containsExactly(period0Seq0, period0Seq0, period0Seq0, period0Seq0); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)) .containsExactly(period0Seq0, period0Seq1, period0Seq1) @@ -943,30 +946,30 @@ public void playlistOperations() throws Exception { .containsExactly(WINDOW_0 /* manifest */, period0Seq0 /* media */, period1Seq1 /* media */) .inOrder(); assertThat(listener.getEvents(EVENT_DOWNSTREAM_FORMAT_CHANGED)) - .containsExactly(period0Seq0, period0Seq1) + .containsExactly(period0Seq0, period1Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_DECODER_ENABLED)) - .containsExactly(period0Seq0, period0Seq1, period0Seq1) + .containsExactly(period0Seq0, period1Seq1, period0Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_DECODER_INIT)) - .containsExactly(period0Seq0, period0Seq1) + .containsExactly(period0Seq0, period1Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_DECODER_FORMAT_CHANGED)) - .containsExactly(period0Seq0, period0Seq1) + .containsExactly(period0Seq0, period1Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_DECODER_DISABLED)) .containsExactly(period0Seq0, period0Seq0); assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)) - .containsExactly(period0Seq0, period0Seq1, period0Seq1) + .containsExactly(period0Seq0, period1Seq1, period0Seq1) .inOrder(); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)) - .containsExactly(period0Seq0, period0Seq1) + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)) + .containsExactly(period0Seq0, period1Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)) - .containsExactly(period0Seq0, period0Seq1) + .containsExactly(period0Seq0, period1Seq1) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_DISABLED)).containsExactly(period0Seq0, period0Seq0); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)).containsExactly(period0Seq1); + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)).containsExactly(period0Seq1); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)) .containsExactly(period0Seq0, period0Seq1) .inOrder(); @@ -985,8 +988,8 @@ public void adPlayback() throws Exception { AtomicReference adPlaybackState = new AtomicReference<>( FakeTimeline.createAdPlaybackState( - /* adsPerAdGroup= */ 1, /* adGroupTimesUs...= */ - windowOffsetInFirstPeriodUs, + /* adsPerAdGroup= */ 1, + /* adGroupTimesUs...= */ windowOffsetInFirstPeriodUs, windowOffsetInFirstPeriodUs + 5 * C.MICROS_PER_SECOND, C.TIME_END_OF_SOURCE)); AtomicInteger playedAdCount = new AtomicInteger(0); @@ -1002,17 +1005,22 @@ public void adPlayback() throws Exception { FakeMediaSource fakeMediaSource = new FakeMediaSource( adTimeline, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, (unusedFormat, mediaPeriodId) -> { if (mediaPeriodId.isAd()) { - return ImmutableList.of(oneByteSample(/* timeUs= */ 0), END_OF_STREAM_ITEM); + return ImmutableList.of( + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM); } else { // Provide a single sample before and after the midroll ad and another after the // postroll. return ImmutableList.of( - oneByteSample(windowOffsetInFirstPeriodUs + C.MICROS_PER_SECOND), - oneByteSample(windowOffsetInFirstPeriodUs + 6 * C.MICROS_PER_SECOND), - oneByteSample(windowOffsetInFirstPeriodUs + contentDurationsUs), + oneByteSample( + windowOffsetInFirstPeriodUs + C.MICROS_PER_SECOND, C.BUFFER_FLAG_KEY_FRAME), + oneByteSample( + windowOffsetInFirstPeriodUs + 6 * C.MICROS_PER_SECOND, + C.BUFFER_FLAG_KEY_FRAME), + oneByteSample( + windowOffsetInFirstPeriodUs + contentDurationsUs, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM); } }, @@ -1056,16 +1064,6 @@ public void onPositionDiscontinuity( // Ensure everything is preloaded. .waitForIsLoading(true) .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) .waitForPlaybackState(Player.STATE_READY) // Wait in each content part to ensure previously triggered events get a chance to be // delivered. This prevents flakiness caused by playback progressing too fast. @@ -1143,10 +1141,8 @@ public void onPositionDiscontinuity( .containsExactly( contentAfterPreroll, midrollAd, contentAfterMidroll, postrollAd, contentAfterPostroll) .inOrder(); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) - .containsExactly( - prerollAd, prerollAd, prerollAd, prerollAd, prerollAd, prerollAd, prerollAd, prerollAd, - prerollAd, prerollAd, prerollAd, prerollAd) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) + .containsExactly(prerollAd, prerollAd) .inOrder(); assertThat(listener.getEvents(EVENT_TRACKS_CHANGED)) .containsExactly( @@ -1206,7 +1202,7 @@ public void onPositionDiscontinuity( contentAfterPostroll) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)).containsExactly(prerollAd); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)) .containsExactly( prerollAd, contentAfterPreroll, @@ -1224,7 +1220,7 @@ public void onPositionDiscontinuity( postrollAd, contentAfterPostroll) .inOrder(); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)) + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)) .containsExactly(contentAfterPreroll, contentAfterMidroll, contentAfterPostroll) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)) @@ -1269,15 +1265,19 @@ public void seekAfterMidroll() throws Exception { FakeMediaSource fakeMediaSource = new FakeMediaSource( adTimeline, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, (unusedFormat, mediaPeriodId) -> { if (mediaPeriodId.isAd()) { - return ImmutableList.of(oneByteSample(/* timeUs= */ 0), END_OF_STREAM_ITEM); + return ImmutableList.of( + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM); } else { // Provide a sample before the midroll and another after the seek point below (6s). return ImmutableList.of( - oneByteSample(windowOffsetInFirstPeriodUs + C.MICROS_PER_SECOND), - oneByteSample(windowOffsetInFirstPeriodUs + 7 * C.MICROS_PER_SECOND), + oneByteSample( + windowOffsetInFirstPeriodUs + C.MICROS_PER_SECOND, C.BUFFER_FLAG_KEY_FRAME), + oneByteSample( + windowOffsetInFirstPeriodUs + 7 * C.MICROS_PER_SECOND, + C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM); } }, @@ -1288,10 +1288,6 @@ public void seekAfterMidroll() throws Exception { // Ensure everything is preloaded. .waitForIsLoading(true) .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) - .waitForIsLoading(true) - .waitForIsLoading(false) // Seek behind the midroll. .seek(6 * C.MICROS_PER_SECOND) // Wait until loading started again to assert loading events without flakiness. @@ -1340,12 +1336,8 @@ public void seekAfterMidroll() throws Exception { .inOrder(); assertThat(listener.getEvents(EVENT_SEEK_STARTED)).containsExactly(contentBeforeMidroll); assertThat(listener.getEvents(EVENT_SEEK_PROCESSED)).containsExactly(contentAfterMidroll); - assertThat(listener.getEvents(EVENT_LOADING_CHANGED)) + assertThat(listener.getEvents(EVENT_IS_LOADING_CHANGED)) .containsExactly( - contentBeforeMidroll, - contentBeforeMidroll, - contentBeforeMidroll, - contentBeforeMidroll, contentBeforeMidroll, contentBeforeMidroll, midrollAd, @@ -1385,14 +1377,14 @@ public void seekAfterMidroll() throws Exception { assertThat(listener.getEvents(EVENT_VIDEO_ENABLED)) .containsExactly(contentBeforeMidroll, midrollAd) .inOrder(); - assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INIT)) + assertThat(listener.getEvents(EVENT_VIDEO_DECODER_INITIALIZED)) .containsExactly(contentBeforeMidroll, midrollAd, contentAfterMidroll) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_INPUT_FORMAT_CHANGED)) .containsExactly(contentBeforeMidroll, midrollAd, contentAfterMidroll) .inOrder(); assertThat(listener.getEvents(EVENT_VIDEO_DISABLED)).containsExactly(contentBeforeMidroll); - assertThat(listener.getEvents(EVENT_DROPPED_FRAMES)).containsExactly(contentAfterMidroll); + assertThat(listener.getEvents(EVENT_DROPPED_VIDEO_FRAMES)).containsExactly(contentAfterMidroll); assertThat(listener.getEvents(EVENT_VIDEO_SIZE_CHANGED)) .containsExactly(contentBeforeMidroll, midrollAd, contentAfterMidroll) .inOrder(); @@ -1435,7 +1427,7 @@ public void drmEvents_singlePeriod() throws Exception { TestAnalyticsListener listener = runAnalyticsTest(mediaSource); populateEventIds(listener.lastReportedTimeline); - assertThat(listener.getEvents(EVENT_DRM_ERROR)).isEmpty(); + assertThat(listener.getEvents(EVENT_DRM_SESSION_MANAGER_ERROR)).isEmpty(); assertThat(listener.getEvents(EVENT_DRM_SESSION_ACQUIRED)).containsExactly(period0); assertThat(listener.getEvents(EVENT_DRM_KEYS_LOADED)).containsExactly(period0); // The release event is lost because it's posted to "ExoPlayerTest thread" after that thread @@ -1452,7 +1444,7 @@ public void drmEvents_periodWithSameDrmData_keysReused() throws Exception { TestAnalyticsListener listener = runAnalyticsTest(mediaSource); populateEventIds(listener.lastReportedTimeline); - assertThat(listener.getEvents(EVENT_DRM_ERROR)).isEmpty(); + assertThat(listener.getEvents(EVENT_DRM_SESSION_MANAGER_ERROR)).isEmpty(); assertThat(listener.getEvents(EVENT_DRM_SESSION_ACQUIRED)) .containsExactly(period0, period1) .inOrder(); @@ -1474,7 +1466,7 @@ public void drmEvents_periodWithDifferentDrmData_keysLoadedAgain() throws Except TestAnalyticsListener listener = runAnalyticsTest(mediaSource); populateEventIds(listener.lastReportedTimeline); - assertThat(listener.getEvents(EVENT_DRM_ERROR)).isEmpty(); + assertThat(listener.getEvents(EVENT_DRM_SESSION_MANAGER_ERROR)).isEmpty(); assertThat(listener.getEvents(EVENT_DRM_SESSION_ACQUIRED)) .containsExactly(period0, period1) .inOrder(); @@ -1495,7 +1487,7 @@ public void drmEvents_errorHandling() throws Exception { TestAnalyticsListener listener = runAnalyticsTest(mediaSource); populateEventIds(listener.lastReportedTimeline); - assertThat(listener.getEvents(EVENT_DRM_ERROR)).containsExactly(period0); + assertThat(listener.getEvents(EVENT_DRM_SESSION_MANAGER_ERROR)).containsExactly(period0); assertThat(listener.getEvents(EVENT_PLAYER_ERROR)).containsExactly(period0); } @@ -1503,11 +1495,9 @@ public void drmEvents_errorHandling() throws Exception { public void onPlayerError_thrownDuringRendererEnableAtPeriodTransition_isReportedForNewPeriod() throws Exception { FakeMediaSource source0 = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.VIDEO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT); FakeMediaSource source1 = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT); RenderersFactory renderersFactory = (eventHandler, videoListener, audioListener, textOutput, metadataOutput) -> new Renderer[] { @@ -1537,11 +1527,9 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) public void onPlayerError_thrownDuringRenderAtPeriodTransition_isReportedForNewPeriod() throws Exception { FakeMediaSource source0 = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.VIDEO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT); FakeMediaSource source1 = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT); RenderersFactory renderersFactory = (eventHandler, videoListener, audioListener, textOutput, metadataOutput) -> new Renderer[] { @@ -1572,8 +1560,7 @@ public void render(long positionUs, long realtimeUs) throws ExoPlaybackException onPlayerError_thrownDuringRendererReplaceStreamAtPeriodTransition_isReportedForNewPeriod() throws Exception { FakeMediaSource source = - new FakeMediaSource( - new FakeTimeline(/* windowCount= */ 1), ExoPlayerTestRunner.AUDIO_FORMAT); + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT); RenderersFactory renderersFactory = (eventHandler, videoListener, audioListener, textOutput, metadataOutput) -> new Renderer[] { @@ -1604,6 +1591,257 @@ protected void onStreamChanged( assertThat(listener.getEvents(EVENT_PLAYER_ERROR)).containsExactly(period1); } + @Test + public void onEvents_isReportedWithCorrectEventTimes() throws Exception { + SimpleExoPlayer player = + new TestExoPlayerBuilder(ApplicationProvider.getApplicationContext()).build(); + AnalyticsListener listener = mock(AnalyticsListener.class); + Format[] formats = + new Format[] { + new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(), + new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build() + }; + player.addAnalyticsListener(listener); + + // Trigger some simultaneous events. + player.setMediaSource(new FakeMediaSource(new FakeTimeline(), formats)); + player.seekTo(2_000); + player.setPlaybackParameters(new PlaybackParameters(/* speed= */ 2.0f)); + ShadowLooper.runMainLooperToNextTask(); + + // Move to another item and fail with a third one to trigger events with different EventTimes. + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + player.addMediaSource(new FakeMediaSource(new FakeTimeline(), formats)); + player.play(); + TestPlayerRunHelper.runUntilPositionDiscontinuity( + player, Player.DISCONTINUITY_REASON_PERIOD_TRANSITION); + player.setMediaItem(MediaItem.fromUri("http://this-will-throw-an-exception.mp4")); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_IDLE); + ShadowLooper.runMainLooperToNextTask(); + player.release(); + + // Verify that expected individual callbacks have been called and capture EventTimes. + ArgumentCaptor individualTimelineChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onTimelineChanged(individualTimelineChangedEventTimes.capture(), anyInt()); + ArgumentCaptor individualMediaItemTransitionEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onMediaItemTransition(individualMediaItemTransitionEventTimes.capture(), any(), anyInt()); + ArgumentCaptor individualPositionDiscontinuityEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onPositionDiscontinuity(individualPositionDiscontinuityEventTimes.capture(), anyInt()); + ArgumentCaptor individualPlaybackStateChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onPlaybackStateChanged(individualPlaybackStateChangedEventTimes.capture(), anyInt()); + ArgumentCaptor individualIsLoadingChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onIsLoadingChanged(individualIsLoadingChangedEventTimes.capture(), anyBoolean()); + ArgumentCaptor individualTracksChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onTracksChanged(individualTracksChangedEventTimes.capture(), any(), any()); + ArgumentCaptor individualPlayWhenReadyChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onPlayWhenReadyChanged( + individualPlayWhenReadyChangedEventTimes.capture(), anyBoolean(), anyInt()); + ArgumentCaptor individualIsPlayingChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onIsPlayingChanged(individualIsPlayingChangedEventTimes.capture(), anyBoolean()); + ArgumentCaptor individualPlayerErrorEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()).onPlayerError(individualPlayerErrorEventTimes.capture(), any()); + ArgumentCaptor individualPlaybackParametersChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onPlaybackParametersChanged( + individualPlaybackParametersChangedEventTimes.capture(), any()); + ArgumentCaptor individualLoadStartedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onLoadStarted(individualLoadStartedEventTimes.capture(), any(), any()); + ArgumentCaptor individualLoadCompletedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onLoadCompleted(individualLoadCompletedEventTimes.capture(), any(), any()); + ArgumentCaptor individualLoadErrorEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onLoadError(individualLoadErrorEventTimes.capture(), any(), any(), any(), anyBoolean()); + ArgumentCaptor individualVideoEnabledEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onVideoEnabled(individualVideoEnabledEventTimes.capture(), any()); + ArgumentCaptor individualAudioEnabledEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onAudioEnabled(individualAudioEnabledEventTimes.capture(), any()); + ArgumentCaptor individualDownstreamFormatChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onDownstreamFormatChanged(individualDownstreamFormatChangedEventTimes.capture(), any()); + ArgumentCaptor individualVideoInputFormatChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onVideoInputFormatChanged( + individualVideoInputFormatChangedEventTimes.capture(), any(), any()); + ArgumentCaptor individualAudioInputFormatChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onAudioInputFormatChanged( + individualAudioInputFormatChangedEventTimes.capture(), any(), any()); + ArgumentCaptor individualVideoDecoderInitializedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onVideoDecoderInitialized( + individualVideoDecoderInitializedEventTimes.capture(), any(), anyLong()); + ArgumentCaptor individualAudioDecoderInitializedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onAudioDecoderInitialized( + individualAudioDecoderInitializedEventTimes.capture(), any(), anyLong()); + ArgumentCaptor individualVideoDisabledEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onVideoDisabled(individualVideoDisabledEventTimes.capture(), any()); + ArgumentCaptor individualAudioDisabledEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onAudioDisabled(individualAudioDisabledEventTimes.capture(), any()); + ArgumentCaptor individualRenderedFirstFrameEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onRenderedFirstFrame(individualRenderedFirstFrameEventTimes.capture(), any()); + ArgumentCaptor individualVideoSizeChangedEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onVideoSizeChanged( + individualVideoSizeChangedEventTimes.capture(), + anyInt(), + anyInt(), + anyInt(), + anyFloat()); + ArgumentCaptor individualAudioPositionAdvancingEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onAudioPositionAdvancing(individualAudioPositionAdvancingEventTimes.capture(), anyLong()); + ArgumentCaptor individualVideoProcessingOffsetEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onVideoFrameProcessingOffset( + individualVideoProcessingOffsetEventTimes.capture(), anyLong(), anyInt()); + ArgumentCaptor individualDroppedFramesEventTimes = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(listener, atLeastOnce()) + .onDroppedVideoFrames(individualDroppedFramesEventTimes.capture(), anyInt(), anyLong()); + + // Verify the EventTimes reported with onEvents are a non-empty subset of the individual + // callback EventTimes. We can only assert they are a non-empty subset because there may be + // multiple events of the same type arriving in the same message queue iteration. + ArgumentCaptor eventsCaptor = + ArgumentCaptor.forClass(AnalyticsListener.Events.class); + verify(listener, atLeastOnce()).onEvents(eq(player), eventsCaptor.capture()); + SparseArray> onEventsEventTimes = new SparseArray<>(); + for (AnalyticsListener.Events events : eventsCaptor.getAllValues()) { + for (int i = 0; i < events.size(); i++) { + @AnalyticsListener.EventFlags int event = events.get(i); + if (onEventsEventTimes.get(event) == null) { + onEventsEventTimes.put(event, new ArrayList<>()); + } + onEventsEventTimes.get(event).add(events.getEventTime(event)); + } + } + // SparseArray.get returns null if the key doesn't exist, thus verifying the sets are non-empty. + assertThat(individualTimelineChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_TIMELINE_CHANGED)) + .inOrder(); + assertThat(individualMediaItemTransitionEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_MEDIA_ITEM_TRANSITION)) + .inOrder(); + assertThat(individualPositionDiscontinuityEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_POSITION_DISCONTINUITY)) + .inOrder(); + assertThat(individualPlaybackStateChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_PLAYBACK_STATE_CHANGED)) + .inOrder(); + assertThat(individualIsLoadingChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_IS_LOADING_CHANGED)) + .inOrder(); + assertThat(individualTracksChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_TRACKS_CHANGED)) + .inOrder(); + assertThat(individualPlayWhenReadyChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_PLAY_WHEN_READY_CHANGED)) + .inOrder(); + assertThat(individualIsPlayingChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_IS_PLAYING_CHANGED)) + .inOrder(); + assertThat(individualPlayerErrorEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_PLAYER_ERROR)) + .inOrder(); + assertThat(individualPlaybackParametersChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_PLAYBACK_PARAMETERS_CHANGED)) + .inOrder(); + assertThat(individualLoadStartedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_LOAD_STARTED)) + .inOrder(); + assertThat(individualLoadCompletedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_LOAD_COMPLETED)) + .inOrder(); + assertThat(individualLoadErrorEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_LOAD_ERROR)) + .inOrder(); + assertThat(individualVideoEnabledEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_VIDEO_ENABLED)) + .inOrder(); + assertThat(individualAudioEnabledEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_AUDIO_ENABLED)) + .inOrder(); + assertThat(individualDownstreamFormatChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_DOWNSTREAM_FORMAT_CHANGED)) + .inOrder(); + assertThat(individualVideoInputFormatChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_VIDEO_INPUT_FORMAT_CHANGED)) + .inOrder(); + assertThat(individualAudioInputFormatChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_AUDIO_INPUT_FORMAT_CHANGED)) + .inOrder(); + assertThat(individualVideoDecoderInitializedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_VIDEO_DECODER_INITIALIZED)) + .inOrder(); + assertThat(individualAudioDecoderInitializedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_AUDIO_DECODER_INITIALIZED)) + .inOrder(); + assertThat(individualVideoDisabledEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_VIDEO_DISABLED)) + .inOrder(); + assertThat(individualAudioDisabledEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_AUDIO_DISABLED)) + .inOrder(); + assertThat(individualRenderedFirstFrameEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_RENDERED_FIRST_FRAME)) + .inOrder(); + assertThat(individualVideoSizeChangedEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_VIDEO_SIZE_CHANGED)) + .inOrder(); + assertThat(individualAudioPositionAdvancingEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_AUDIO_POSITION_ADVANCING)) + .inOrder(); + assertThat(individualVideoProcessingOffsetEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_VIDEO_FRAME_PROCESSING_OFFSET)) + .inOrder(); + assertThat(individualDroppedFramesEventTimes.getAllValues()) + .containsAtLeastElementsIn(onEventsEventTimes.get(EVENT_DROPPED_VIDEO_FRAMES)) + .inOrder(); + } + private void populateEventIds(Timeline timeline) { period0 = new EventWindowAndPeriodId( @@ -1646,6 +1884,37 @@ private void populateEventIds(Timeline timeline) { } } + @Test + public void recursiveListenerInvocation_arrivesInCorrectOrder() { + AnalyticsCollector analyticsCollector = new AnalyticsCollector(Clock.DEFAULT); + analyticsCollector.setPlayer( + new SimpleExoPlayer.Builder(ApplicationProvider.getApplicationContext()).build(), + Looper.myLooper()); + AnalyticsListener listener1 = mock(AnalyticsListener.class); + AnalyticsListener listener2 = + spy( + new AnalyticsListener() { + @Override + public void onPlayerError(EventTime eventTime, ExoPlaybackException error) { + analyticsCollector.onSurfaceSizeChanged(/* width= */ 0, /* height= */ 0); + } + }); + AnalyticsListener listener3 = mock(AnalyticsListener.class); + analyticsCollector.addListener(listener1); + analyticsCollector.addListener(listener2); + analyticsCollector.addListener(listener3); + + analyticsCollector.onPlayerError(ExoPlaybackException.createForSource(new IOException())); + + InOrder inOrder = Mockito.inOrder(listener1, listener2, listener3); + inOrder.verify(listener1).onPlayerError(any(), any()); + inOrder.verify(listener2).onPlayerError(any(), any()); + inOrder.verify(listener3).onPlayerError(any(), any()); + inOrder.verify(listener1).onSurfaceSizeChanged(any(), eq(0), eq(0)); + inOrder.verify(listener2).onSurfaceSizeChanged(any(), eq(0), eq(0)); + inOrder.verify(listener3).onSurfaceSizeChanged(any(), eq(0), eq(0)); + } + private static TestAnalyticsListener runAnalyticsTest(MediaSource mediaSource) throws Exception { return runAnalyticsTest(mediaSource, /* actionSchedule= */ null); } @@ -1748,7 +2017,7 @@ public TestAnalyticsListener() { lastReportedTimeline = Timeline.EMPTY; } - public List getEvents(int eventType) { + public List getEvents(long eventType) { ArrayList eventTimes = new ArrayList<>(); Iterator eventIterator = reportedEvents.iterator(); while (eventIterator.hasNext()) { @@ -1808,12 +2077,12 @@ public void onRepeatModeChanged(EventTime eventTime, int repeatMode) { @Override public void onShuffleModeChanged(EventTime eventTime, boolean shuffleModeEnabled) { - reportedEvents.add(new ReportedEvent(EVENT_SHUFFLE_MODE_CHANGED, eventTime)); + reportedEvents.add(new ReportedEvent(EVENT_SHUFFLE_MODE_ENABLED_CHANGED, eventTime)); } @Override public void onIsLoadingChanged(EventTime eventTime, boolean isLoading) { - reportedEvents.add(new ReportedEvent(EVENT_LOADING_CHANGED, eventTime)); + reportedEvents.add(new ReportedEvent(EVENT_IS_LOADING_CHANGED, eventTime)); } @Override @@ -1916,7 +2185,7 @@ public void onAudioEnabled(EventTime eventTime, DecoderCounters counters) { @Override public void onAudioDecoderInitialized( EventTime eventTime, String decoderName, long initializationDurationMs) { - reportedEvents.add(new ReportedEvent(EVENT_AUDIO_DECODER_INIT, eventTime)); + reportedEvents.add(new ReportedEvent(EVENT_AUDIO_DECODER_INITIALIZED, eventTime)); } @Override @@ -1930,7 +2199,7 @@ public void onAudioDisabled(EventTime eventTime, DecoderCounters counters) { } @Override - public void onAudioSessionId(EventTime eventTime, int audioSessionId) { + public void onAudioSessionIdChanged(EventTime eventTime, int audioSessionId) { reportedEvents.add(new ReportedEvent(EVENT_AUDIO_SESSION_ID, eventTime)); } @@ -1953,7 +2222,7 @@ public void onVideoEnabled(EventTime eventTime, DecoderCounters counters) { @Override public void onVideoDecoderInitialized( EventTime eventTime, String decoderName, long initializationDurationMs) { - reportedEvents.add(new ReportedEvent(EVENT_VIDEO_DECODER_INIT, eventTime)); + reportedEvents.add(new ReportedEvent(EVENT_VIDEO_DECODER_INITIALIZED, eventTime)); } @Override @@ -1963,7 +2232,7 @@ public void onVideoInputFormatChanged(EventTime eventTime, Format format) { @Override public void onDroppedVideoFrames(EventTime eventTime, int droppedFrames, long elapsedMs) { - reportedEvents.add(new ReportedEvent(EVENT_DROPPED_FRAMES, eventTime)); + reportedEvents.add(new ReportedEvent(EVENT_DROPPED_VIDEO_FRAMES, eventTime)); } @Override @@ -1978,7 +2247,7 @@ public void onVideoFrameProcessingOffset( } @Override - public void onRenderedFirstFrame(EventTime eventTime, Surface surface) { + public void onRenderedFirstFrame(EventTime eventTime, @Nullable Surface surface) { reportedEvents.add(new ReportedEvent(EVENT_RENDERED_FIRST_FRAME, eventTime)); } @@ -2004,7 +2273,7 @@ public void onDrmKeysLoaded(EventTime eventTime) { @Override public void onDrmSessionManagerError(EventTime eventTime, Exception error) { - reportedEvents.add(new ReportedEvent(EVENT_DRM_ERROR, eventTime)); + reportedEvents.add(new ReportedEvent(EVENT_DRM_SESSION_MANAGER_ERROR, eventTime)); } @Override @@ -2024,10 +2293,10 @@ public void onDrmSessionReleased(EventTime eventTime) { private static final class ReportedEvent { - public final int eventType; + public final long eventType; public final EventWindowAndPeriodId eventWindowAndPeriodId; - public ReportedEvent(int eventType, EventTime eventTime) { + public ReportedEvent(long eventType, EventTime eventTime) { this.eventType = eventType; this.eventWindowAndPeriodId = new EventWindowAndPeriodId(eventTime.windowIndex, eventTime.mediaPeriodId); @@ -2035,7 +2304,12 @@ public ReportedEvent(int eventType, EventTime eventTime) { @Override public String toString() { - return "{" + "type=" + eventType + ", windowAndPeriodId=" + eventWindowAndPeriodId + '}'; + return "{" + + "type=" + + Long.numberOfTrailingZeros(eventType) + + ", windowAndPeriodId=" + + eventWindowAndPeriodId + + '}'; } } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/analytics/DefaultPlaybackSessionManagerTest.java b/library/core/src/test/java/com/google/android/exoplayer2/analytics/DefaultPlaybackSessionManagerTest.java index 5f97ad78f23..034d9712ce7 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/analytics/DefaultPlaybackSessionManagerTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/analytics/DefaultPlaybackSessionManagerTest.java @@ -61,7 +61,7 @@ public void setUp() { @Test public void updateSessions_withoutMediaPeriodId_createsNewSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); EventTime eventTime = createEventTime(timeline, /* windowIndex= */ 0, /* mediaPeriodId */ null); sessionManager.updateSessions(eventTime); @@ -73,7 +73,7 @@ public void updateSessions_withoutMediaPeriodId_createsNewSession() { @Test public void updateSessions_withMediaPeriodId_createsNewSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -92,7 +92,7 @@ public void updateSessions_withMediaPeriodId_createsNewSession() { @Test public void updateSessions_ofSameWindow_withMediaPeriodId_afterWithoutMediaPeriodId_doesNotCreateNewSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -113,7 +113,7 @@ public void updateSessions_withMediaPeriodId_createsNewSession() { @Test public void updateSessions_ofSameWindow_withAd_afterWithoutMediaPeriodId_createsNewSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), @@ -141,7 +141,7 @@ public void updateSessions_ofSameWindow_withAd_afterWithoutMediaPeriodId_creates @Test public void updateSessions_ofSameWindow_withoutMediaPeriodId_afterMediaPeriodId_doesNotCreateNewSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -310,7 +310,7 @@ public void updateSessions_withMediaPeriodId_ofOtherWindow_createsNewSession() { @Test public void updateSessions_ofSameWindow_withNewWindowSequenceNumber_createsNewSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId1 = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -339,7 +339,7 @@ public void updateSessions_ofSameWindow_withNewWindowSequenceNumber_createsNewSe @Test public void updateSessions_withoutMediaPeriodId_andPreviouslyCreatedSessions_doesNotCreateNewSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId1 = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -372,7 +372,7 @@ public void updateSessions_ofSameWindow_withNewWindowSequenceNumber_createsNewSe @Test public void updateSessions_afterSessionForMediaPeriodId_withSameMediaPeriodId_returnsSameValue() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -390,7 +390,7 @@ public void updateSessions_afterSessionForMediaPeriodId_withSameMediaPeriodId_re @Test public void updateSessions_withoutMediaPeriodId_afterSessionForMediaPeriodId_returnsSameValue() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -417,9 +417,11 @@ public void updateSessions_withoutMediaPeriodId_afterSessionForMediaPeriodId_ret /* id= */ 0, /* isSeekable= */ true, /* isDynamic= */ false, - /* durationUs =*/ 10 * C.MICROS_PER_SECOND, + /* durationUs= */ 10 * C.MICROS_PER_SECOND, new AdPlaybackState( - /* adGroupTimesUs=... */ 2 * C.MICROS_PER_SECOND, 5 * C.MICROS_PER_SECOND) + /* adsId= */ new Object(), + /* adGroupTimesUs=... */ 2 * C.MICROS_PER_SECOND, + 5 * C.MICROS_PER_SECOND) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdCount(/* adGroupIndex= */ 1, /* adCount= */ 1))); EventTime adEventTime1 = @@ -472,7 +474,7 @@ public void updateSessions_withoutMediaPeriodId_afterSessionForMediaPeriodId_ret @Test public void getSessionForMediaPeriodId_returnsValue_butDoesNotCreateSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -486,7 +488,7 @@ public void getSessionForMediaPeriodId_returnsValue_butDoesNotCreateSession() { public void belongsToSession_withSameWindowIndex_returnsTrue() { EventTime eventTime = createEventTime(Timeline.EMPTY, /* windowIndex= */ 0, /* mediaPeriodId= */ null); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); EventTime eventTimeWithTimeline = createEventTime(timeline, /* windowIndex= */ 0, /* mediaPeriodId= */ null); MediaPeriodId mediaPeriodId = @@ -530,7 +532,7 @@ public void belongsToSession_withOtherWindowIndex_returnsFalse() { @Test public void belongsToSession_withOtherWindowSequenceNumber_returnsFalse() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); MediaPeriodId mediaPeriodId1 = new MediaPeriodId( timeline.getUidOfPeriod(/* periodIndex= */ 0), /* windowSequenceNumber= */ 0); @@ -590,7 +592,7 @@ public void initialTimelineUpdate_finishesAllSessionsOutsideTimeline() { createEventTime(Timeline.EMPTY, /* windowIndex= */ 1, /* mediaPeriodId= */ null); sessionManager.updateSessions(eventTime1); sessionManager.updateSessions(eventTime2); - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); EventTime newTimelineEventTime = createEventTime(timeline, /* windowIndex= */ 0, /* mediaPeriodId= */ null); @@ -700,8 +702,9 @@ public void timelineUpdate_withContent_doesNotFinishFuturePostrollAd() { /* id= */ 0, /* isSeekable= */ true, /* isDynamic= */ false, - /* durationUs =*/ 10 * C.MICROS_PER_SECOND, - new AdPlaybackState(/* adGroupTimesUs=... */ C.TIME_END_OF_SOURCE) + /* durationUs= */ 10 * C.MICROS_PER_SECOND, + new AdPlaybackState( + /* adsId= */ new Object(), /* adGroupTimesUs=... */ C.TIME_END_OF_SOURCE) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1))); EventTime adEventTime = createEventTime( @@ -824,7 +827,7 @@ public void positionDiscontinuity_toNewWindow_withSeekTransitionReason_finishesS @Test public void positionDiscontinuity_toSameWindow_withoutMediaPeriodId_doesNotFinishSession() { - Timeline timeline = new FakeTimeline(/* windowCount= */ 1); + Timeline timeline = new FakeTimeline(); EventTime eventTime1 = createEventTime( timeline, @@ -902,8 +905,11 @@ public void positionDiscontinuity_fromAdToContent_finishesAd() { /* id= */ 0, /* isSeekable= */ true, /* isDynamic= */ false, - /* durationUs =*/ 10 * C.MICROS_PER_SECOND, - new AdPlaybackState(/* adGroupTimesUs=... */ 0, 5 * C.MICROS_PER_SECOND) + /* durationUs= */ 10 * C.MICROS_PER_SECOND, + new AdPlaybackState( + /* adsId= */ new Object(), /* adGroupTimesUs=... */ + 0, + 5 * C.MICROS_PER_SECOND) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdCount(/* adGroupIndex= */ 1, /* adCount= */ 1))); EventTime adEventTime1 = @@ -981,9 +987,11 @@ public void positionDiscontinuity_fromContentToAd_doesNotFinishSessions() { /* id= */ 0, /* isSeekable= */ true, /* isDynamic= */ false, - /* durationUs =*/ 10 * C.MICROS_PER_SECOND, + /* durationUs= */ 10 * C.MICROS_PER_SECOND, new AdPlaybackState( - /* adGroupTimesUs=... */ 2 * C.MICROS_PER_SECOND, 5 * C.MICROS_PER_SECOND) + /* adsId= */ new Object(), /* adGroupTimesUs=... */ + 2 * C.MICROS_PER_SECOND, + 5 * C.MICROS_PER_SECOND) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdCount(/* adGroupIndex= */ 1, /* adCount= */ 1))); EventTime adEventTime1 = @@ -1031,8 +1039,11 @@ public void positionDiscontinuity_fromAdToAd_finishesPastAds_andNotifiesAdPlayba /* id= */ 0, /* isSeekable= */ true, /* isDynamic= */ false, - /* durationUs =*/ 10 * C.MICROS_PER_SECOND, - new AdPlaybackState(/* adGroupTimesUs=... */ 0, 5 * C.MICROS_PER_SECOND) + /* durationUs= */ 10 * C.MICROS_PER_SECOND, + new AdPlaybackState( + /* adsId= */ new Object(), /* adGroupTimesUs=... */ + 0, + 5 * C.MICROS_PER_SECOND) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdCount(/* adGroupIndex= */ 1, /* adCount= */ 1))); EventTime adEventTime1 = diff --git a/library/core/src/test/java/com/google/android/exoplayer2/analytics/PlaybackStatsListenerTest.java b/library/core/src/test/java/com/google/android/exoplayer2/analytics/PlaybackStatsListenerTest.java index 1f19c2af588..c736444a43b 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/analytics/PlaybackStatsListenerTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/analytics/PlaybackStatsListenerTest.java @@ -17,71 +17,60 @@ import static com.google.common.truth.Truth.assertThat; import static org.mockito.ArgumentMatchers.any; -import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.never; import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; +import static org.robolectric.shadows.ShadowLooper.runMainLooperToNextTask; -import android.os.SystemClock; import androidx.annotation.Nullable; +import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.PlaybackParameters; import com.google.android.exoplayer2.Player; -import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; import com.google.android.exoplayer2.source.MediaSource; +import com.google.android.exoplayer2.testutil.FakeMediaSource; import com.google.android.exoplayer2.testutil.FakeTimeline; +import com.google.android.exoplayer2.testutil.TestExoPlayerBuilder; +import com.google.common.collect.ImmutableList; +import java.util.stream.Collectors; +import org.junit.After; +import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; +import org.mockito.ArgumentCaptor; /** Unit test for {@link PlaybackStatsListener}. */ @RunWith(AndroidJUnit4.class) public final class PlaybackStatsListenerTest { - private static final AnalyticsListener.EventTime EMPTY_TIMELINE_EVENT_TIME = - new AnalyticsListener.EventTime( - /* realtimeMs= */ 500, - Timeline.EMPTY, - /* windowIndex= */ 0, - /* mediaPeriodId= */ null, - /* eventPlaybackPositionMs= */ 0, - /* currentTimeline= */ Timeline.EMPTY, - /* currentWindowIndex= */ 0, - /* currentMediaPeriodId= */ null, - /* currentPlaybackPositionMs= */ 0, - /* totalBufferedDurationMs= */ 0); - private static final Timeline TEST_TIMELINE = new FakeTimeline(/* windowCount= */ 1); - private static final MediaSource.MediaPeriodId TEST_MEDIA_PERIOD_ID = - new MediaSource.MediaPeriodId( - TEST_TIMELINE.getPeriod(/* periodIndex= */ 0, new Timeline.Period(), /* setIds= */ true) - .uid, - /* windowSequenceNumber= */ 42); - private static final AnalyticsListener.EventTime TEST_EVENT_TIME = - new AnalyticsListener.EventTime( - /* realtimeMs= */ 500, - TEST_TIMELINE, - /* windowIndex= */ 0, - TEST_MEDIA_PERIOD_ID, - /* eventPlaybackPositionMs= */ 123, - TEST_TIMELINE, - /* currentWindowIndex= */ 0, - TEST_MEDIA_PERIOD_ID, - /* currentPlaybackPositionMs= */ 123, - /* totalBufferedDurationMs= */ 456); + private SimpleExoPlayer player; + + @Before + public void setUp() { + player = new TestExoPlayerBuilder(ApplicationProvider.getApplicationContext()).build(); + } + + @After + public void tearDown() { + player.release(); + } @Test public void events_duringInitialIdleState_dontCreateNewPlaybackStats() { PlaybackStatsListener playbackStatsListener = new PlaybackStatsListener(/* keepHistory= */ true, /* callback= */ null); + player.addAnalyticsListener(playbackStatsListener); - playbackStatsListener.onPositionDiscontinuity( - EMPTY_TIMELINE_EVENT_TIME, Player.DISCONTINUITY_REASON_SEEK); - playbackStatsListener.onPlaybackParametersChanged( - EMPTY_TIMELINE_EVENT_TIME, new PlaybackParameters(/* speed= */ 2.0f)); - playbackStatsListener.onPlayWhenReadyChanged( - EMPTY_TIMELINE_EVENT_TIME, - /* playWhenReady= */ true, - Player.PLAY_WHEN_READY_CHANGE_REASON_USER_REQUEST); + player.seekTo(/* positionMs= */ 1234); + runMainLooperToNextTask(); + player.setPlaybackParameters(new PlaybackParameters(/* speed= */ 2f)); + runMainLooperToNextTask(); + player.play(); + runMainLooperToNextTask(); assertThat(playbackStatsListener.getPlaybackStats()).isNull(); } @@ -90,8 +79,10 @@ public void events_duringInitialIdleState_dontCreateNewPlaybackStats() { public void stateChangeEvent_toNonIdle_createsInitialPlaybackStats() { PlaybackStatsListener playbackStatsListener = new PlaybackStatsListener(/* keepHistory= */ true, /* callback= */ null); + player.addAnalyticsListener(playbackStatsListener); - playbackStatsListener.onPlaybackStateChanged(EMPTY_TIMELINE_EVENT_TIME, Player.STATE_BUFFERING); + player.prepare(); + runMainLooperToNextTask(); assertThat(playbackStatsListener.getPlaybackStats()).isNotNull(); } @@ -100,21 +91,25 @@ public void stateChangeEvent_toNonIdle_createsInitialPlaybackStats() { public void timelineChangeEvent_toNonEmpty_createsInitialPlaybackStats() { PlaybackStatsListener playbackStatsListener = new PlaybackStatsListener(/* keepHistory= */ true, /* callback= */ null); + player.addAnalyticsListener(playbackStatsListener); - playbackStatsListener.onTimelineChanged( - TEST_EVENT_TIME, Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + player.setMediaItem(MediaItem.fromUri("http://test.org")); + runMainLooperToNextTask(); assertThat(playbackStatsListener.getPlaybackStats()).isNotNull(); } @Test - public void playback_withKeepHistory_updatesStats() { + public void playback_withKeepHistory_updatesStats() throws Exception { PlaybackStatsListener playbackStatsListener = new PlaybackStatsListener(/* keepHistory= */ true, /* callback= */ null); + player.addAnalyticsListener(playbackStatsListener); - playbackStatsListener.onPlaybackStateChanged(TEST_EVENT_TIME, Player.STATE_BUFFERING); - playbackStatsListener.onPlaybackStateChanged(TEST_EVENT_TIME, Player.STATE_READY); - playbackStatsListener.onPlaybackStateChanged(TEST_EVENT_TIME, Player.STATE_ENDED); + player.setMediaSource(new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + runMainLooperToNextTask(); @Nullable PlaybackStats playbackStats = playbackStatsListener.getPlaybackStats(); assertThat(playbackStats).isNotNull(); @@ -122,13 +117,16 @@ public void playback_withKeepHistory_updatesStats() { } @Test - public void playback_withoutKeepHistory_updatesStats() { + public void playback_withoutKeepHistory_updatesStats() throws Exception { PlaybackStatsListener playbackStatsListener = new PlaybackStatsListener(/* keepHistory= */ false, /* callback= */ null); + player.addAnalyticsListener(playbackStatsListener); - playbackStatsListener.onPlaybackStateChanged(TEST_EVENT_TIME, Player.STATE_BUFFERING); - playbackStatsListener.onPlaybackStateChanged(TEST_EVENT_TIME, Player.STATE_READY); - playbackStatsListener.onPlaybackStateChanged(TEST_EVENT_TIME, Player.STATE_ENDED); + player.setMediaSource(new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + runMainLooperToNextTask(); @Nullable PlaybackStats playbackStats = playbackStatsListener.getPlaybackStats(); assertThat(playbackStats).isNotNull(); @@ -140,68 +138,45 @@ public void finishedSession_callsCallback() { PlaybackStatsListener.Callback callback = mock(PlaybackStatsListener.Callback.class); PlaybackStatsListener playbackStatsListener = new PlaybackStatsListener(/* keepHistory= */ true, callback); + player.addAnalyticsListener(playbackStatsListener); - // Create session with an event and finish it by simulating removal from playlist. - playbackStatsListener.onPlaybackStateChanged(TEST_EVENT_TIME, Player.STATE_BUFFERING); + // Create session with some events and finish it by removing it from the playlist. + player.setMediaSource(new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1))); + player.prepare(); + runMainLooperToNextTask(); verify(callback, never()).onPlaybackStatsReady(any(), any()); - playbackStatsListener.onTimelineChanged( - EMPTY_TIMELINE_EVENT_TIME, Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); - - verify(callback).onPlaybackStatsReady(eq(TEST_EVENT_TIME), any()); - } - - @Test - public void finishAllSessions_callsAllPendingCallbacks() { - AnalyticsListener.EventTime eventTimeWindow0 = - new AnalyticsListener.EventTime( - /* realtimeMs= */ 0, - Timeline.EMPTY, - /* windowIndex= */ 0, - /* mediaPeriodId= */ null, - /* eventPlaybackPositionMs= */ 0, - Timeline.EMPTY, - /* currentWindowIndex= */ 0, - /* currentMediaPeriodId= */ null, - /* currentPlaybackPositionMs= */ 0, - /* totalBufferedDurationMs= */ 0); - AnalyticsListener.EventTime eventTimeWindow1 = - new AnalyticsListener.EventTime( - /* realtimeMs= */ 0, - Timeline.EMPTY, - /* windowIndex= */ 1, - /* mediaPeriodId= */ null, - /* eventPlaybackPositionMs= */ 0, - Timeline.EMPTY, - /* currentWindowIndex= */ 1, - /* currentMediaPeriodId= */ null, - /* currentPlaybackPositionMs= */ 0, - /* totalBufferedDurationMs= */ 0); - PlaybackStatsListener.Callback callback = mock(PlaybackStatsListener.Callback.class); - PlaybackStatsListener playbackStatsListener = - new PlaybackStatsListener(/* keepHistory= */ true, callback); - playbackStatsListener.onPlaybackStateChanged(eventTimeWindow0, Player.STATE_BUFFERING); - playbackStatsListener.onPlaybackStateChanged(eventTimeWindow1, Player.STATE_BUFFERING); - - playbackStatsListener.finishAllSessions(); + player.clearMediaItems(); + runMainLooperToNextTask(); - verify(callback, times(2)).onPlaybackStatsReady(any(), any()); - verify(callback).onPlaybackStatsReady(eq(eventTimeWindow0), any()); - verify(callback).onPlaybackStatsReady(eq(eventTimeWindow1), any()); + verify(callback).onPlaybackStatsReady(any(), any()); } @Test - public void finishAllSessions_doesNotCallCallbackAgainWhenSessionWouldBeAutomaticallyFinished() { + public void playerRelease_callsAllPendingCallbacks() throws Exception { PlaybackStatsListener.Callback callback = mock(PlaybackStatsListener.Callback.class); PlaybackStatsListener playbackStatsListener = new PlaybackStatsListener(/* keepHistory= */ true, callback); - playbackStatsListener.onPlaybackStateChanged(TEST_EVENT_TIME, Player.STATE_BUFFERING); - SystemClock.setCurrentTimeMillis(TEST_EVENT_TIME.realtimeMs + 100); - - playbackStatsListener.finishAllSessions(); - // Simulate removing the playback item to ensure the session would finish if it hadn't already. - playbackStatsListener.onTimelineChanged( - EMPTY_TIMELINE_EVENT_TIME, Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); - - verify(callback).onPlaybackStatsReady(any(), any()); + player.addAnalyticsListener(playbackStatsListener); + + MediaSource mediaSource = new FakeMediaSource(new FakeTimeline(/* windowCount= */ 1)); + player.setMediaSources(ImmutableList.of(mediaSource, mediaSource)); + player.prepare(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_READY); + // Play close to the end of the first item to ensure the second session is already created, but + // the first one isn't finished yet. + TestPlayerRunHelper.playUntilPosition( + player, /* windowIndex= */ 0, /* positionMs= */ player.getDuration()); + runMainLooperToNextTask(); + player.release(); + runMainLooperToNextTask(); + + ArgumentCaptor eventTimeCaptor = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(callback, times(2)).onPlaybackStatsReady(eventTimeCaptor.capture(), any()); + assertThat( + eventTimeCaptor.getAllValues().stream() + .map(eventTime -> eventTime.windowIndex) + .collect(Collectors.toList())) + .containsExactly(0, 1); } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/audio/DecoderAudioRendererTest.java b/library/core/src/test/java/com/google/android/exoplayer2/audio/DecoderAudioRendererTest.java index 7e9126be986..f24e09346f4 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/audio/DecoderAudioRendererTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/audio/DecoderAudioRendererTest.java @@ -15,8 +15,8 @@ */ package com.google.android.exoplayer2.audio; +import static com.google.android.exoplayer2.C.FORMAT_HANDLED; import static com.google.android.exoplayer2.RendererCapabilities.ADAPTIVE_NOT_SEAMLESS; -import static com.google.android.exoplayer2.RendererCapabilities.FORMAT_HANDLED; import static com.google.android.exoplayer2.RendererCapabilities.TUNNELING_NOT_SUPPORTED; import static com.google.android.exoplayer2.RendererCapabilities.TUNNELING_SUPPORTED; import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; @@ -38,6 +38,7 @@ import com.google.android.exoplayer2.drm.DrmSessionManager; import com.google.android.exoplayer2.drm.ExoMediaCrypto; import com.google.android.exoplayer2.testutil.FakeSampleStream; +import com.google.android.exoplayer2.upstream.DefaultAllocator; import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.collect.ImmutableList; import org.junit.Before; @@ -68,7 +69,7 @@ public String getName() { } @Override - @FormatSupport + @C.FormatSupport protected int supportsFormatInternal(Format format) { return FORMAT_HANDLED; } @@ -102,15 +103,19 @@ public void supportsFormatAtApi21() { @Test public void immediatelyReadEndOfStreamPlaysAudioSinkToEndOfStream() throws Exception { - audioRenderer.enable( - RendererConfiguration.DEFAULT, - new Format[] {FORMAT}, + FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), FORMAT, - ImmutableList.of(END_OF_STREAM_ITEM)), + ImmutableList.of(END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); + audioRenderer.enable( + RendererConfiguration.DEFAULT, + new Format[] {FORMAT}, + fakeSampleStream, /* positionUs= */ 0, /* joining= */ false, /* mayRenderStartOfStream= */ true, diff --git a/library/core/src/test/java/com/google/android/exoplayer2/audio/DefaultAudioSinkTest.java b/library/core/src/test/java/com/google/android/exoplayer2/audio/DefaultAudioSinkTest.java index 2f86988d424..7b9e639cd64 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/audio/DefaultAudioSinkTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/audio/DefaultAudioSinkTest.java @@ -30,6 +30,7 @@ import java.nio.ByteBuffer; import java.nio.ByteOrder; import java.util.Arrays; +import org.junit.Assert; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -307,6 +308,18 @@ public void getCurrentPosition_returnsUnset_afterExperimentalFlush() throws Exce .isEqualTo(CURRENT_POSITION_NOT_SET); } + @Test + public void configure_throwsConfigurationException_withInvalidInput() { + Format format = new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build(); + AudioSink.ConfigurationException thrown = + Assert.assertThrows( + AudioSink.ConfigurationException.class, + () -> + defaultAudioSink.configure( + format, /* specifiedBufferSize= */ 0, /* outputChannels= */ null)); + assertThat(thrown.format).isEqualTo(format); + } + private void configureDefaultAudioSink(int channelCount) throws AudioSink.ConfigurationException { configureDefaultAudioSink(channelCount, /* trimStartFrames= */ 0, /* trimEndFrames= */ 0); } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/audio/MediaCodecAudioRendererTest.java b/library/core/src/test/java/com/google/android/exoplayer2/audio/MediaCodecAudioRendererTest.java index 922431d2108..c69deeaeef5 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/audio/MediaCodecAudioRendererTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/audio/MediaCodecAudioRendererTest.java @@ -22,10 +22,14 @@ import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyInt; import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.Mockito.atLeastOnce; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; +import static org.robolectric.Shadows.shadowOf; import android.media.MediaFormat; +import android.os.Handler; +import android.os.Looper; import android.os.SystemClock; import androidx.annotation.Nullable; import androidx.test.core.app.ApplicationProvider; @@ -39,6 +43,7 @@ import com.google.android.exoplayer2.mediacodec.MediaCodecInfo; import com.google.android.exoplayer2.mediacodec.MediaCodecSelector; import com.google.android.exoplayer2.testutil.FakeSampleStream; +import com.google.android.exoplayer2.upstream.DefaultAllocator; import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.collect.ImmutableList; import java.util.Collections; @@ -46,6 +51,7 @@ import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; +import org.mockito.ArgumentCaptor; import org.mockito.Mock; import org.mockito.junit.MockitoJUnit; import org.mockito.junit.MockitoRule; @@ -71,6 +77,7 @@ public class MediaCodecAudioRendererTest { private MediaCodecSelector mediaCodecSelector; @Mock private AudioSink audioSink; + @Mock private AudioRendererEventListener audioRendererEventListener; @Before public void setUp() throws Exception { @@ -94,13 +101,15 @@ public void setUp() throws Exception { /* forceDisableAdaptive= */ false, /* forceSecure= */ false)); + Handler eventHandler = new Handler(Looper.getMainLooper()); + mediaCodecAudioRenderer = new MediaCodecAudioRenderer( ApplicationProvider.getApplicationContext(), mediaCodecSelector, /* enableDecoderFallback= */ false, - /* eventHandler= */ null, - /* eventListener= */ null, + eventHandler, + audioRendererEventListener, audioSink); } @@ -110,8 +119,9 @@ public void render_configuresAudioSink_afterFormatChange() throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ AUDIO_AAC, ImmutableList.of( @@ -123,6 +133,7 @@ public void render_configuresAudioSink_afterFormatChange() throws Exception { oneByteSample(/* timeUs= */ 200, C.BUFFER_FLAG_KEY_FRAME), oneByteSample(/* timeUs= */ 250, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); mediaCodecAudioRenderer.enable( RendererConfiguration.DEFAULT, @@ -165,8 +176,9 @@ public void render_configuresAudioSink_afterGaplessFormatChange() throws Excepti FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ AUDIO_AAC, ImmutableList.of( @@ -178,6 +190,7 @@ public void render_configuresAudioSink_afterGaplessFormatChange() throws Excepti oneByteSample(/* timeUs= */ 200, C.BUFFER_FLAG_KEY_FRAME), oneByteSample(/* timeUs= */ 250, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); mediaCodecAudioRenderer.enable( RendererConfiguration.DEFAULT, @@ -228,11 +241,11 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF if (!format.equals(AUDIO_AAC)) { setPendingPlaybackException( ExoPlaybackException.createForRenderer( - new AudioSink.ConfigurationException("Test"), + new AudioSink.ConfigurationException("Test", format), "rendererName", /* rendererIndex= */ 0, format, - FORMAT_HANDLED)); + C.FORMAT_HANDLED)); } } }; @@ -241,12 +254,14 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ AUDIO_AAC, ImmutableList.of( oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); exceptionThrowingRenderer.enable( RendererConfiguration.DEFAULT, @@ -279,6 +294,23 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF exceptionThrowingRenderer.render(/* positionUs= */ 750, SystemClock.elapsedRealtime() * 1000); } + @Test + public void + render_callsAudioRendererEventListener_whenAudioSinkListenerOnAudioSinkErrorIsCalled() { + final ArgumentCaptor listenerCaptor = + ArgumentCaptor.forClass(AudioSink.Listener.class); + verify(audioSink, atLeastOnce()).setListener(listenerCaptor.capture()); + AudioSink.Listener audioSinkListener = listenerCaptor.getValue(); + + Exception error = + new AudioSink.WriteException( + /* errorCode= */ 1, new Format.Builder().build(), /* isRecoverable= */ true); + audioSinkListener.onAudioSinkError(error); + + shadowOf(Looper.getMainLooper()).idle(); + verify(audioRendererEventListener).onAudioSinkError(error); + } + private static Format getAudioSinkFormat(Format inputFormat) { return new Format.Builder() .setSampleMimeType(MimeTypes.AUDIO_RAW) diff --git a/library/core/src/test/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManagerTest.java b/library/core/src/test/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManagerTest.java index a700350b0b2..5ac26a76b31 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManagerTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/drm/DefaultDrmSessionManagerTest.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.drm; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import static java.util.concurrent.TimeUnit.SECONDS; @@ -24,7 +25,6 @@ import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.testutil.FakeExoMediaDrm; import com.google.android.exoplayer2.testutil.TestUtil; -import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.collect.ImmutableList; import java.util.UUID; @@ -61,10 +61,11 @@ public void acquireSession_triggersKeyLoadAndSessionIsOpened() throws Exception .build(/* mediaDrmCallback= */ licenseServer); drmSessionManager.prepare(); DrmSession drmSession = - drmSessionManager.acquireSession( - /* playbackLooper= */ Assertions.checkNotNull(Looper.myLooper()), - /* eventDispatcher= */ null, - FORMAT_WITH_DRM_INIT_DATA); + checkNotNull( + drmSessionManager.acquireSession( + /* playbackLooper= */ checkNotNull(Looper.myLooper()), + /* eventDispatcher= */ null, + FORMAT_WITH_DRM_INIT_DATA)); waitForOpenedWithKeys(drmSession); assertThat(drmSession.getState()).isEqualTo(DrmSession.STATE_OPENED_WITH_KEYS); @@ -84,10 +85,11 @@ public void keepaliveEnabled_sessionsKeptForRequestedTime() throws Exception { drmSessionManager.prepare(); DrmSession drmSession = - drmSessionManager.acquireSession( - /* playbackLooper= */ Assertions.checkNotNull(Looper.myLooper()), - /* eventDispatcher= */ null, - FORMAT_WITH_DRM_INIT_DATA); + checkNotNull( + drmSessionManager.acquireSession( + /* playbackLooper= */ checkNotNull(Looper.myLooper()), + /* eventDispatcher= */ null, + FORMAT_WITH_DRM_INIT_DATA)); waitForOpenedWithKeys(drmSession); assertThat(drmSession.getState()).isEqualTo(DrmSession.STATE_OPENED_WITH_KEYS); @@ -109,10 +111,11 @@ public void keepaliveDisabled_sessionsReleasedImmediately() throws Exception { drmSessionManager.prepare(); DrmSession drmSession = - drmSessionManager.acquireSession( - /* playbackLooper= */ Assertions.checkNotNull(Looper.myLooper()), - /* eventDispatcher= */ null, - FORMAT_WITH_DRM_INIT_DATA); + checkNotNull( + drmSessionManager.acquireSession( + /* playbackLooper= */ checkNotNull(Looper.myLooper()), + /* eventDispatcher= */ null, + FORMAT_WITH_DRM_INIT_DATA)); waitForOpenedWithKeys(drmSession); drmSession.release(/* eventDispatcher= */ null); @@ -131,10 +134,11 @@ public void managerRelease_allKeepaliveSessionsImmediatelyReleased() throws Exce drmSessionManager.prepare(); DrmSession drmSession = - drmSessionManager.acquireSession( - /* playbackLooper= */ Assertions.checkNotNull(Looper.myLooper()), - /* eventDispatcher= */ null, - FORMAT_WITH_DRM_INIT_DATA); + checkNotNull( + drmSessionManager.acquireSession( + /* playbackLooper= */ checkNotNull(Looper.myLooper()), + /* eventDispatcher= */ null, + FORMAT_WITH_DRM_INIT_DATA)); waitForOpenedWithKeys(drmSession); drmSession.release(/* eventDispatcher= */ null); @@ -161,10 +165,11 @@ public void maxConcurrentSessionsExceeded_allKeepAliveSessionsEagerlyReleased() drmSessionManager.prepare(); DrmSession firstDrmSession = - drmSessionManager.acquireSession( - /* playbackLooper= */ Assertions.checkNotNull(Looper.myLooper()), - /* eventDispatcher= */ null, - FORMAT_WITH_DRM_INIT_DATA); + checkNotNull( + drmSessionManager.acquireSession( + /* playbackLooper= */ checkNotNull(Looper.myLooper()), + /* eventDispatcher= */ null, + FORMAT_WITH_DRM_INIT_DATA)); waitForOpenedWithKeys(firstDrmSession); firstDrmSession.release(/* eventDispatcher= */ null); @@ -172,10 +177,11 @@ public void maxConcurrentSessionsExceeded_allKeepAliveSessionsEagerlyReleased() // drmSessionManager's internal reference. assertThat(firstDrmSession.getState()).isEqualTo(DrmSession.STATE_OPENED_WITH_KEYS); DrmSession secondDrmSession = - drmSessionManager.acquireSession( - /* playbackLooper= */ Assertions.checkNotNull(Looper.myLooper()), - /* eventDispatcher= */ null, - secondFormatWithDrmInitData); + checkNotNull( + drmSessionManager.acquireSession( + /* playbackLooper= */ checkNotNull(Looper.myLooper()), + /* eventDispatcher= */ null, + secondFormatWithDrmInitData)); // The drmSessionManager had to release firstDrmSession in order to acquire secondDrmSession. assertThat(firstDrmSession.getState()).isEqualTo(DrmSession.STATE_RELEASED); @@ -195,10 +201,11 @@ public void sessionReacquired_keepaliveTimeOutCancelled() throws Exception { drmSessionManager.prepare(); DrmSession firstDrmSession = - drmSessionManager.acquireSession( - /* playbackLooper= */ Assertions.checkNotNull(Looper.myLooper()), - /* eventDispatcher= */ null, - FORMAT_WITH_DRM_INIT_DATA); + checkNotNull( + drmSessionManager.acquireSession( + /* playbackLooper= */ checkNotNull(Looper.myLooper()), + /* eventDispatcher= */ null, + FORMAT_WITH_DRM_INIT_DATA)); waitForOpenedWithKeys(firstDrmSession); firstDrmSession.release(/* eventDispatcher= */ null); @@ -207,10 +214,11 @@ public void sessionReacquired_keepaliveTimeOutCancelled() throws Exception { // Acquire a session for the same init data 5s in to the 10s timeout (so expect the same // instance). DrmSession secondDrmSession = - drmSessionManager.acquireSession( - /* playbackLooper= */ Assertions.checkNotNull(Looper.myLooper()), - /* eventDispatcher= */ null, - FORMAT_WITH_DRM_INIT_DATA); + checkNotNull( + drmSessionManager.acquireSession( + /* playbackLooper= */ checkNotNull(Looper.myLooper()), + /* eventDispatcher= */ null, + FORMAT_WITH_DRM_INIT_DATA)); assertThat(secondDrmSession).isSameInstanceAs(firstDrmSession); // Let the timeout definitely expire, and check the session didn't get released. diff --git a/library/core/src/test/java/com/google/android/exoplayer2/drm/OfflineLicenseHelperTest.java b/library/core/src/test/java/com/google/android/exoplayer2/drm/OfflineLicenseHelperTest.java index ae579b1b7cf..881abb7c2b0 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/drm/OfflineLicenseHelperTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/drm/OfflineLicenseHelperTest.java @@ -26,6 +26,7 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.drm.DrmInitData.SchemeData; +import com.google.android.exoplayer2.drm.ExoMediaDrm.KeyRequest; import java.util.HashMap; import org.junit.After; import org.junit.Before; @@ -48,7 +49,10 @@ public void setUp() throws Exception { when(mediaDrm.openSession()).thenReturn(new byte[] {1, 2, 3}); when(mediaDrm.getKeyRequest(any(), any(), anyInt(), any())) .thenReturn( - new ExoMediaDrm.KeyRequest(/* data= */ new byte[0], /* licenseServerUrl= */ "")); + new KeyRequest( + /* data= */ new byte[0], + /* licenseServerUrl= */ "", + KeyRequest.REQUEST_TYPE_INITIAL)); offlineLicenseHelper = new OfflineLicenseHelper( new DefaultDrmSessionManager.Builder() diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/EndToEndGaplessTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/EndToEndGaplessTest.java new file mode 100644 index 00000000000..9cd334ac0f9 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/EndToEndGaplessTest.java @@ -0,0 +1,159 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import static com.google.common.truth.Truth.assertThat; +import static java.lang.Integer.max; + +import android.media.AudioFormat; +import android.media.MediaFormat; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.mediacodec.MediaCodecUtil; +import com.google.android.exoplayer2.robolectric.RandomizedMp3Decoder; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.util.Assertions; +import com.google.common.collect.ImmutableList; +import com.google.common.primitives.Bytes; +import java.io.ByteArrayOutputStream; +import java.util.Arrays; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; +import org.robolectric.shadows.MediaCodecInfoBuilder; +import org.robolectric.shadows.ShadowAudioTrack; +import org.robolectric.shadows.ShadowMediaCodec; +import org.robolectric.shadows.ShadowMediaCodecList; + +/** End to end playback test for gapless audio playbacks. */ +@RunWith(AndroidJUnit4.class) +@Config(sdk = 29) +public class EndToEndGaplessTest { + private static final int CODEC_INPUT_BUFFER_SIZE = 5120; + private static final int CODEC_OUTPUT_BUFFER_SIZE = 5120; + private static final String DECODER_NAME = "RandomizedMp3Decoder"; + + private RandomizedMp3Decoder mp3Decoder; + private AudioTrackListener audioTrackListener; + + @Before + public void setUp() throws Exception { + audioTrackListener = new AudioTrackListener(); + ShadowAudioTrack.addAudioDataListener(audioTrackListener); + + mp3Decoder = new RandomizedMp3Decoder(); + ShadowMediaCodec.addDecoder( + DECODER_NAME, + new ShadowMediaCodec.CodecConfig( + CODEC_INPUT_BUFFER_SIZE, CODEC_OUTPUT_BUFFER_SIZE, mp3Decoder)); + + MediaFormat mp3Format = new MediaFormat(); + mp3Format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_MPEG); + ShadowMediaCodecList.addCodec( + MediaCodecInfoBuilder.newBuilder() + .setName(DECODER_NAME) + .setCapabilities( + MediaCodecInfoBuilder.CodecCapabilitiesBuilder.newBuilder() + .setMediaFormat(mp3Format) + .build()) + .build()); + } + + @After + public void cleanUp() { + MediaCodecUtil.clearDecoderInfoCache(); + ShadowMediaCodecList.reset(); + ShadowMediaCodec.clearCodecs(); + } + + @Test + public void testPlayback_twoIdenticalMp3Files() throws Exception { + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(ApplicationProvider.getApplicationContext()) + .setClock(new AutoAdvancingFakeClock()) + .build(); + + player.setMediaItems( + ImmutableList.of( + MediaItem.fromUri("asset:///media/mp3/test.mp3"), + MediaItem.fromUri("asset:///media/mp3/test.mp3"))); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + + Format playerAudioFormat = player.getAudioFormat(); + assertThat(playerAudioFormat).isNotNull(); + + int bytesPerFrame = audioTrackListener.getAudioTrackOutputFormat().getFrameSizeInBytes(); + int paddingBytes = max(0, playerAudioFormat.encoderPadding) * bytesPerFrame; + int delayBytes = max(0, playerAudioFormat.encoderDelay) * bytesPerFrame; + assertThat(paddingBytes).isEqualTo(2808); + assertThat(delayBytes).isEqualTo(1152); + + byte[] decoderOutputBytes = Bytes.concat(mp3Decoder.getAllOutputBytes().toArray(new byte[0][])); + int bytesPerAudioFile = decoderOutputBytes.length / 2; + assertThat(bytesPerAudioFile).isEqualTo(92160); + + byte[] expectedTrimmedByteContent = + Bytes.concat( + // Track one is trimmed at its beginning and its end. + Arrays.copyOfRange(decoderOutputBytes, delayBytes, bytesPerAudioFile - paddingBytes), + // Track two is only trimmed at its beginning, but not its end. + Arrays.copyOfRange( + decoderOutputBytes, bytesPerAudioFile + delayBytes, decoderOutputBytes.length)); + + byte[] audioTrackReceivedBytes = audioTrackListener.getAllReceivedBytes(); + assertThat(audioTrackReceivedBytes).isEqualTo(expectedTrimmedByteContent); + } + + private static class AudioTrackListener implements ShadowAudioTrack.OnAudioDataWrittenListener { + private final ByteArrayOutputStream audioTrackReceivedBytesStream = new ByteArrayOutputStream(); + // Output format from the audioTrack. + private AudioFormat format; + private ShadowAudioTrack audioTrack; + + @Override + public synchronized void onAudioDataWritten( + ShadowAudioTrack audioTrack, byte[] audioData, AudioFormat format) { + if (this.audioTrack == null) { + this.audioTrack = audioTrack; + } else { + Assertions.checkArgument( + audioTrack == this.audioTrack, "Data written from a different AudioTrack"); + } + + if (!format.equals(this.format)) { + this.format = format; + } + audioTrackReceivedBytesStream.write(audioData, 0, audioData.length); + } + + public byte[] getAllReceivedBytes() { + return audioTrackReceivedBytesStream.toByteArray(); + } + + public AudioFormat getAudioTrackOutputFormat() { + return format; + } + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/FlacPlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/FlacPlaybackTest.java new file mode 100644 index 00000000000..cb2c945ec39 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/FlacPlaybackTest.java @@ -0,0 +1,84 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import androidx.test.core.app.ApplicationProvider; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameter; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; +import org.robolectric.annotation.Config; + +/** End-to-end tests using FLAC samples. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(ParameterizedRobolectricTestRunner.class) +public class FlacPlaybackTest { + + @Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of( + "bear.flac", + "bear_no_min_max_frame_size.flac", + "bear_no_num_samples.flac", + "bear_no_seek_table_no_num_samples.flac", + "bear_one_metadata_block.flac", + "bear_uncommon_sample_rate.flac", + "bear_with_id3.flac", + "bear_with_picture.flac", + "bear_with_vorbis_comments.flac"); + } + + @Parameter public String inputFile; + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/flac/" + inputFile)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/flac/" + inputFile + ".dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/FlvPlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/FlvPlaybackTest.java new file mode 100644 index 00000000000..598aae8249f --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/FlvPlaybackTest.java @@ -0,0 +1,76 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import android.graphics.SurfaceTexture; +import android.view.Surface; +import androidx.test.core.app.ApplicationProvider; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; +import org.robolectric.annotation.Config; + +/** End-to-end tests using FLV samples. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(ParameterizedRobolectricTestRunner.class) +public final class FlvPlaybackTest { + @Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of("sample.flv", "sample-with-key-frame-index.flv"); + } + + @ParameterizedRobolectricTestRunner.Parameter public String inputFile; + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + player.setVideoSurface(new Surface(new SurfaceTexture(/* texName= */ 1))); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/flv/" + inputFile)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/flv/" + inputFile + ".dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/MkaPlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/MkaPlaybackTest.java new file mode 100644 index 00000000000..78ff4299d1b --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/MkaPlaybackTest.java @@ -0,0 +1,77 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import androidx.test.core.app.ApplicationProvider; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; +import org.robolectric.annotation.Config; + +/** End-to-end tests using MKA samples. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(ParameterizedRobolectricTestRunner.class) +public final class MkaPlaybackTest { + @Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of( + "bear-flac-16bit.mka", + "bear-flac-24bit.mka", + "bear-opus.mka", + "bear-opus-negative-gain.mka"); + } + + @ParameterizedRobolectricTestRunner.Parameter public String inputFile; + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/mka/" + inputFile)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/mka/" + inputFile + ".dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/MkvPlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/MkvPlaybackTest.java new file mode 100644 index 00000000000..7aa9fff35d6 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/MkvPlaybackTest.java @@ -0,0 +1,82 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import android.graphics.SurfaceTexture; +import android.view.Surface; +import androidx.test.core.app.ApplicationProvider; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; +import org.robolectric.annotation.Config; + +/** End-to-end tests using MKV samples. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(ParameterizedRobolectricTestRunner.class) +public final class MkvPlaybackTest { + @Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of( + "sample.mkv", + "sample_with_htc_rotation_track_name.mkv", + "sample_with_ssa_subtitles.mkv", + "sample_with_null_terminated_ssa_subtitles.mkv", + "sample_with_srt.mkv", + "sample_with_null_terminated_srt.mkv"); + } + + @ParameterizedRobolectricTestRunner.Parameter public String inputFile; + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + player.setVideoSurface(new Surface(new SurfaceTexture(/* texName= */ 1))); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/mkv/" + inputFile)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/mkv/" + inputFile + ".dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Mp3PlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Mp3PlaybackTest.java new file mode 100644 index 00000000000..1b1b9a886b6 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Mp3PlaybackTest.java @@ -0,0 +1,80 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import androidx.test.core.app.ApplicationProvider; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; +import org.robolectric.annotation.Config; + +/** End-to-end tests using MP3 samples. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(ParameterizedRobolectricTestRunner.class) +public final class Mp3PlaybackTest { + @Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of( + "bear-cbr-constant-frame-size-no-seek-table.mp3", + "bear-cbr-variable-frame-size-no-seek-table.mp3", + "bear-id3.mp3", + "bear-vbr-no-seek-table.mp3", + "bear-vbr-xing-header.mp3", + "play-trimmed.mp3", + "test.mp3"); + } + + @ParameterizedRobolectricTestRunner.Parameter public String inputFile; + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/mp3/" + inputFile)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/mp3/" + inputFile + ".dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Mp4PlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Mp4PlaybackTest.java index 5fd7453beb8..492276b659e 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Mp4PlaybackTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Mp4PlaybackTest.java @@ -13,53 +13,87 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package com.google.android.exoplayer2.e2etest; +import android.content.Context; import android.graphics.SurfaceTexture; import android.view.Surface; import androidx.test.core.app.ApplicationProvider; -import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.SimpleExoPlayer; import com.google.android.exoplayer2.robolectric.PlaybackOutput; import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; import com.google.android.exoplayer2.testutil.DumpFileAsserts; -import com.google.android.exoplayer2.testutil.TestExoPlayer; +import com.google.common.collect.ImmutableList; import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameter; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; import org.robolectric.annotation.Config; /** End-to-end tests using MP4 samples. */ // TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. @Config(sdk = 29) -@RunWith(AndroidJUnit4.class) +@RunWith(ParameterizedRobolectricTestRunner.class) public class Mp4PlaybackTest { + + @Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of( + "midroll-5s.mp4", + "postroll-5s.mp4", + "preroll-5s.mp4", + "sample_ac3_fragmented.mp4", + "sample_ac3.mp4", + "sample_ac4_fragmented.mp4", + "sample_ac4.mp4", + "sample_android_slow_motion.mp4", + "sample_eac3_fragmented.mp4", + "sample_eac3.mp4", + "sample_eac3joc_fragmented.mp4", + "sample_eac3joc.mp4", + "sample_fragmented.mp4", + "sample_fragmented_seekable.mp4", + "sample_fragmented_sei.mp4", + "sample_mdat_too_long.mp4", + "sample.mp4", + "sample_opus_fragmented.mp4", + "sample_opus.mp4", + "sample_partially_fragmented.mp4", + "testvid_1022ms.mp4"); + } + + @Parameter public String inputFile; + @Rule public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.forAllSupportedMimeTypes(); @Test - public void h264VideoAacAudio() throws Exception { + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory renderersFactory = new CapturingRenderersFactory(applicationContext); SimpleExoPlayer player = - new SimpleExoPlayer.Builder(ApplicationProvider.getApplicationContext()) + new SimpleExoPlayer.Builder(applicationContext, renderersFactory) .setClock(new AutoAdvancingFakeClock()) .build(); player.setVideoSurface(new Surface(new SurfaceTexture(/* texName= */ 1))); - PlaybackOutput playbackOutput = PlaybackOutput.register(player, mediaCodecConfig); - player.setMediaItem(MediaItem.fromUri("asset:///media/mp4/sample.mp4")); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, renderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/mp4/" + inputFile)); player.prepare(); player.play(); - TestExoPlayer.runUntilPlaybackState(player, Player.STATE_ENDED); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); player.release(); DumpFileAsserts.assertOutput( - ApplicationProvider.getApplicationContext(), - playbackOutput, - "playbackdumps/mp4/sample.mp4.dump"); + applicationContext, playbackOutput, "playbackdumps/mp4/" + inputFile + ".dump"); } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/OggPlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/OggPlaybackTest.java new file mode 100644 index 00000000000..f95f5f00848 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/OggPlaybackTest.java @@ -0,0 +1,78 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import androidx.test.core.app.ApplicationProvider; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.annotation.Config; + +/** End-to-end tests using OGG samples. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(ParameterizedRobolectricTestRunner.class) +public final class OggPlaybackTest { + @ParameterizedRobolectricTestRunner.Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of( + "bear.opus", + "bear_flac.ogg", + "bear_flac_noseektable.ogg", + "bear_vorbis.ogg", + "bear_vorbis_gap.ogg", + "bear_vorbis_with_large_metadata.ogg"); + } + + @ParameterizedRobolectricTestRunner.Parameter public String inputFile; + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/ogg/" + inputFile)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/ogg/" + inputFile + ".dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/PlaylistPlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/PlaylistPlaybackTest.java new file mode 100644 index 00000000000..3e6d1cfb125 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/PlaylistPlaybackTest.java @@ -0,0 +1,88 @@ +/* + * Copyright (C) 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; + +/** End-to-end tests for playlists. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(AndroidJUnit4.class) +public final class PlaylistPlaybackTest { + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test_bypassOnThenOn() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.addMediaItem(MediaItem.fromUri("asset:///media/wav/sample.wav")); + player.addMediaItem(MediaItem.fromUri("asset:///media/mka/bear-opus.mka")); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/playlists/bypass-on-then-off.dump"); + } + + @Test + public void test_bypassOffThenOn() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.addMediaItem(MediaItem.fromUri("asset:///media/mka/bear-opus.mka")); + player.addMediaItem(MediaItem.fromUri("asset:///media/wav/sample.wav")); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/playlists/bypass-off-then-on.dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/SilencePlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/SilencePlaybackTest.java new file mode 100644 index 00000000000..886b45aed63 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/SilencePlaybackTest.java @@ -0,0 +1,86 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.source.SilenceMediaSource; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; + +/** End-to-end tests using {@link SilenceMediaSource}. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(AndroidJUnit4.class) +public final class SilencePlaybackTest { + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test_500ms() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaSource(new SilenceMediaSource(/* durationUs= */ 500_000)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/silence/500ms.dump"); + } + + @Test + public void test_0ms() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaSource(new SilenceMediaSource(/* durationUs= */ 0)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/silence/0ms.dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/TsPlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/TsPlaybackTest.java index 52184f57510..0587a105bda 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/TsPlaybackTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/TsPlaybackTest.java @@ -15,51 +15,88 @@ */ package com.google.android.exoplayer2.e2etest; +import android.content.Context; import android.graphics.SurfaceTexture; import android.view.Surface; import androidx.test.core.app.ApplicationProvider; -import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.SimpleExoPlayer; import com.google.android.exoplayer2.robolectric.PlaybackOutput; import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; import com.google.android.exoplayer2.testutil.DumpFileAsserts; -import com.google.android.exoplayer2.testutil.TestExoPlayer; +import com.google.common.collect.ImmutableList; import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameter; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; import org.robolectric.annotation.Config; /** End-to-end tests using TS samples. */ // TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. @Config(sdk = 29) -@RunWith(AndroidJUnit4.class) +@RunWith(ParameterizedRobolectricTestRunner.class) public class TsPlaybackTest { + @Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of( + "bbb_2500ms.ts", + "elephants_dream.mpg", + "sample.ac3", + "sample_ac3.ts", + "sample.ac4", + "sample_ac4.ts", + "sample.eac3", + "sample_eac3.ts", + "sample_eac3joc.ec3", + "sample_eac3joc.ts", + "sample.adts", + "sample_ait.ts", + "sample_cbs_truncated.adts", + "sample_h262_mpeg_audio.ps", + "sample_h262_mpeg_audio.ts", + "sample_h263.ts", + "sample_h264_dts_audio.ts", + "sample_h264_mpeg_audio.ts", + "sample_h264_no_access_unit_delimiters.ts", + "sample_h265.ts", + "sample_latm.ts", + "sample_scte35.ts", + "sample_with_id3.adts", + "sample_with_junk"); + } + + @Parameter public String inputFile; + @Rule public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.forAllSupportedMimeTypes(); @Test - public void mpegVideoMpegAudioScte35() throws Exception { + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); SimpleExoPlayer player = - new SimpleExoPlayer.Builder(ApplicationProvider.getApplicationContext()) + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(new AutoAdvancingFakeClock()) .build(); player.setVideoSurface(new Surface(new SurfaceTexture(/* texName= */ 1))); - PlaybackOutput playbackOutput = PlaybackOutput.register(player, mediaCodecConfig); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); - player.setMediaItem(MediaItem.fromUri("asset:///media/ts/sample_scte35.ts")); + player.setMediaItem(MediaItem.fromUri("asset:///media/ts/" + inputFile)); player.prepare(); player.play(); - TestExoPlayer.runUntilPlaybackState(player, Player.STATE_ENDED); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); player.release(); DumpFileAsserts.assertOutput( - ApplicationProvider.getApplicationContext(), - playbackOutput, - "playbackdumps/ts/sample_scte35.ts.dump"); + applicationContext, playbackOutput, "playbackdumps/ts/" + inputFile + ".dump"); } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Vp9PlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Vp9PlaybackTest.java new file mode 100644 index 00000000000..1d0fc0cb4dd --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/Vp9PlaybackTest.java @@ -0,0 +1,80 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import android.graphics.SurfaceTexture; +import android.view.Surface; +import androidx.test.core.app.ApplicationProvider; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; +import org.robolectric.annotation.Config; + +/** End-to-end tests using VP9 samples. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(ParameterizedRobolectricTestRunner.class) +public final class Vp9PlaybackTest { + @Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of( + "bear-vp9-odd-dimensions.webm", + "bear-vp9.webm", + "invalid-bitstream.webm", + "roadtrip-vp92-10bit.webm"); + } + + @ParameterizedRobolectricTestRunner.Parameter public String inputFile; + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + player.setVideoSurface(new Surface(new SurfaceTexture(/* texName= */ 1))); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/vp9/" + inputFile)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + applicationContext, playbackOutput, "playbackdumps/vp9/" + inputFile + ".dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/e2etest/WavPlaybackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/WavPlaybackTest.java new file mode 100644 index 00000000000..99418204b68 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/e2etest/WavPlaybackTest.java @@ -0,0 +1,74 @@ +/* + * Copyright (C) 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.e2etest; + +import android.content.Context; +import androidx.test.core.app.ApplicationProvider; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.robolectric.PlaybackOutput; +import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.annotation.Config; + +/** End-to-end tests using WAV samples. */ +// TODO(b/143232359): Remove once https://issuetracker.google.com/143232359 is resolved. +@Config(sdk = 29) +@RunWith(ParameterizedRobolectricTestRunner.class) +public final class WavPlaybackTest { + @ParameterizedRobolectricTestRunner.Parameters(name = "{0}") + public static ImmutableList mediaSamples() { + return ImmutableList.of("sample.wav", "sample_ima_adpcm.wav", "sample_with_trailing_bytes.wav"); + } + + @ParameterizedRobolectricTestRunner.Parameter public String inputFile; + + @Rule + public ShadowMediaCodecConfig mediaCodecConfig = + ShadowMediaCodecConfig.forAllSupportedMimeTypes(); + + @Test + public void test() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); + SimpleExoPlayer player = + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(new AutoAdvancingFakeClock()) + .build(); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + + player.setMediaItem(MediaItem.fromUri("asset:///media/wav/" + inputFile)); + player.prepare(); + player.play(); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + ApplicationProvider.getApplicationContext(), + playbackOutput, + "playbackdumps/wav/" + inputFile + ".dump"); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecAdapterTest.java b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecAdapterTest.java index dc32ce65a14..48ecd2e5825 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecAdapterTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecAdapterTest.java @@ -24,7 +24,6 @@ import android.media.MediaFormat; import android.os.HandlerThread; import androidx.test.ext.junit.runners.AndroidJUnit4; -import com.google.android.exoplayer2.C; import java.io.IOException; import java.lang.reflect.Constructor; import org.junit.After; @@ -38,36 +37,37 @@ public class AsynchronousMediaCodecAdapterTest { private AsynchronousMediaCodecAdapter adapter; private MediaCodec codec; - private TestHandlerThread handlerThread; + private HandlerThread callbackThread; + private HandlerThread queueingThread; private MediaCodec.BufferInfo bufferInfo; @Before public void setUp() throws IOException { codec = MediaCodec.createByCodecName("h264"); - handlerThread = new TestHandlerThread("TestHandlerThread"); + callbackThread = new HandlerThread("TestCallbackThread"); + queueingThread = new HandlerThread("TestQueueingThread"); adapter = - new AsynchronousMediaCodecAdapter( - codec, - /* enableAsynchronousQueueing= */ false, - /* trackType= */ C.TRACK_TYPE_VIDEO, - handlerThread); + new AsynchronousMediaCodecAdapter.Factory( + /* callbackThreadSupplier= */ () -> callbackThread, + /* queueingThreadSupplier= */ () -> queueingThread, + /* forceQueueingSynchronizationWorkaround= */ false, + /* synchronizeCodecInteractionsWithQueueing= */ false) + .createAdapter(codec); bufferInfo = new MediaCodec.BufferInfo(); } @After public void tearDown() { - adapter.shutdown(); - - assertThat(handlerThread.hasQuit()).isTrue(); + adapter.release(); } @Test public void dequeueInputBufferIndex_withoutInputBuffer_returnsTryAgainLater() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); // After adapter.start(), the ShadowMediaCodec offers one input buffer. We pause the looper so // that the buffer is not propagated to the adapter. - shadowOf(handlerThread.getLooper()).pause(); + shadowOf(callbackThread.getLooper()).pause(); adapter.start(); assertThat(adapter.dequeueInputBufferIndex()).isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); @@ -76,59 +76,27 @@ public void dequeueInputBufferIndex_withoutInputBuffer_returnsTryAgainLater() { @Test public void dequeueInputBufferIndex_withInputBuffer_returnsInputBuffer() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); adapter.start(); // After start(), the ShadowMediaCodec offers input buffer 0. We advance the looper to make sure // and messages have been propagated to the adapter. - shadowOf(handlerThread.getLooper()).idle(); + shadowOf(callbackThread.getLooper()).idle(); assertThat(adapter.dequeueInputBufferIndex()).isEqualTo(0); } - @Test - public void dequeueInputBufferIndex_withPendingFlush_returnsTryAgainLater() { - adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); - adapter.start(); - // After adapter.start(), the ShadowMediaCodec offers input buffer 0. We run all currently - // enqueued messages and pause the looper so that flush is not completed. - ShadowLooper shadowLooper = shadowOf(handlerThread.getLooper()); - shadowLooper.idle(); - shadowLooper.pause(); - adapter.flush(); - - assertThat(adapter.dequeueInputBufferIndex()).isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); - } - - @Test - public void dequeueInputBufferIndex_withFlushCompletedAndInputBuffer_returnsInputBuffer() { - adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); - adapter.start(); - // After adapter.start(), the ShadowMediaCodec offers input buffer 0. We advance the looper to - // make sure all messages have been propagated to the adapter. - ShadowLooper shadowLooper = shadowOf(handlerThread.getLooper()); - shadowLooper.idle(); - - adapter.flush(); - // Progress the looper to complete flush(): the adapter should call codec.start(), triggering - // the ShadowMediaCodec to offer input buffer 0. - shadowLooper.idle(); - - assertThat(adapter.dequeueInputBufferIndex()).isEqualTo(0); - } @Test public void dequeueInputBufferIndex_withMediaCodecError_throwsException() throws Exception { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); // Pause the looper so that we interact with the adapter from this thread only. - shadowOf(handlerThread.getLooper()).pause(); + shadowOf(callbackThread.getLooper()).pause(); adapter.start(); // Set an error directly on the adapter (not through the looper). - adapter.onError(codec, createCodecException()); + adapter.onError(createCodecException()); assertThrows(IllegalStateException.class, () -> adapter.dequeueInputBufferIndex()); } @@ -136,14 +104,14 @@ public void dequeueInputBufferIndex_withMediaCodecError_throwsException() throws @Test public void dequeueInputBufferIndex_afterShutdown_returnsTryAgainLater() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); adapter.start(); // After start(), the ShadowMediaCodec offers input buffer 0, which is available only if we // progress the adapter's looper. We progress the looper so that we call shutdown() on a // non-empty adapter. - shadowOf(handlerThread.getLooper()).idle(); + shadowOf(callbackThread.getLooper()).idle(); - adapter.shutdown(); + adapter.release(); assertThat(adapter.dequeueInputBufferIndex()).isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); } @@ -151,12 +119,12 @@ public void dequeueInputBufferIndex_afterShutdown_returnsTryAgainLater() { @Test public void dequeueOutputBufferIndex_withoutOutputBuffer_returnsTryAgainLater() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); adapter.start(); // After start(), the ShadowMediaCodec offers an output format change. We progress the looper // so that the format change is propagated to the adapter. - shadowOf(handlerThread.getLooper()).idle(); + shadowOf(callbackThread.getLooper()).idle(); assertThat(adapter.dequeueOutputBufferIndex(bufferInfo)) .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); @@ -168,17 +136,21 @@ public void dequeueOutputBufferIndex_withoutOutputBuffer_returnsTryAgainLater() @Test public void dequeueOutputBufferIndex_withOutputBuffer_returnsOutputBuffer() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); adapter.start(); // After start(), the ShadowMediaCodec offers input buffer 0, which is available only if we // progress the adapter's looper. - ShadowLooper shadowLooper = shadowOf(handlerThread.getLooper()); - shadowLooper.idle(); + ShadowLooper callbackShadowLooper = shadowOf(callbackThread.getLooper()); + callbackShadowLooper.idle(); int index = adapter.dequeueInputBufferIndex(); adapter.queueInputBuffer(index, 0, 0, 0, 0); - // Progress the looper so that the ShadowMediaCodec processes the input buffer. - shadowLooper.idle(); + // Progress the queueuing looper first so the asynchronous enqueuer submits the input buffer, + // the ShadowMediaCodec processes the input buffer and produces an output buffer. Then, progress + // the callback looper so that the available output buffer callback is handled and the output + // buffer reaches the adapter. + shadowOf(queueingThread.getLooper()).idle(); + callbackShadowLooper.idle(); // The ShadowMediaCodec will first offer an output format and then the output buffer. assertThat(adapter.dequeueOutputBufferIndex(bufferInfo)) @@ -188,35 +160,16 @@ public void dequeueOutputBufferIndex_withOutputBuffer_returnsOutputBuffer() { assertThat(adapter.dequeueOutputBufferIndex(bufferInfo)).isEqualTo(index); } - @Test - public void dequeueOutputBufferIndex_withPendingFlush_returnsTryAgainLater() { - adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); - adapter.start(); - // After start(), the ShadowMediaCodec offers input buffer 0, which is available only if we - // progress the adapter's looper. - ShadowLooper shadowLooper = shadowOf(handlerThread.getLooper()); - shadowLooper.idle(); - - // Flush enqueues a task in the looper, but we will pause the looper to leave flush() - // in an incomplete state. - shadowLooper.pause(); - adapter.flush(); - - assertThat(adapter.dequeueOutputBufferIndex(bufferInfo)) - .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); - } - @Test public void dequeueOutputBufferIndex_withMediaCodecError_throwsException() throws Exception { // Pause the looper so that we interact with the adapter from this thread only. adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); - shadowOf(handlerThread.getLooper()).pause(); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + shadowOf(callbackThread.getLooper()).pause(); adapter.start(); // Set an error directly on the adapter. - adapter.onError(codec, createCodecException()); + adapter.onError(createCodecException()); assertThrows(IllegalStateException.class, () -> adapter.dequeueOutputBufferIndex(bufferInfo)); } @@ -224,18 +177,18 @@ public void dequeueOutputBufferIndex_withMediaCodecError_throwsException() throw @Test public void dequeueOutputBufferIndex_afterShutdown_returnsTryAgainLater() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); adapter.start(); // After start(), the ShadowMediaCodec offers input buffer 0, which is available only if we // progress the adapter's looper. - ShadowLooper shadowLooper = shadowOf(handlerThread.getLooper()); + ShadowLooper shadowLooper = shadowOf(callbackThread.getLooper()); shadowLooper.idle(); int index = adapter.dequeueInputBufferIndex(); adapter.queueInputBuffer(index, 0, 0, 0, 0); // Progress the looper so that the ShadowMediaCodec processes the input buffer. shadowLooper.idle(); - adapter.shutdown(); + adapter.release(); assertThat(adapter.dequeueOutputBufferIndex(bufferInfo)) .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); @@ -244,10 +197,10 @@ public void dequeueOutputBufferIndex_afterShutdown_returnsTryAgainLater() { @Test public void getOutputFormat_withoutFormatReceived_throwsException() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); // After start() the ShadowMediaCodec offers an output format change. Pause the looper so that // the format change is not propagated to the adapter. - shadowOf(handlerThread.getLooper()).pause(); + shadowOf(callbackThread.getLooper()).pause(); adapter.start(); assertThrows(IllegalStateException.class, () -> adapter.getOutputFormat()); @@ -256,14 +209,14 @@ public void getOutputFormat_withoutFormatReceived_throwsException() { @Test public void getOutputFormat_withMultipleFormats_returnsCorrectFormat() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); adapter.start(); // After start(), the ShadowMediaCodec offers an output format, which is available only if we // progress the adapter's looper. - shadowOf(handlerThread.getLooper()).idle(); + shadowOf(callbackThread.getLooper()).idle(); - // Add another format directly on the adapter. - adapter.onOutputFormatChanged(codec, createMediaFormat("format2")); + // Add another format on the adapter. + adapter.onOutputFormatChanged(createMediaFormat("format2")); assertThat(adapter.dequeueOutputBufferIndex(bufferInfo)) .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); @@ -280,11 +233,11 @@ public void getOutputFormat_withMultipleFormats_returnsCorrectFormat() { @Test public void getOutputFormat_afterFlush_returnsPreviousFormat() { adapter.configure( - createMediaFormat("foo"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + createMediaFormat("format"), /* surface= */ null, /* crypto= */ null, /* flags= */ 0); adapter.start(); // After start(), the ShadowMediaCodec offers an output format, which is available only if we // progress the adapter's looper. - ShadowLooper shadowLooper = shadowOf(handlerThread.getLooper()); + ShadowLooper shadowLooper = shadowOf(callbackThread.getLooper()); shadowLooper.idle(); adapter.dequeueOutputBufferIndex(bufferInfo); @@ -310,22 +263,4 @@ private static MediaCodec.CodecException createCodecException() throws Exception return constructor.newInstance( /* errorCode= */ 0, /* actionCode= */ 0, /* detailMessage= */ "error from codec"); } - - private static class TestHandlerThread extends HandlerThread { - private boolean quit; - - TestHandlerThread(String label) { - super(label); - } - - public boolean hasQuit() { - return quit; - } - - @Override - public boolean quit() { - quit = true; - return super.quit(); - } - } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecBufferEnqueuerTest.java b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecBufferEnqueuerTest.java index 37d31569c35..e5fdd126f41 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecBufferEnqueuerTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecBufferEnqueuerTest.java @@ -19,6 +19,7 @@ import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; import static org.mockito.Mockito.doAnswer; +import static org.robolectric.Shadows.shadowOf; import android.media.MediaCodec; import android.media.MediaFormat; @@ -28,7 +29,7 @@ import com.google.android.exoplayer2.decoder.CryptoInfo; import com.google.android.exoplayer2.util.ConditionVariable; import java.io.IOException; -import java.util.concurrent.atomic.AtomicLong; +import java.nio.ByteBuffer; import org.junit.After; import org.junit.Before; import org.junit.Rule; @@ -55,7 +56,11 @@ public void setUp() throws IOException { codec.start(); handlerThread = new TestHandlerThread("TestHandlerThread"); enqueuer = - new AsynchronousMediaCodecBufferEnqueuer(codec, handlerThread, mockConditionVariable); + new AsynchronousMediaCodecBufferEnqueuer( + codec, + handlerThread, + /* forceQueueingSynchronizationWorkaround= */ false, + mockConditionVariable); } @After @@ -63,7 +68,34 @@ public void tearDown() { enqueuer.shutdown(); codec.stop(); codec.release(); - assertThat(TestHandlerThread.INSTANCES_STARTED.get()).isEqualTo(0); + assertThat(!handlerThread.hasStarted() || handlerThread.hasQuit()).isTrue(); + } + + @Test + public void queueInputBuffer_queuesInputBufferOnMediaCodec() { + enqueuer.start(); + int inputBufferIndex = codec.dequeueInputBuffer(0); + assertThat(inputBufferIndex).isAtLeast(0); + byte[] inputData = new byte[] {0, 1, 2, 3}; + codec.getInputBuffer(inputBufferIndex).put(inputData); + + enqueuer.queueInputBuffer( + inputBufferIndex, + /* offset= */ 0, + /* size= */ 4, + /* presentationTimeUs= */ 0, + /* flags= */ 0); + shadowOf(handlerThread.getLooper()).idle(); + + MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); + assertThat(codec.dequeueOutputBuffer(bufferInfo, 0)) + .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + assertThat(codec.dequeueOutputBuffer(bufferInfo, 0)).isEqualTo(inputBufferIndex); + ByteBuffer outputBuffer = codec.getOutputBuffer(inputBufferIndex); + assertThat(outputBuffer.limit()).isEqualTo(4); + byte[] outputData = new byte[4]; + outputBuffer.get(outputData); + assertThat(outputData).isEqualTo(inputData); } @Test @@ -111,7 +143,7 @@ public void queueSecureInputBuffer_withPendingCryptoException_throwsCryptoExcept enqueuer.queueSecureInputBuffer( /* index= */ 0, /* offset= */ 0, - /* info= */ info, + info, /* presentationTimeUs= */ 0, /* flags= */ 0)); } @@ -191,29 +223,6 @@ public void shutdown_onInterruptedException_throwsIllegalStateException() assertThrows(IllegalStateException.class, () -> enqueuer.shutdown()); } - private static class TestHandlerThread extends HandlerThread { - private static final AtomicLong INSTANCES_STARTED = new AtomicLong(0); - - TestHandlerThread(String name) { - super(name); - } - - @Override - public synchronized void start() { - super.start(); - INSTANCES_STARTED.incrementAndGet(); - } - - @Override - public boolean quit() { - boolean quit = super.quit(); - if (quit) { - INSTANCES_STARTED.decrementAndGet(); - } - return quit; - } - } - private static CryptoInfo createCryptoInfo() { CryptoInfo info = new CryptoInfo(); int numSubSamples = 5; @@ -235,4 +244,33 @@ private static CryptoInfo createCryptoInfo() { clearBlocks); return info; } + + private static class TestHandlerThread extends HandlerThread { + private boolean started; + private boolean quit; + + TestHandlerThread(String label) { + super(label); + } + + public boolean hasStarted() { + return started; + } + + public boolean hasQuit() { + return quit; + } + + @Override + public synchronized void start() { + super.start(); + started = true; + } + + @Override + public boolean quit() { + quit = true; + return super.quit(); + } + } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecCallbackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecCallbackTest.java new file mode 100644 index 00000000000..fbb031a64b6 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/AsynchronousMediaCodecCallbackTest.java @@ -0,0 +1,493 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.mediacodec; + +import static com.google.android.exoplayer2.testutil.TestUtil.assertBufferInfosEqual; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; +import static org.junit.Assert.fail; +import static org.robolectric.Shadows.shadowOf; + +import android.media.MediaCodec; +import android.media.MediaFormat; +import android.os.HandlerThread; +import android.os.Looper; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.IOException; +import java.lang.reflect.Constructor; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.shadows.ShadowLooper; + +/** Unit tests for {@link AsynchronousMediaCodecCallback}. */ +@RunWith(AndroidJUnit4.class) +public class AsynchronousMediaCodecCallbackTest { + + private AsynchronousMediaCodecCallback asynchronousMediaCodecCallback; + private TestHandlerThread callbackThread; + private MediaCodec codec; + + @Before + public void setUp() throws IOException { + callbackThread = new TestHandlerThread("TestCallbackThread"); + codec = MediaCodec.createByCodecName("h264"); + asynchronousMediaCodecCallback = new AsynchronousMediaCodecCallback(callbackThread); + asynchronousMediaCodecCallback.initialize(codec); + } + + @After + public void tearDown() { + codec.release(); + asynchronousMediaCodecCallback.shutdown(); + + assertThat(callbackThread.hasQuit()).isTrue(); + } + + @Test + public void dequeInputBufferIndex_afterCreation_returnsTryAgain() { + assertThat(asynchronousMediaCodecCallback.dequeueInputBufferIndex()) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeInputBufferIndex_returnsEnqueuedBuffers() { + // Send two input buffers to the callback. + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 0); + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 1); + + assertThat(asynchronousMediaCodecCallback.dequeueInputBufferIndex()).isEqualTo(0); + assertThat(asynchronousMediaCodecCallback.dequeueInputBufferIndex()).isEqualTo(1); + assertThat(asynchronousMediaCodecCallback.dequeueInputBufferIndex()) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeInputBufferIndex_withPendingFlush_returnsTryAgain() { + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + // Pause the callback thread so that flush() never completes. + shadowOf(callbackThreadLooper).pause(); + + // Send two input buffers to the callback and then flush(). + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 0); + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 1); + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + assertThat(flushCompleted.get()).isFalse(); + assertThat(asynchronousMediaCodecCallback.dequeueInputBufferIndex()) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeInputBufferIndex_afterFlush_returnsTryAgain() { + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + + // Send two input buffers to the callback and then flush(). + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 0); + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 1); + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + // Progress the callback thread so that flush() completes. + shadowOf(callbackThreadLooper).idle(); + + assertThat(flushCompleted.get()).isTrue(); + assertThat(asynchronousMediaCodecCallback.dequeueInputBufferIndex()) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeInputBufferIndex_afterFlushAndNewInputBuffer_returnsEnqueuedBuffer() { + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + + // Send two input buffers to the callback, then flush(), then send + // another input buffer. + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 0); + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 1); + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + // Progress the callback thread so that flush() completes. + shadowOf(callbackThreadLooper).idle(); + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, 2); + + assertThat(flushCompleted.get()).isTrue(); + assertThat(asynchronousMediaCodecCallback.dequeueInputBufferIndex()).isEqualTo(2); + } + + @Test + public void dequeueInputBufferIndex_afterShutdown_returnsTryAgainLater() { + asynchronousMediaCodecCallback.onInputBufferAvailable(codec, /* index= */ 1); + + asynchronousMediaCodecCallback.shutdown(); + + assertThat(asynchronousMediaCodecCallback.dequeueInputBufferIndex()) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeueInputBufferIndex_afterOnErrorCallback_throwsError() throws Exception { + asynchronousMediaCodecCallback.onError(codec, createCodecException()); + + assertThrows( + MediaCodec.CodecException.class, + () -> asynchronousMediaCodecCallback.dequeueInputBufferIndex()); + } + + @Test + public void dequeueInputBufferIndex_afterFlushCompletedWithError_throwsError() throws Exception { + MediaCodec.CodecException codecException = createCodecException(); + asynchronousMediaCodecCallback.flushAsync( + () -> { + throw codecException; + }); + shadowOf(callbackThread.getLooper()).idle(); + + assertThrows( + MediaCodec.CodecException.class, + () -> asynchronousMediaCodecCallback.dequeueInputBufferIndex()); + } + + @Test + public void dequeOutputBufferIndex_afterCreation_returnsTryAgain() { + MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeOutputBufferIndex_returnsEnqueuedBuffers() { + // Send an output format and two output buffers to the callback. + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, createMediaFormat("format0")); + MediaCodec.BufferInfo bufferInfo1 = new MediaCodec.BufferInfo(); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 0, bufferInfo1); + MediaCodec.BufferInfo bufferInfo2 = new MediaCodec.BufferInfo(); + bufferInfo2.set(1, 1, 1, 1); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 1, bufferInfo2); + + MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)) + .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + assertThat(asynchronousMediaCodecCallback.getOutputFormat().getString("name")) + .isEqualTo("format0"); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(0); + assertBufferInfosEqual(bufferInfo1, outBufferInfo); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(1); + assertBufferInfosEqual(bufferInfo2, outBufferInfo); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeOutputBufferIndex_withPendingFlush_returnsTryAgain() { + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + // Pause the callback thread so that flush() never completes. + shadowOf(callbackThreadLooper).pause(); + + // Send two output buffers to the callback and then flush(). + MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 0, bufferInfo); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 1, bufferInfo); + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + + assertThat(flushCompleted.get()).isFalse(); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(new MediaCodec.BufferInfo())) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeOutputBufferIndex_afterFlush_returnsTryAgain() { + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + + // Send two output buffers to the callback and then flush(). + MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 0, bufferInfo); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 1, bufferInfo); + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + // Progress the callback looper so that flush() completes. + shadowOf(callbackThreadLooper).idle(); + + assertThat(flushCompleted.get()).isTrue(); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(new MediaCodec.BufferInfo())) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeOutputBufferIndex_afterFlushAndNewOutputBuffers_returnsEnqueueBuffer() { + MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + + // Send an output format and two output buffers to the callback, then flush(), then send + // another output buffer. + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, createMediaFormat("format0")); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 0, bufferInfo); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 1, bufferInfo); + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + // Progress the callback looper so that flush() completes. + shadowOf(callbackThreadLooper).idle(); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, 2, bufferInfo); + MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); + + assertThat(flushCompleted.get()).isTrue(); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)) + .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + assertThat(asynchronousMediaCodecCallback.getOutputFormat().getString("name")) + .isEqualTo("format0"); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(2); + } + + @Test + public void dequeOutputBufferIndex_withPendingOutputFormat_returnsPendingOutputFormat() { + MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, new MediaFormat()); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, /* index= */ 0, outBufferInfo); + MediaFormat pendingMediaFormat = new MediaFormat(); + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, pendingMediaFormat); + // flush() should not discard the last format. + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + // Progress the callback looper so that flush() completes. + shadowOf(callbackThreadLooper).idle(); + // Right after flush(), we send an output buffer: the pending output format should be + // dequeued first. + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, /* index= */ 1, outBufferInfo); + + assertThat(flushCompleted.get()).isTrue(); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)) + .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + assertThat(asynchronousMediaCodecCallback.getOutputFormat()).isEqualTo(pendingMediaFormat); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(1); + } + + @Test + public void dequeOutputBufferIndex_withPendingOutputFormatAndNewFormat_returnsNewFormat() { + MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, new MediaFormat()); + MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, /* index= */ 0, bufferInfo); + MediaFormat pendingMediaFormat = new MediaFormat(); + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, pendingMediaFormat); + // flush() should not discard the last format. + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + // Progress the callback looper so that flush() completes. + shadowOf(callbackThreadLooper).idle(); + // The first callback after flush() is a new MediaFormat, it should overwrite the pending + // format. + MediaFormat newFormat = new MediaFormat(); + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, newFormat); + asynchronousMediaCodecCallback.onOutputBufferAvailable(codec, /* index= */ 1, bufferInfo); + + assertThat(flushCompleted.get()).isTrue(); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)) + .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + assertThat(asynchronousMediaCodecCallback.getOutputFormat()).isEqualTo(newFormat); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(1); + } + + @Test + public void dequeueOutputBufferIndex_afterShutdown_returnsTryAgainLater() { + asynchronousMediaCodecCallback.onOutputBufferAvailable( + codec, /* index= */ 1, new MediaCodec.BufferInfo()); + + asynchronousMediaCodecCallback.shutdown(); + + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(new MediaCodec.BufferInfo())) + .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); + } + + @Test + public void dequeueOutputBufferIndex_afterOnErrorCallback_throwsError() throws Exception { + asynchronousMediaCodecCallback.onError(codec, createCodecException()); + + assertThrows( + MediaCodec.CodecException.class, + () -> asynchronousMediaCodecCallback.dequeueOutputBufferIndex(new MediaCodec.BufferInfo())); + } + + @Test + public void dequeueOutputBufferIndex_afterFlushCompletedWithError_throwsError() throws Exception { + MediaCodec.CodecException codecException = createCodecException(); + asynchronousMediaCodecCallback.flushAsync( + () -> { + throw codecException; + }); + shadowOf(callbackThread.getLooper()).idle(); + + assertThrows( + MediaCodec.CodecException.class, + () -> asynchronousMediaCodecCallback.dequeueOutputBufferIndex(new MediaCodec.BufferInfo())); + } + + @Test + public void getOutputFormat_onNewInstance_raisesException() { + try { + asynchronousMediaCodecCallback.getOutputFormat(); + fail(); + } catch (IllegalStateException expected) { + } + } + + @Test + public void getOutputFormat_afterOnOutputFormatCalled_returnsFormat() { + MediaFormat format = new MediaFormat(); + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, format); + MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); + + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(bufferInfo)) + .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + assertThat(asynchronousMediaCodecCallback.getOutputFormat()).isEqualTo(format); + } + + @Test + public void getOutputFormat_afterFlush_returnsCurrentFormat() { + MediaFormat format = new MediaFormat(); + Looper callbackThreadLooper = callbackThread.getLooper(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, format); + asynchronousMediaCodecCallback.dequeueOutputBufferIndex(new MediaCodec.BufferInfo()); + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + // Progress the callback looper so that flush() completes. + shadowOf(callbackThreadLooper).idle(); + + assertThat(flushCompleted.get()).isTrue(); + assertThat(asynchronousMediaCodecCallback.getOutputFormat()).isEqualTo(format); + } + + @Test + public void getOutputFormat_afterFlushWithPendingFormat_returnsPendingFormat() { + MediaCodec.BufferInfo outInfo = new MediaCodec.BufferInfo(); + AtomicBoolean flushCompleted = new AtomicBoolean(); + ShadowLooper shadowCallbackLooper = shadowOf(callbackThread.getLooper()); + shadowCallbackLooper.pause(); + + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, createMediaFormat("format0")); + asynchronousMediaCodecCallback.onOutputBufferAvailable( + codec, /* index= */ 0, new MediaCodec.BufferInfo()); + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, createMediaFormat("format1")); + asynchronousMediaCodecCallback.onOutputBufferAvailable( + codec, /* index= */ 1, new MediaCodec.BufferInfo()); + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ () -> flushCompleted.set(true)); + // Progress the looper so that flush is completed + shadowCallbackLooper.idle(); + // Enqueue an output buffer to make the pending format available. + asynchronousMediaCodecCallback.onOutputBufferAvailable( + codec, /* index= */ 2, new MediaCodec.BufferInfo()); + + assertThat(flushCompleted.get()).isTrue(); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outInfo)) + .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + assertThat(asynchronousMediaCodecCallback.getOutputFormat().getString("name")) + .isEqualTo("format1"); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outInfo)).isEqualTo(2); + } + + @Test + public void + getOutputFormat_withConsecutiveFlushAndPendingFormatFromFirstFlush_returnsPendingFormat() { + MediaCodec.BufferInfo outInfo = new MediaCodec.BufferInfo(); + AtomicInteger flushesCompleted = new AtomicInteger(); + ShadowLooper shadowCallbackLooper = shadowOf(callbackThread.getLooper()); + shadowCallbackLooper.pause(); + + asynchronousMediaCodecCallback.onOutputFormatChanged(codec, createMediaFormat("format0")); + asynchronousMediaCodecCallback.onOutputBufferAvailable( + codec, /* index= */ 0, new MediaCodec.BufferInfo()); + // Flush and progress the looper so that flush is completed. + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ flushesCompleted::incrementAndGet); + shadowCallbackLooper.idle(); + // Flush again, the pending format from the first flush should remain as pending. + asynchronousMediaCodecCallback.flushAsync( + /* onFlushCompleted= */ flushesCompleted::incrementAndGet); + shadowCallbackLooper.idle(); + asynchronousMediaCodecCallback.onOutputBufferAvailable( + codec, /* index= */ 1, new MediaCodec.BufferInfo()); + + assertThat(flushesCompleted.get()).isEqualTo(2); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outInfo)) + .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); + assertThat(asynchronousMediaCodecCallback.getOutputFormat().getString("name")) + .isEqualTo("format0"); + assertThat(asynchronousMediaCodecCallback.dequeueOutputBufferIndex(outInfo)).isEqualTo(1); + } + + @Test + public void flush_withPendingFlush_onlyLastFlushCompletes() { + ShadowLooper callbackLooperShadow = shadowOf(callbackThread.getLooper()); + callbackLooperShadow.pause(); + AtomicInteger flushCompleted = new AtomicInteger(); + + asynchronousMediaCodecCallback.flushAsync(/* onFlushCompleted= */ () -> flushCompleted.set(1)); + asynchronousMediaCodecCallback.flushAsync(/* onFlushCompleted= */ () -> flushCompleted.set(2)); + callbackLooperShadow.idle(); + + assertThat(flushCompleted.get()).isEqualTo(2); + } + + /** Reflectively create a {@link MediaCodec.CodecException}. */ + private static MediaCodec.CodecException createCodecException() throws Exception { + Constructor constructor = + MediaCodec.CodecException.class.getDeclaredConstructor( + Integer.TYPE, Integer.TYPE, String.class); + return constructor.newInstance( + /* errorCode= */ 0, /* actionCode= */ 0, /* detailMessage= */ "error from codec"); + } + + private static MediaFormat createMediaFormat(String name) { + MediaFormat format = new MediaFormat(); + format.setString("name", name); + return format; + } + + private static class TestHandlerThread extends HandlerThread { + private boolean quit; + + TestHandlerThread(String label) { + super(label); + } + + public boolean hasQuit() { + return quit; + } + + @Override + public boolean quit() { + quit = true; + return super.quit(); + } + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/BatchBufferTest.java b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/BatchBufferTest.java index 6579e8ee06e..b488403a68c 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/BatchBufferTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/BatchBufferTest.java @@ -16,14 +16,14 @@ package com.google.android.exoplayer2.mediacodec; +import static com.google.android.exoplayer2.decoder.DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT; +import static com.google.android.exoplayer2.mediacodec.BatchBuffer.DEFAULT_MAX_SAMPLE_COUNT; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.decoder.DecoderInputBuffer; -import com.google.android.exoplayer2.testutil.TestUtil; -import com.google.common.primitives.Bytes; import java.nio.ByteBuffer; import org.junit.Test; import org.junit.runner.RunWith; @@ -32,220 +32,235 @@ @RunWith(AndroidJUnit4.class) public final class BatchBufferTest { - /** Bigger than {@code BatchBuffer.BATCH_SIZE_BYTES} */ - private static final int BUFFER_SIZE_LARGER_THAN_BATCH_SIZE_BYTES = 6 * 1000 * 1024; - /** Smaller than {@code BatchBuffer.BATCH_SIZE_BYTES} */ - private static final int BUFFER_SIZE_MUCH_SMALLER_THAN_BATCH_SIZE_BYTES = 100; - - private static final byte[] TEST_ACCESS_UNIT = - TestUtil.buildTestData(BUFFER_SIZE_MUCH_SMALLER_THAN_BATCH_SIZE_BYTES); - + private final DecoderInputBuffer sampleBuffer = + new DecoderInputBuffer(BUFFER_REPLACEMENT_MODE_DIRECT); private final BatchBuffer batchBuffer = new BatchBuffer(); @Test public void newBatchBuffer_isEmpty() { - assertIsCleared(batchBuffer); + assertThat(batchBuffer.getSampleCount()).isEqualTo(0); + assertThat(batchBuffer.hasSamples()).isFalse(); } @Test - public void clear_empty_isEmpty() { - batchBuffer.clear(); + public void appendSample() { + initSampleBuffer(); + batchBuffer.append(sampleBuffer); - assertIsCleared(batchBuffer); + assertThat(batchBuffer.getSampleCount()).isEqualTo(1); + assertThat(batchBuffer.hasSamples()).isTrue(); } @Test - public void clear_afterInsertingAccessUnit_isEmpty() { - batchBuffer.commitNextAccessUnit(); - + public void appendSample_thenClear_isEmpty() { + initSampleBuffer(); + batchBuffer.append(sampleBuffer); batchBuffer.clear(); - assertIsCleared(batchBuffer); + assertThat(batchBuffer.getSampleCount()).isEqualTo(0); + assertThat(batchBuffer.hasSamples()).isFalse(); } @Test - public void commitNextAccessUnit_addsAccessUnit() { - batchBuffer.commitNextAccessUnit(); + public void appendSample_updatesTimes() { + initSampleBuffer(/* timeUs= */ 1234); + batchBuffer.append(sampleBuffer); - assertThat(batchBuffer.getAccessUnitCount()).isEqualTo(1); + initSampleBuffer(/* timeUs= */ 5678); + batchBuffer.append(sampleBuffer); + + assertThat(batchBuffer.timeUs).isEqualTo(1234); + assertThat(batchBuffer.getFirstSampleTimeUs()).isEqualTo(1234); + assertThat(batchBuffer.getLastSampleTimeUs()).isEqualTo(5678); } @Test - public void commitNextAccessUnit_untilFull_isFullAndNotEmpty() { - fillBatchBuffer(batchBuffer); + public void appendSample_succeedsUntilDefaultMaxSampleCountReached_thenFails() { + for (int i = 0; i < DEFAULT_MAX_SAMPLE_COUNT; i++) { + initSampleBuffer(/* timeUs= */ i); + assertThat(batchBuffer.append(sampleBuffer)).isTrue(); + assertThat(batchBuffer.getSampleCount()).isEqualTo(i + 1); + } - assertThat(batchBuffer.isEmpty()).isFalse(); - assertThat(batchBuffer.isFull()).isTrue(); + initSampleBuffer(/* timeUs= */ DEFAULT_MAX_SAMPLE_COUNT); + assertThat(batchBuffer.append(sampleBuffer)).isFalse(); + assertThat(batchBuffer.getSampleCount()).isEqualTo(DEFAULT_MAX_SAMPLE_COUNT); + assertThat(batchBuffer.getLastSampleTimeUs()).isEqualTo(DEFAULT_MAX_SAMPLE_COUNT - 1); } @Test - public void commitNextAccessUnit_whenFull_throws() { - batchBuffer.setMaxAccessUnitCount(1); - batchBuffer.commitNextAccessUnit(); + public void appendSample_succeedsUntilCustomMaxSampleCountReached_thenFails() { + int customMaxSampleCount = DEFAULT_MAX_SAMPLE_COUNT * 2; + batchBuffer.setMaxSampleCount(customMaxSampleCount); + for (int i = 0; i < customMaxSampleCount; i++) { + initSampleBuffer(/* timeUs= */ i); + assertThat(batchBuffer.append(sampleBuffer)).isTrue(); + assertThat(batchBuffer.getSampleCount()).isEqualTo(i + 1); + } - assertThrows(IllegalStateException.class, batchBuffer::commitNextAccessUnit); + initSampleBuffer(/* timeUs= */ customMaxSampleCount); + assertThat(batchBuffer.append(sampleBuffer)).isFalse(); + assertThat(batchBuffer.getSampleCount()).isEqualTo(customMaxSampleCount); + assertThat(batchBuffer.getLastSampleTimeUs()).isEqualTo(customMaxSampleCount - 1); } @Test - public void commitNextAccessUnit_whenAccessUnitIsDecodeOnly_isDecodeOnly() { - batchBuffer.getNextAccessUnitBuffer().setFlags(C.BUFFER_FLAG_DECODE_ONLY); - - batchBuffer.commitNextAccessUnit(); + public void appendFirstSample_withDecodeOnlyFlag_setsDecodeOnlyFlag() { + initSampleBuffer(); + sampleBuffer.setFlags(C.BUFFER_FLAG_DECODE_ONLY); + batchBuffer.append(sampleBuffer); assertThat(batchBuffer.isDecodeOnly()).isTrue(); } @Test - public void commitNextAccessUnit_whenAccessUnitIsEndOfStream_isEndOfSteam() { - batchBuffer.getNextAccessUnitBuffer().setFlags(C.BUFFER_FLAG_END_OF_STREAM); + public void appendSecondSample_toDecodeOnlyBuffer_withDecodeOnlyFlag_succeeds() { + initSampleBuffer(); + sampleBuffer.setFlags(C.BUFFER_FLAG_DECODE_ONLY); + batchBuffer.append(sampleBuffer); - batchBuffer.commitNextAccessUnit(); + initSampleBuffer(); + sampleBuffer.setFlags(C.BUFFER_FLAG_DECODE_ONLY); - assertThat(batchBuffer.isEndOfStream()).isTrue(); + assertThat(batchBuffer.append(sampleBuffer)).isTrue(); } @Test - public void commitNextAccessUnit_whenAccessUnitIsKeyFrame_isKeyFrame() { - batchBuffer.getNextAccessUnitBuffer().setFlags(C.BUFFER_FLAG_KEY_FRAME); + public void appendSecondSample_toDecodeOnlyBuffer_withoutDecodeOnlyFlag_fails() { + initSampleBuffer(); + sampleBuffer.setFlags(C.BUFFER_FLAG_DECODE_ONLY); + batchBuffer.append(sampleBuffer); - batchBuffer.commitNextAccessUnit(); + initSampleBuffer(); - assertThat(batchBuffer.isKeyFrame()).isTrue(); + assertThat(batchBuffer.append(sampleBuffer)).isFalse(); } @Test - public void commitNextAccessUnit_withData_dataIsCopiedInTheBatch() { - batchBuffer.getNextAccessUnitBuffer().ensureSpaceForWrite(TEST_ACCESS_UNIT.length); - batchBuffer.getNextAccessUnitBuffer().data.put(TEST_ACCESS_UNIT); + public void appendSecondSample_toNonDecodeOnlyBuffer_withDecodeOnlyFlag_fails() { + initSampleBuffer(); + batchBuffer.append(sampleBuffer); - batchBuffer.commitNextAccessUnit(); - batchBuffer.flip(); + initSampleBuffer(); + sampleBuffer.setFlags(C.BUFFER_FLAG_DECODE_ONLY); - assertThat(batchBuffer.getAccessUnitCount()).isEqualTo(1); - assertThat(batchBuffer.data).isEqualTo(ByteBuffer.wrap(TEST_ACCESS_UNIT)); + assertThat(batchBuffer.append(sampleBuffer)).isFalse(); } @Test - public void commitNextAccessUnit_nextAccessUnit_isClear() { - batchBuffer.getNextAccessUnitBuffer().ensureSpaceForWrite(TEST_ACCESS_UNIT.length); - batchBuffer.getNextAccessUnitBuffer().data.put(TEST_ACCESS_UNIT); - batchBuffer.getNextAccessUnitBuffer().setFlags(C.BUFFER_FLAG_KEY_FRAME); - - batchBuffer.commitNextAccessUnit(); + public void appendSecondSample_withKeyframeFlag_setsKeyframeFlag() { + initSampleBuffer(); + sampleBuffer.setFlags(C.BUFFER_FLAG_KEY_FRAME); + batchBuffer.append(sampleBuffer); - DecoderInputBuffer nextAccessUnit = batchBuffer.getNextAccessUnitBuffer(); - assertThat(nextAccessUnit.data).isNotNull(); - assertThat(nextAccessUnit.data.position()).isEqualTo(0); - assertThat(nextAccessUnit.isKeyFrame()).isFalse(); + assertThat(batchBuffer.isKeyFrame()).isTrue(); } @Test - public void commitNextAccessUnit_twice_bothAccessUnitAreConcatenated() { - // Commit TEST_ACCESS_UNIT - batchBuffer.getNextAccessUnitBuffer().ensureSpaceForWrite(TEST_ACCESS_UNIT.length); - batchBuffer.getNextAccessUnitBuffer().data.put(TEST_ACCESS_UNIT); - batchBuffer.commitNextAccessUnit(); - // Commit TEST_ACCESS_UNIT again - batchBuffer.getNextAccessUnitBuffer().ensureSpaceForWrite(TEST_ACCESS_UNIT.length); - batchBuffer.getNextAccessUnitBuffer().data.put(TEST_ACCESS_UNIT); + public void appendSecondSample_withKeyframeFlag_doesNotSetKeyframeFlag() { + initSampleBuffer(); + batchBuffer.append(sampleBuffer); - batchBuffer.commitNextAccessUnit(); - batchBuffer.flip(); + initSampleBuffer(); + sampleBuffer.setFlags(C.BUFFER_FLAG_KEY_FRAME); + batchBuffer.append(sampleBuffer); - byte[] expected = Bytes.concat(TEST_ACCESS_UNIT, TEST_ACCESS_UNIT); - assertThat(batchBuffer.data).isEqualTo(ByteBuffer.wrap(expected)); + assertThat(batchBuffer.isKeyFrame()).isFalse(); } @Test - public void commitNextAccessUnit_whenAccessUnitIsHugeAndBatchBufferNotEmpty_isMarkedPending() { - batchBuffer.getNextAccessUnitBuffer().ensureSpaceForWrite(TEST_ACCESS_UNIT.length); - batchBuffer.getNextAccessUnitBuffer().data.put(TEST_ACCESS_UNIT); - batchBuffer.commitNextAccessUnit(); - byte[] hugeAccessUnit = TestUtil.buildTestData(BUFFER_SIZE_LARGER_THAN_BATCH_SIZE_BYTES); - batchBuffer.getNextAccessUnitBuffer().ensureSpaceForWrite(hugeAccessUnit.length); - batchBuffer.getNextAccessUnitBuffer().data.put(hugeAccessUnit); - batchBuffer.commitNextAccessUnit(); + public void appendSecondSample_doesNotClearKeyframeFlag() { + initSampleBuffer(); + sampleBuffer.setFlags(C.BUFFER_FLAG_KEY_FRAME); + batchBuffer.append(sampleBuffer); - batchBuffer.batchWasConsumed(); - batchBuffer.flip(); + initSampleBuffer(); + batchBuffer.append(sampleBuffer); - assertThat(batchBuffer.getAccessUnitCount()).isEqualTo(1); - assertThat(batchBuffer.data).isEqualTo(ByteBuffer.wrap(hugeAccessUnit)); + assertThat(batchBuffer.isKeyFrame()).isTrue(); } @Test - public void batchWasConsumed_whenNotEmpty_isEmpty() { - batchBuffer.commitNextAccessUnit(); - - batchBuffer.batchWasConsumed(); + public void appendSample_withEndOfStreamFlag_throws() { + sampleBuffer.setFlags(C.BUFFER_FLAG_END_OF_STREAM); - assertIsCleared(batchBuffer); + assertThrows(IllegalArgumentException.class, () -> batchBuffer.append(sampleBuffer)); } @Test - public void batchWasConsumed_whenFull_isEmpty() { - fillBatchBuffer(batchBuffer); + public void appendSample_withEncryptedFlag_throws() { + sampleBuffer.setFlags(C.BUFFER_FLAG_ENCRYPTED); - batchBuffer.batchWasConsumed(); - - assertIsCleared(batchBuffer); + assertThrows(IllegalArgumentException.class, () -> batchBuffer.append(sampleBuffer)); } @Test - public void getMaxAccessUnitCount_whenSetToAPositiveValue_returnsIt() { - batchBuffer.setMaxAccessUnitCount(20); + public void appendSample_withSupplementalDataFlag_throws() { + sampleBuffer.setFlags(C.BUFFER_FLAG_HAS_SUPPLEMENTAL_DATA); - assertThat(batchBuffer.getMaxAccessUnitCount()).isEqualTo(20); + assertThrows(IllegalArgumentException.class, () -> batchBuffer.append(sampleBuffer)); } @Test - public void setMaxAccessUnitCount_whenSetToNegative_throws() { - assertThrows(IllegalArgumentException.class, () -> batchBuffer.setMaxAccessUnitCount(-19)); + public void appendTwoSamples_batchesData() { + initSampleBuffer(/* timeUs= */ 1234); + batchBuffer.append(sampleBuffer); + initSampleBuffer(/* timeUs= */ 5678); + batchBuffer.append(sampleBuffer); + batchBuffer.flip(); + + ByteBuffer expected = ByteBuffer.allocate(Long.BYTES * 2); + expected.putLong(1234); + expected.putLong(5678); + expected.flip(); + + assertThat(batchBuffer.data).isEqualTo(expected); } @Test - public void setMaxAccessUnitCount_whenSetToZero_throws() { - assertThrows(IllegalArgumentException.class, () -> batchBuffer.setMaxAccessUnitCount(0)); + public void appendFirstSample_exceedingMaxSize_succeeds() { + sampleBuffer.ensureSpaceForWrite(BatchBuffer.MAX_SIZE_BYTES + 1); + sampleBuffer.data.position(BatchBuffer.MAX_SIZE_BYTES + 1); + sampleBuffer.flip(); + assertThat(batchBuffer.append(sampleBuffer)).isTrue(); } @Test - public void setMaxAccessUnitCount_whenSetToTheNumberOfAccessUnitInTheBatch_isFull() { - batchBuffer.commitNextAccessUnit(); - - batchBuffer.setMaxAccessUnitCount(1); - - assertThat(batchBuffer.isFull()).isTrue(); + public void appendSecondSample_exceedingMaxSize_fails() { + initSampleBuffer(); + batchBuffer.append(sampleBuffer); + + int exceedsMaxSize = BatchBuffer.MAX_SIZE_BYTES - sampleBuffer.data.limit() + 1; + sampleBuffer.clear(); + sampleBuffer.ensureSpaceForWrite(exceedsMaxSize); + sampleBuffer.data.position(exceedsMaxSize); + sampleBuffer.flip(); + assertThat(batchBuffer.append(sampleBuffer)).isFalse(); } @Test - public void batchWasConsumed_whenAccessUnitIsPending_pendingAccessUnitIsInTheBatch() { - batchBuffer.commitNextAccessUnit(); - batchBuffer.getNextAccessUnitBuffer().setFlags(C.BUFFER_FLAG_DECODE_ONLY); - batchBuffer.getNextAccessUnitBuffer().ensureSpaceForWrite(TEST_ACCESS_UNIT.length); - batchBuffer.getNextAccessUnitBuffer().data.put(TEST_ACCESS_UNIT); - batchBuffer.commitNextAccessUnit(); - - batchBuffer.batchWasConsumed(); - batchBuffer.flip(); - - assertThat(batchBuffer.getAccessUnitCount()).isEqualTo(1); - assertThat(batchBuffer.isDecodeOnly()).isTrue(); - assertThat(batchBuffer.data).isEqualTo(ByteBuffer.wrap(TEST_ACCESS_UNIT)); + public void appendSecondSample_equalsMaxSize_succeeds() { + initSampleBuffer(); + batchBuffer.append(sampleBuffer); + + int exceedsMaxSize = BatchBuffer.MAX_SIZE_BYTES - sampleBuffer.data.limit(); + sampleBuffer.clear(); + sampleBuffer.ensureSpaceForWrite(exceedsMaxSize); + sampleBuffer.data.position(exceedsMaxSize); + sampleBuffer.flip(); + assertThat(batchBuffer.append(sampleBuffer)).isTrue(); } - private static void fillBatchBuffer(BatchBuffer batchBuffer) { - int maxAccessUnit = batchBuffer.getMaxAccessUnitCount(); - while (!batchBuffer.isFull()) { - assertThat(maxAccessUnit--).isNotEqualTo(0); - batchBuffer.commitNextAccessUnit(); - } + private void initSampleBuffer() { + initSampleBuffer(/* timeUs= */ 0); } - private static void assertIsCleared(BatchBuffer batchBuffer) { - assertThat(batchBuffer.getFirstAccessUnitTimeUs()).isEqualTo(C.TIME_UNSET); - assertThat(batchBuffer.getLastAccessUnitTimeUs()).isEqualTo(C.TIME_UNSET); - assertThat(batchBuffer.getAccessUnitCount()).isEqualTo(0); - assertThat(batchBuffer.isEmpty()).isTrue(); - assertThat(batchBuffer.isFull()).isFalse(); + private void initSampleBuffer(long timeUs) { + sampleBuffer.clear(); + sampleBuffer.timeUs = timeUs; + sampleBuffer.ensureSpaceForWrite(Long.BYTES); + sampleBuffer.data.putLong(timeUs); + sampleBuffer.flip(); } + } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/MediaCodecAsyncCallbackTest.java b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/MediaCodecAsyncCallbackTest.java deleted file mode 100644 index 7cf3f323916..00000000000 --- a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/MediaCodecAsyncCallbackTest.java +++ /dev/null @@ -1,249 +0,0 @@ -/* - * Copyright (C) 2019 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package com.google.android.exoplayer2.mediacodec; - -import static com.google.android.exoplayer2.testutil.TestUtil.assertBufferInfosEqual; -import static com.google.common.truth.Truth.assertThat; -import static org.junit.Assert.fail; - -import android.media.MediaCodec; -import android.media.MediaFormat; -import androidx.test.ext.junit.runners.AndroidJUnit4; -import java.io.IOException; -import org.junit.Before; -import org.junit.Test; -import org.junit.runner.RunWith; - -/** Unit tests for {@link MediaCodecAsyncCallback}. */ -@RunWith(AndroidJUnit4.class) -public class MediaCodecAsyncCallbackTest { - - private MediaCodecAsyncCallback mediaCodecAsyncCallback; - private MediaCodec codec; - - @Before - public void setUp() throws IOException { - mediaCodecAsyncCallback = new MediaCodecAsyncCallback(); - codec = MediaCodec.createByCodecName("h264"); - } - - @Test - public void dequeInputBufferIndex_afterCreation_returnsTryAgain() { - assertThat(mediaCodecAsyncCallback.dequeueInputBufferIndex()) - .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); - } - - @Test - public void dequeInputBufferIndex_returnsEnqueuedBuffers() { - // Send two input buffers to the mediaCodecAsyncCallback. - mediaCodecAsyncCallback.onInputBufferAvailable(codec, 0); - mediaCodecAsyncCallback.onInputBufferAvailable(codec, 1); - - assertThat(mediaCodecAsyncCallback.dequeueInputBufferIndex()).isEqualTo(0); - assertThat(mediaCodecAsyncCallback.dequeueInputBufferIndex()).isEqualTo(1); - assertThat(mediaCodecAsyncCallback.dequeueInputBufferIndex()) - .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); - } - - @Test - public void dequeInputBufferIndex_afterFlush_returnsTryAgain() { - // Send two input buffers to the mediaCodecAsyncCallback and then flush(). - mediaCodecAsyncCallback.onInputBufferAvailable(codec, 0); - mediaCodecAsyncCallback.onInputBufferAvailable(codec, 1); - mediaCodecAsyncCallback.flush(); - - assertThat(mediaCodecAsyncCallback.dequeueInputBufferIndex()) - .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); - } - - @Test - public void dequeInputBufferIndex_afterFlushAndNewInputBuffer_returnsEnqueuedBuffer() { - // Send two input buffers to the mediaCodecAsyncCallback, then flush(), then send - // another input buffer. - mediaCodecAsyncCallback.onInputBufferAvailable(codec, 0); - mediaCodecAsyncCallback.onInputBufferAvailable(codec, 1); - mediaCodecAsyncCallback.flush(); - mediaCodecAsyncCallback.onInputBufferAvailable(codec, 2); - - assertThat(mediaCodecAsyncCallback.dequeueInputBufferIndex()).isEqualTo(2); - } - - @Test - public void dequeOutputBufferIndex_afterCreation_returnsTryAgain() { - MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)) - .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); - } - - @Test - public void dequeOutputBufferIndex_returnsEnqueuedBuffers() { - // Send two output buffers to the mediaCodecAsyncCallback. - MediaCodec.BufferInfo bufferInfo1 = new MediaCodec.BufferInfo(); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, 0, bufferInfo1); - - MediaCodec.BufferInfo bufferInfo2 = new MediaCodec.BufferInfo(); - bufferInfo2.set(1, 1, 1, 1); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, 1, bufferInfo2); - - MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); - - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(0); - assertBufferInfosEqual(bufferInfo1, outBufferInfo); - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(1); - assertBufferInfosEqual(bufferInfo2, outBufferInfo); - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)) - .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); - } - - @Test - public void dequeOutputBufferIndex_afterFlush_returnsTryAgain() { - // Send two output buffers to the mediaCodecAsyncCallback and then flush(). - MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, 0, bufferInfo); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, 1, bufferInfo); - mediaCodecAsyncCallback.flush(); - MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); - - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)) - .isEqualTo(MediaCodec.INFO_TRY_AGAIN_LATER); - } - - @Test - public void dequeOutputBufferIndex_afterFlushAndNewOutputBuffers_returnsEnqueueBuffer() { - // Send two output buffers to the mediaCodecAsyncCallback, then flush(), then send - // another output buffer. - MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, 0, bufferInfo); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, 1, bufferInfo); - mediaCodecAsyncCallback.flush(); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, 2, bufferInfo); - MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); - - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(2); - } - - @Test - public void dequeOutputBufferIndex_withPendingOutputFormat_returnsPendingOutputFormat() { - MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); - - mediaCodecAsyncCallback.onOutputFormatChanged(codec, new MediaFormat()); - MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, /* index= */ 0, bufferInfo); - MediaFormat pendingMediaFormat = new MediaFormat(); - mediaCodecAsyncCallback.onOutputFormatChanged(codec, pendingMediaFormat); - // Flush should not discard the last format. - mediaCodecAsyncCallback.flush(); - // First callback after flush is an output buffer, pending output format should be pushed first. - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, /* index= */ 1, bufferInfo); - - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)) - .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); - assertThat(mediaCodecAsyncCallback.getOutputFormat()).isEqualTo(pendingMediaFormat); - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(1); - } - - @Test - public void dequeOutputBufferIndex_withPendingOutputFormatAndNewFormat_returnsNewFormat() { - mediaCodecAsyncCallback.onOutputFormatChanged(codec, new MediaFormat()); - MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, /* index= */ 0, bufferInfo); - MediaFormat pendingMediaFormat = new MediaFormat(); - mediaCodecAsyncCallback.onOutputFormatChanged(codec, pendingMediaFormat); - // Flush should not discard the last format - mediaCodecAsyncCallback.flush(); - // The first callback after flush is a new MediaFormat, it should overwrite the pending format. - MediaFormat newFormat = new MediaFormat(); - mediaCodecAsyncCallback.onOutputFormatChanged(codec, newFormat); - mediaCodecAsyncCallback.onOutputBufferAvailable(codec, /* index= */ 1, bufferInfo); - MediaCodec.BufferInfo outBufferInfo = new MediaCodec.BufferInfo(); - - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)) - .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); - assertThat(mediaCodecAsyncCallback.getOutputFormat()).isEqualTo(newFormat); - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(outBufferInfo)).isEqualTo(1); - } - - @Test - public void getOutputFormat_onNewInstance_raisesException() { - try { - mediaCodecAsyncCallback.getOutputFormat(); - fail(); - } catch (IllegalStateException expected) { - } - } - - @Test - public void getOutputFormat_afterOnOutputFormatCalled_returnsFormat() { - MediaFormat format = new MediaFormat(); - mediaCodecAsyncCallback.onOutputFormatChanged(codec, format); - MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); - - assertThat(mediaCodecAsyncCallback.dequeueOutputBufferIndex(bufferInfo)) - .isEqualTo(MediaCodec.INFO_OUTPUT_FORMAT_CHANGED); - assertThat(mediaCodecAsyncCallback.getOutputFormat()).isEqualTo(format); - } - - @Test - public void getOutputFormat_afterFlush_raisesCurrentFormat() { - MediaFormat format = new MediaFormat(); - mediaCodecAsyncCallback.onOutputFormatChanged(codec, format); - MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); - mediaCodecAsyncCallback.dequeueOutputBufferIndex(bufferInfo); - mediaCodecAsyncCallback.flush(); - - assertThat(mediaCodecAsyncCallback.getOutputFormat()).isEqualTo(format); - } - - @Test - public void maybeThrowExoPlaybackException_withoutErrorFromCodec_doesNotThrow() { - mediaCodecAsyncCallback.maybeThrowMediaCodecException(); - } - - @Test - public void maybeThrowExoPlaybackException_withErrorFromCodec_Throws() { - IllegalStateException exception = new IllegalStateException(); - mediaCodecAsyncCallback.onMediaCodecError(exception); - - try { - mediaCodecAsyncCallback.maybeThrowMediaCodecException(); - fail(); - } catch (IllegalStateException expected) { - } - } - - @Test - public void maybeThrowExoPlaybackException_doesNotThrowTwice() { - IllegalStateException exception = new IllegalStateException(); - mediaCodecAsyncCallback.onMediaCodecError(exception); - - try { - mediaCodecAsyncCallback.maybeThrowMediaCodecException(); - fail(); - } catch (IllegalStateException expected) { - } - - mediaCodecAsyncCallback.maybeThrowMediaCodecException(); - } - - @Test - public void maybeThrowExoPlaybackException_afterFlush_doesNotThrow() { - IllegalStateException exception = new IllegalStateException(); - mediaCodecAsyncCallback.onMediaCodecError(exception); - mediaCodecAsyncCallback.flush(); - - mediaCodecAsyncCallback.maybeThrowMediaCodecException(); - } -} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/MediaCodecInfoTest.java b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/MediaCodecInfoTest.java new file mode 100644 index 00000000000..346d2773a6c --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/mediacodec/MediaCodecInfoTest.java @@ -0,0 +1,436 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.mediacodec; + +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_AUDIO_CHANNEL_COUNT_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_INITIALIZATION_DATA_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_MIME_TYPE_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_COLOR_INFO_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_RESOLUTION_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_ROTATION_CHANGED; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_NO; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITH_FLUSH; +import static com.google.android.exoplayer2.decoder.DecoderReuseEvaluation.REUSE_RESULT_YES_WITH_RECONFIGURATION; +import static com.google.android.exoplayer2.util.MimeTypes.AUDIO_AAC; +import static com.google.android.exoplayer2.util.MimeTypes.VIDEO_AV1; +import static com.google.android.exoplayer2.util.MimeTypes.VIDEO_H264; +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.decoder.DecoderReuseEvaluation; +import com.google.android.exoplayer2.video.ColorInfo; +import com.google.common.collect.ImmutableList; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link MediaCodecInfo}. */ +@RunWith(AndroidJUnit4.class) +public final class MediaCodecInfoTest { + + private static final Format FORMAT_H264_HD = + new Format.Builder() + .setSampleMimeType(VIDEO_H264) + .setWidth(1024) + .setHeight(768) + .setInitializationData(ImmutableList.of(new byte[] {1, 0, 2, 4})) + .build(); + private static final Format FORMAT_H264_4K = + new Format.Builder() + .setSampleMimeType(VIDEO_H264) + .setWidth(3840) + .setHeight(2160) + .setInitializationData(ImmutableList.of(new byte[] {3, 8, 4, 0})) + .build(); + private static final Format FORMAT_AAC_STEREO = + new Format.Builder() + .setSampleMimeType(AUDIO_AAC) + .setChannelCount(2) + .setSampleRate(44100) + .setAverageBitrate(2000) + .setInitializationData(ImmutableList.of(new byte[] {4, 4, 1, 0, 0})) + .build(); + private static final Format FORMAT_AAC_SURROUND = + new Format.Builder() + .setSampleMimeType(AUDIO_AAC) + .setChannelCount(5) + .setSampleRate(44100) + .setAverageBitrate(5000) + .setInitializationData(ImmutableList.of(new byte[] {4, 4, 1, 0, 0})) + .build(); + + @Test + public void canKeepCodec_withDifferentMimeType_returnsNo() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ true); + + Format hdAv1Format = FORMAT_H264_HD.buildUpon().setSampleMimeType(VIDEO_AV1).build(); + assertThat(codecInfo.canReuseCodec(FORMAT_H264_HD, hdAv1Format)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_H264_HD, + hdAv1Format, + REUSE_RESULT_NO, + DISCARD_REASON_MIME_TYPE_CHANGED)); + } + + @Test + public void canKeepCodec_withRotation_returnsNo() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ true); + + Format hdRotatedFormat = FORMAT_H264_HD.buildUpon().setRotationDegrees(90).build(); + assertThat(codecInfo.canReuseCodec(FORMAT_H264_HD, hdRotatedFormat)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_H264_HD, + hdRotatedFormat, + REUSE_RESULT_NO, + DISCARD_REASON_VIDEO_ROTATION_CHANGED)); + } + + @Test + public void canKeepCodec_withResolutionChange_adaptiveCodec_returnsYesWithReconfiguration() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ true); + + assertThat(codecInfo.canReuseCodec(FORMAT_H264_HD, FORMAT_H264_4K)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_H264_HD, + FORMAT_H264_4K, + REUSE_RESULT_YES_WITH_RECONFIGURATION, + /* discardReasons= */ 0)); + } + + @Test + public void canKeepCodec_withResolutionChange_nonAdaptiveCodec_returnsNo() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + assertThat(codecInfo.canReuseCodec(FORMAT_H264_HD, FORMAT_H264_4K)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_H264_HD, + FORMAT_H264_4K, + REUSE_RESULT_NO, + DISCARD_REASON_VIDEO_RESOLUTION_CHANGED)); + } + + @Test + public void canKeepCodec_noResolutionChange_nonAdaptiveCodec_returnsYesWithReconfiguration() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdVariantFormat = + FORMAT_H264_HD.buildUpon().setInitializationData(ImmutableList.of(new byte[] {0})).build(); + assertThat(codecInfo.canReuseCodec(FORMAT_H264_HD, hdVariantFormat)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_H264_HD, + hdVariantFormat, + REUSE_RESULT_YES_WITH_RECONFIGURATION, + /* discardReasons= */ 0)); + } + + @Test + public void canKeepCodec_colorInfoOmittedFromNewFormat_returnsNo() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdrVariantFormat = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT601)).build(); + assertThat(codecInfo.canReuseCodec(hdrVariantFormat, FORMAT_H264_4K)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + hdrVariantFormat, + FORMAT_H264_4K, + REUSE_RESULT_NO, + DISCARD_REASON_VIDEO_COLOR_INFO_CHANGED)); + } + + @Test + public void canKeepCodec_colorInfoOmittedFromOldFormat_returnsNo() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdrVariantFormat = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT601)).build(); + assertThat(codecInfo.canReuseCodec(FORMAT_H264_4K, hdrVariantFormat)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_H264_4K, + hdrVariantFormat, + REUSE_RESULT_NO, + DISCARD_REASON_VIDEO_COLOR_INFO_CHANGED)); + } + + @Test + public void canKeepCodec_colorInfoChange_returnsNo() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdrVariantFormat1 = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT601)).build(); + Format hdrVariantFormat2 = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT709)).build(); + assertThat(codecInfo.canReuseCodec(hdrVariantFormat1, hdrVariantFormat2)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + hdrVariantFormat1, + hdrVariantFormat2, + REUSE_RESULT_NO, + DISCARD_REASON_VIDEO_COLOR_INFO_CHANGED)); + } + + @Test + public void canKeepCodec_audioWithDifferentChannelCounts_returnsNo() { + MediaCodecInfo codecInfo = buildAacCodecInfo(); + + assertThat(codecInfo.canReuseCodec(FORMAT_AAC_STEREO, FORMAT_AAC_SURROUND)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_AAC_STEREO, + FORMAT_AAC_SURROUND, + REUSE_RESULT_NO, + DISCARD_REASON_AUDIO_CHANNEL_COUNT_CHANGED)); + } + + @Test + public void canKeepCodec_audioWithSameChannelCounts_returnsYesWithFlush() { + MediaCodecInfo codecInfo = buildAacCodecInfo(); + + Format stereoVariantFormat = FORMAT_AAC_STEREO.buildUpon().setAverageBitrate(100).build(); + assertThat(codecInfo.canReuseCodec(FORMAT_AAC_STEREO, stereoVariantFormat)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_AAC_STEREO, + stereoVariantFormat, + REUSE_RESULT_YES_WITH_FLUSH, + /* discardReasons= */ 0)); + } + + @Test + public void canKeepCodec_audioWithDifferentInitializationData_returnsNo() { + MediaCodecInfo codecInfo = buildAacCodecInfo(); + + Format stereoVariantFormat = + FORMAT_AAC_STEREO + .buildUpon() + .setInitializationData(ImmutableList.of(new byte[] {0})) + .build(); + assertThat(codecInfo.canReuseCodec(FORMAT_AAC_STEREO, stereoVariantFormat)) + .isEqualTo( + new DecoderReuseEvaluation( + codecInfo.name, + FORMAT_AAC_STEREO, + stereoVariantFormat, + REUSE_RESULT_NO, + DISCARD_REASON_INITIALIZATION_DATA_CHANGED)); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_withDifferentMimeType_returnsFalse() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ true); + + Format hdAv1Format = FORMAT_H264_HD.buildUpon().setSampleMimeType(VIDEO_AV1).build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_H264_HD, hdAv1Format, /* isNewFormatComplete= */ true)) + .isFalse(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_withRotation_returnsFalse() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ true); + + Format hdRotatedFormat = FORMAT_H264_HD.buildUpon().setRotationDegrees(90).build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_H264_HD, hdRotatedFormat, /* isNewFormatComplete= */ true)) + .isFalse(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_withResolutionChange_adaptiveCodec_returnsTrue() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ true); + + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_H264_HD, FORMAT_H264_4K, /* isNewFormatComplete= */ true)) + .isTrue(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_withResolutionChange_nonAdaptiveCodec_returnsFalse() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_H264_HD, FORMAT_H264_4K, /* isNewFormatComplete= */ true)) + .isFalse(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_noResolutionChange_nonAdaptiveCodec_returnsTrue() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdVariantFormat = + FORMAT_H264_HD.buildUpon().setInitializationData(ImmutableList.of(new byte[] {0})).build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_H264_HD, hdVariantFormat, /* isNewFormatComplete= */ true)) + .isTrue(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_colorInfoOmittedFromCompleteNewFormat_returnsFalse() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdrVariantFormat = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT601)).build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + hdrVariantFormat, FORMAT_H264_4K, /* isNewFormatComplete= */ true)) + .isFalse(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_colorInfoOmittedFromIncompleteNewFormat_returnsTrue() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdrVariantFormat = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT601)).build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + hdrVariantFormat, FORMAT_H264_4K, /* isNewFormatComplete= */ false)) + .isTrue(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_colorInfoOmittedFromOldFormat_returnsFalse() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdrVariantFormat = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT601)).build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_H264_4K, hdrVariantFormat, /* isNewFormatComplete= */ true)) + .isFalse(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_colorInfoChange_returnsFalse() { + MediaCodecInfo codecInfo = buildH264CodecInfo(/* adaptive= */ false); + + Format hdrVariantFormat1 = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT601)).build(); + Format hdrVariantFormat2 = + FORMAT_H264_4K.buildUpon().setColorInfo(buildColorInfo(C.COLOR_SPACE_BT709)).build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + hdrVariantFormat1, hdrVariantFormat2, /* isNewFormatComplete= */ true)) + .isFalse(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + hdrVariantFormat1, hdrVariantFormat2, /* isNewFormatComplete= */ false)) + .isFalse(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_audioWithDifferentChannelCounts_returnsFalse() { + MediaCodecInfo codecInfo = buildAacCodecInfo(); + + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_AAC_STEREO, FORMAT_AAC_SURROUND, /* isNewFormatComplete= */ true)) + .isFalse(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_audioWithSameChannelCounts_returnsFalse() { + MediaCodecInfo codecInfo = buildAacCodecInfo(); + + Format stereoVariantFormat = FORMAT_AAC_STEREO.buildUpon().setAverageBitrate(100).build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_AAC_STEREO, stereoVariantFormat, /* isNewFormatComplete= */ true)) + .isFalse(); + } + + @Test + @SuppressWarnings("deprecation") + public void isSeamlessAdaptationSupported_audioWithDifferentInitializationData_returnsFalse() { + MediaCodecInfo codecInfo = buildAacCodecInfo(); + + Format stereoVariantFormat = + FORMAT_AAC_STEREO + .buildUpon() + .setInitializationData(ImmutableList.of(new byte[] {0})) + .build(); + assertThat( + codecInfo.isSeamlessAdaptationSupported( + FORMAT_AAC_STEREO, stereoVariantFormat, /* isNewFormatComplete= */ true)) + .isFalse(); + } + + private static MediaCodecInfo buildH264CodecInfo(boolean adaptive) { + return new MediaCodecInfo( + "h264", + VIDEO_H264, + VIDEO_H264, + /* capabilities= */ null, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ true, + adaptive, + /* tunneling= */ false, + /* secure= */ false); + } + + private static MediaCodecInfo buildAacCodecInfo() { + return new MediaCodecInfo( + "aac", + AUDIO_AAC, + AUDIO_AAC, + /* capabilities= */ null, + /* hardwareAccelerated= */ false, + /* softwareOnly= */ true, + /* vendor= */ false, + /* adaptive= */ false, + /* tunneling= */ false, + /* secure= */ false); + } + + private static ColorInfo buildColorInfo(@C.ColorSpace int colorSpace) { + return new ColorInfo( + colorSpace, C.COLOR_RANGE_FULL, C.COLOR_TRANSFER_HLG, /* hdrStaticInfo= */ null); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/metadata/MetadataRendererTest.java b/library/core/src/test/java/com/google/android/exoplayer2/metadata/MetadataRendererTest.java index 796f56becff..42dcaa572d3 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/metadata/MetadataRendererTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/metadata/MetadataRendererTest.java @@ -15,11 +15,14 @@ */ package com.google.android.exoplayer2.metadata; +import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; +import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.sample; import static com.google.common.truth.Truth.assertThat; import static java.nio.charset.StandardCharsets.ISO_8859_1; import static java.nio.charset.StandardCharsets.UTF_8; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlaybackException; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.drm.DrmSessionEventListener; @@ -29,8 +32,8 @@ import com.google.android.exoplayer2.metadata.id3.TextInformationFrame; import com.google.android.exoplayer2.metadata.scte35.TimeSignalCommand; import com.google.android.exoplayer2.testutil.FakeSampleStream; -import com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem; import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.android.exoplayer2.upstream.DefaultAllocator; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.collect.ImmutableList; @@ -144,16 +147,19 @@ public void decodeMetadata_skipsMalformedWrappedMetadata() throws Exception { private static List runRenderer(byte[] input) throws ExoPlaybackException { List metadata = new ArrayList<>(); MetadataRenderer renderer = new MetadataRenderer(metadata::add, /* outputLooper= */ null); - renderer.replaceStream( - new Format[] {EMSG_FORMAT}, + FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), EMSG_FORMAT, ImmutableList.of( - FakeSampleStreamItem.sample(/* timeUs= */ 0, /* flags= */ 0, input), - FakeSampleStreamItem.END_OF_STREAM_ITEM)), + sample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME, input), END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); + renderer.replaceStream( + new Format[] {EMSG_FORMAT}, + fakeSampleStream, /* startPositionUs= */ 0L, /* offsetUs= */ 0L); renderer.render(/* positionUs= */ 0, /* elapsedRealtimeUs= */ 0); // Read the format diff --git a/library/core/src/test/java/com/google/android/exoplayer2/offline/DownloadHelperTest.java b/library/core/src/test/java/com/google/android/exoplayer2/offline/DownloadHelperTest.java index 76f92674307..6700f0de7a3 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/offline/DownloadHelperTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/offline/DownloadHelperTest.java @@ -38,8 +38,8 @@ import com.google.android.exoplayer2.testutil.FakeTimeline; import com.google.android.exoplayer2.testutil.FakeTimeline.TimelineWindowDefinition; import com.google.android.exoplayer2.trackselection.DefaultTrackSelector; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.trackselection.MappingTrackSelector.MappedTrackInfo; -import com.google.android.exoplayer2.trackselection.TrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.util.MimeTypes; import java.io.IOException; @@ -99,8 +99,7 @@ public static void staticSetUp() { trackGroupTextZh); TrackGroupArray trackGroupArraySingle = new TrackGroupArray(TRACK_GROUP_VIDEO_SINGLE, trackGroupAudioUs); - trackGroupArrays = - new TrackGroupArray[] {trackGroupArrayAll, trackGroupArraySingle}; + trackGroupArrays = new TrackGroupArray[] {trackGroupArrayAll, trackGroupArraySingle}; testMediaItem = new MediaItem.Builder().setUri("http://test.uri").setCustomCacheKey("cacheKey").build(); @@ -194,17 +193,17 @@ public void getMappedTrackInfo_returnsMappedTrackInfo() throws Exception { public void getTrackSelections_returnsInitialSelection() throws Exception { prepareDownloadHelper(downloadHelper); - List selectedText0 = + List selectedText0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 0); - List selectedAudio0 = + List selectedAudio0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 1); - List selectedVideo0 = + List selectedVideo0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 2); - List selectedText1 = + List selectedText1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 0); - List selectedAudio1 = + List selectedAudio1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 1); - List selectedVideo1 = + List selectedVideo1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 2); assertSingleTrackSelectionEquals(selectedText0, trackGroupTextUs, 0); @@ -222,17 +221,17 @@ public void getTrackSelections_afterClearTrackSelections_isEmpty() throws Except // Clear only one period selection to verify second period selection is untouched. downloadHelper.clearTrackSelections(/* periodIndex= */ 0); - List selectedText0 = + List selectedText0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 0); - List selectedAudio0 = + List selectedAudio0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 1); - List selectedVideo0 = + List selectedVideo0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 2); - List selectedText1 = + List selectedText1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 0); - List selectedAudio1 = + List selectedAudio1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 1); - List selectedVideo1 = + List selectedVideo1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 2); assertThat(selectedText0).isEmpty(); @@ -258,17 +257,17 @@ public void getTrackSelections_afterReplaceTrackSelections_returnsNewSelections( // Replace only one period selection to verify second period selection is untouched. downloadHelper.replaceTrackSelections(/* periodIndex= */ 0, parameters); - List selectedText0 = + List selectedText0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 0); - List selectedAudio0 = + List selectedAudio0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 1); - List selectedVideo0 = + List selectedVideo0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 2); - List selectedText1 = + List selectedText1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 0); - List selectedAudio1 = + List selectedAudio1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 1); - List selectedVideo1 = + List selectedVideo1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 2); assertSingleTrackSelectionEquals(selectedText0, trackGroupTextZh, 0); @@ -294,17 +293,17 @@ public void getTrackSelections_afterAddTrackSelections_returnsCombinedSelections // Add only to one period selection to verify second period selection is untouched. downloadHelper.addTrackSelection(/* periodIndex= */ 0, parameters); - List selectedText0 = + List selectedText0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 0); - List selectedAudio0 = + List selectedAudio0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 1); - List selectedVideo0 = + List selectedVideo0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 2); - List selectedText1 = + List selectedText1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 0); - List selectedAudio1 = + List selectedAudio1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 1); - List selectedVideo1 = + List selectedVideo1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 2); assertSingleTrackSelectionEquals(selectedText0, trackGroupTextUs, 0); @@ -327,17 +326,17 @@ public void getTrackSelections_afterAddAudioLanguagesToSelection_returnsCombined // Add a non-default language, and a non-existing language (which will select the default). downloadHelper.addAudioLanguagesToSelection("ZH", "Klingonese"); - List selectedText0 = + List selectedText0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 0); - List selectedAudio0 = + List selectedAudio0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 1); - List selectedVideo0 = + List selectedVideo0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 2); - List selectedText1 = + List selectedText1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 0); - List selectedAudio1 = + List selectedAudio1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 1); - List selectedVideo1 = + List selectedVideo1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 2); assertThat(selectedVideo0).isEmpty(); @@ -361,17 +360,17 @@ public void getTrackSelections_afterAddTextLanguagesToSelection_returnsCombinedS // Add a non-default language, and a non-existing language (which will select the default). downloadHelper.addTextLanguagesToSelection( /* selectUndeterminedTextLanguage= */ true, "ZH", "Klingonese"); - List selectedText0 = + List selectedText0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 0); - List selectedAudio0 = + List selectedAudio0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 1); - List selectedVideo0 = + List selectedVideo0 = downloadHelper.getTrackSelections(/* periodIndex= */ 0, /* rendererIndex= */ 2); - List selectedText1 = + List selectedText1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 0); - List selectedAudio1 = + List selectedAudio1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 1); - List selectedVideo1 = + List selectedVideo1 = downloadHelper.getTrackSelections(/* periodIndex= */ 1, /* rendererIndex= */ 2); assertThat(selectedVideo0).isEmpty(); @@ -464,13 +463,13 @@ private static Format createTextFormat(String language) { } private static void assertSingleTrackSelectionEquals( - List trackSelectionList, TrackGroup trackGroup, int... tracks) { + List trackSelectionList, TrackGroup trackGroup, int... tracks) { assertThat(trackSelectionList).hasSize(1); assertTrackSelectionEquals(trackSelectionList.get(0), trackGroup, tracks); } private static void assertTrackSelectionEquals( - TrackSelection trackSelection, TrackGroup trackGroup, int... tracks) { + ExoTrackSelection trackSelection, TrackGroup trackGroup, int... tracks) { assertThat(trackSelection.getTrackGroup()).isEqualTo(trackGroup); assertThat(trackSelection.length()).isEqualTo(tracks.length); int[] selectedTracksInGroup = new int[trackSelection.length()]; @@ -493,13 +492,14 @@ public MediaPeriod createPeriod(MediaPeriodId id, Allocator allocator, long star int periodIndex = TEST_TIMELINE.getIndexOfPeriod(id.periodUid); return new FakeMediaPeriod( trackGroupArrays[periodIndex], + allocator, TEST_TIMELINE.getWindow(0, new Timeline.Window()).positionInFirstPeriodUs, new EventDispatcher() .withParameters(/* windowIndex= */ 0, id, /* mediaTimeOffsetMs= */ 0)) { @Override - public List getStreamKeys(List trackSelections) { + public List getStreamKeys(List trackSelections) { List result = new ArrayList<>(); - for (TrackSelection trackSelection : trackSelections) { + for (ExoTrackSelection trackSelection : trackSelections) { int groupIndex = trackGroupArrays[periodIndex].indexOf(trackSelection.getTrackGroup()); for (int i = 0; i < trackSelection.length(); i++) { result.add( diff --git a/library/core/src/test/java/com/google/android/exoplayer2/offline/DownloadManagerTest.java b/library/core/src/test/java/com/google/android/exoplayer2/offline/DownloadManagerTest.java index 31bc7e5a926..b32e627ca78 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/offline/DownloadManagerTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/offline/DownloadManagerTest.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.offline; +import static com.google.android.exoplayer2.robolectric.RobolectricUtil.createRobolectricConditionVariable; import static com.google.common.truth.Truth.assertThat; import static java.util.Arrays.asList; @@ -24,11 +25,11 @@ import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.robolectric.TestDownloadManagerListener; import com.google.android.exoplayer2.scheduler.Requirements; import com.google.android.exoplayer2.testutil.DownloadBuilder; import com.google.android.exoplayer2.testutil.DummyMainThread; import com.google.android.exoplayer2.testutil.DummyMainThread.TestRunnable; -import com.google.android.exoplayer2.testutil.TestDownloadManagerListener; import com.google.android.exoplayer2.testutil.TestUtil; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.ConditionVariable; @@ -895,10 +896,10 @@ private static final class FakeDownloader implements Downloader { private FakeDownloader(DownloadRequest request) { this.request = request; - downloadStarted = TestUtil.createRobolectricConditionVariable(); - removeStarted = TestUtil.createRobolectricConditionVariable(); - finished = TestUtil.createRobolectricConditionVariable(); - blocker = TestUtil.createRobolectricConditionVariable(); + downloadStarted = createRobolectricConditionVariable(); + removeStarted = createRobolectricConditionVariable(); + finished = createRobolectricConditionVariable(); + blocker = createRobolectricConditionVariable(); startCount = new AtomicInteger(); bytesDownloaded = new AtomicInteger(); } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/ClippingMediaSourceTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/ClippingMediaSourceTest.java index 8fce3b25acd..5e391053597 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/ClippingMediaSourceTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/ClippingMediaSourceTest.java @@ -32,7 +32,6 @@ import com.google.android.exoplayer2.source.ClippingMediaSource.IllegalClippingException; import com.google.android.exoplayer2.source.MaskingMediaSource.PlaceholderTimeline; import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; -import com.google.android.exoplayer2.testutil.FakeMediaPeriod; import com.google.android.exoplayer2.testutil.FakeMediaSource; import com.google.android.exoplayer2.testutil.FakeTimeline; import com.google.android.exoplayer2.testutil.FakeTimeline.TimelineWindowDefinition; @@ -69,7 +68,7 @@ public void noClipping() throws IOException { TEST_PERIOD_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -90,7 +89,7 @@ public void clippingUnseekableWindowThrows() throws IOException { TEST_PERIOD_DURATION_US, /* isSeekable= */ false, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -112,7 +111,7 @@ public void clippingUnseekableWindowWithUnknownDurationThrows() throws IOExcepti /* durationUs= */ C.TIME_UNSET, /* isSeekable= */ false, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -134,7 +133,7 @@ public void clippingStart() throws IOException { TEST_PERIOD_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -153,7 +152,7 @@ public void clippingEnd() throws IOException { TEST_PERIOD_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -189,7 +188,7 @@ public void clippingToEndOfSourceWithDurationSetsDuration() throws IOException { /* durationUs= */ TEST_PERIOD_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -208,7 +207,7 @@ public void clippingToEndOfSourceWithUnsetDurationDoesNotSetDuration() throws IO /* durationUs= */ C.TIME_UNSET, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -226,7 +225,7 @@ public void clippingStartAndEnd() throws IOException { TEST_PERIOD_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -249,7 +248,7 @@ public void clippingFromDefaultPosition() throws IOException { /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -272,7 +271,7 @@ public void allowDynamicUpdatesWithOverlappingLiveWindow() throws IOException { /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); Timeline timeline2 = @@ -283,7 +282,7 @@ public void allowDynamicUpdatesWithOverlappingLiveWindow() throws IOException { /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -323,7 +322,7 @@ public void allowDynamicUpdatesWithNonOverlappingLiveWindow() throws IOException /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); Timeline timeline2 = @@ -334,7 +333,7 @@ public void allowDynamicUpdatesWithNonOverlappingLiveWindow() throws IOException /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -374,7 +373,7 @@ public void disallowDynamicUpdatesWithOverlappingLiveWindow() throws IOException /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); Timeline timeline2 = @@ -385,7 +384,7 @@ public void disallowDynamicUpdatesWithOverlappingLiveWindow() throws IOException /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -426,7 +425,7 @@ public void disallowDynamicUpdatesWithNonOverlappingLiveWindow() throws IOExcept /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); Timeline timeline2 = @@ -437,7 +436,7 @@ public void disallowDynamicUpdatesWithNonOverlappingLiveWindow() throws IOExcept /* windowDefaultStartPositionUs= */ TEST_CLIP_AMOUNT_US, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); @@ -556,13 +555,13 @@ private static MediaLoadData getClippingMediaSourceMediaLoadData( TEST_PERIOD_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); FakeMediaSource fakeMediaSource = new FakeMediaSource(timeline) { @Override - protected FakeMediaPeriod createFakeMediaPeriod( + protected MediaPeriod createMediaPeriod( MediaPeriodId id, TrackGroupArray trackGroupArray, Allocator allocator, @@ -579,7 +578,7 @@ protected FakeMediaPeriod createFakeMediaPeriod( /* trackSelectionData= */ null, C.usToMs(eventStartUs), C.usToMs(eventEndUs))); - return super.createFakeMediaPeriod( + return super.createMediaPeriod( id, trackGroupArray, allocator, diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/ConcatenatingMediaSourceTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/ConcatenatingMediaSourceTest.java index 7ba0cc02e5a..f34ab81e0ff 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/ConcatenatingMediaSourceTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/ConcatenatingMediaSourceTest.java @@ -499,8 +499,8 @@ public void customCallbackBeforePreparationMove() throws Exception { mediaSource.addMediaSources( Arrays.asList(new MediaSource[] {createFakeMediaSource(), createFakeMediaSource()})); mediaSource.moveMediaSource( - /* fromIndex */ 1, /* toIndex */ - 0, + /* currentIndex= */ 1, + /* newIndex= */ 0, Util.createHandlerForCurrentLooper(), runnableInvoked::countDown); }); @@ -624,8 +624,8 @@ public void customCallbackAfterPreparationMove() throws Exception { testThread.runOnMainThread( () -> mediaSource.moveMediaSource( - /* fromIndex */ 1, /* toIndex */ - 0, + /* currentIndex= */ 1, + /* newIndex= */ 0, Util.createHandlerForCurrentLooper(), timelineGrabber)); Timeline timeline = timelineGrabber.assertTimelineChangeBlocking(); @@ -854,7 +854,7 @@ public void duplicateMediaSources() throws IOException, InterruptedException { @Test public void duplicateNestedMediaSources() throws IOException, InterruptedException { - Timeline childTimeline = new FakeTimeline(/* windowCount= */ 1); + Timeline childTimeline = new FakeTimeline(); FakeMediaSource childSource = new FakeMediaSource(childTimeline); ConcatenatingMediaSource nestedConcatenation = new ConcatenatingMediaSource(); diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/MediaSourceDrmHelperTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/DefaultDrmSessionManagerProviderTest.java similarity index 75% rename from library/core/src/test/java/com/google/android/exoplayer2/source/MediaSourceDrmHelperTest.java rename to library/core/src/test/java/com/google/android/exoplayer2/source/DefaultDrmSessionManagerProviderTest.java index 45384f05ec5..4e597b63717 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/MediaSourceDrmHelperTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/DefaultDrmSessionManagerProviderTest.java @@ -21,20 +21,21 @@ import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.drm.DefaultDrmSessionManagerProvider; import com.google.android.exoplayer2.drm.DrmSessionManager; import org.junit.Test; import org.junit.runner.RunWith; -/** Unit tests for {@link MediaSourceDrmHelper}. */ +/** Unit tests for {@link DefaultDrmSessionManagerProvider}. */ @RunWith(AndroidJUnit4.class) -public class MediaSourceDrmHelperTest { +public class DefaultDrmSessionManagerProviderTest { @Test public void create_noDrmProperties_createsNoopManager() { DrmSessionManager drmSessionManager = - new MediaSourceDrmHelper().create(MediaItem.fromUri(Uri.EMPTY)); + new DefaultDrmSessionManagerProvider().get(MediaItem.fromUri(Uri.EMPTY)); - assertThat(drmSessionManager).isEqualTo(DrmSessionManager.DUMMY); + assertThat(drmSessionManager).isEqualTo(DrmSessionManager.DRM_UNSUPPORTED); } @Test @@ -46,8 +47,8 @@ public void create_createsManager() { .setDrmUuid(C.WIDEVINE_UUID) .build(); - DrmSessionManager drmSessionManager = new MediaSourceDrmHelper().create(mediaItem); + DrmSessionManager drmSessionManager = new DefaultDrmSessionManagerProvider().get(mediaItem); - assertThat(drmSessionManager).isNotEqualTo(DrmSessionManager.DUMMY); + assertThat(drmSessionManager).isNotEqualTo(DrmSessionManager.DRM_UNSUPPORTED); } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/DefaultMediaSourceFactoryTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/DefaultMediaSourceFactoryTest.java index d02f04d0977..67f06e4f85a 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/DefaultMediaSourceFactoryTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/DefaultMediaSourceFactoryTest.java @@ -204,7 +204,7 @@ public void createMediaSource_withAdTagUri_callsAdsLoader() { MediaItem mediaItem = new MediaItem.Builder().setUri(URI_MEDIA).setAdTagUri(adTagUri).build(); DefaultMediaSourceFactory defaultMediaSourceFactory = new DefaultMediaSourceFactory((Context) ApplicationProvider.getApplicationContext()) - .setAdsLoaderProvider(ignoredAdTagUri -> mock(AdsLoader.class)) + .setAdsLoaderProvider(ignoredAdsConfiguration -> mock(AdsLoader.class)) .setAdViewProvider(mock(AdsLoader.AdViewProvider.class)); MediaSource mediaSource = defaultMediaSourceFactory.createMediaSource(mediaItem); @@ -235,4 +235,66 @@ public void createMediaSource_withAdTagUriProvidersNull_playsWithoutAdNoExceptio assertThat(mediaSource).isNotInstanceOf(AdsMediaSource.class); } + + @Test + public void createMediaSource_undefinedLiveProperties_livePropertiesUnset() { + DefaultMediaSourceFactory defaultMediaSourceFactory = + new DefaultMediaSourceFactory((Context) ApplicationProvider.getApplicationContext()); + MediaItem mediaItem = new MediaItem.Builder().setUri(URI_MEDIA + "/file.mp4").build(); + MediaSource mediaSource = defaultMediaSourceFactory.createMediaSource(mediaItem); + + MediaItem mediaItemFromSource = mediaSource.getMediaItem(); + + assertThat(mediaItemFromSource.liveConfiguration.targetOffsetMs).isEqualTo(C.TIME_UNSET); + assertThat(mediaItemFromSource.liveConfiguration.minOffsetMs).isEqualTo(C.TIME_UNSET); + assertThat(mediaItemFromSource.liveConfiguration.maxOffsetMs).isEqualTo(C.TIME_UNSET); + assertThat(mediaItemFromSource.liveConfiguration.minPlaybackSpeed).isEqualTo(C.RATE_UNSET); + assertThat(mediaItemFromSource.liveConfiguration.maxPlaybackSpeed).isEqualTo(C.RATE_UNSET); + } + + @Test + public void createMediaSource_withoutMediaItemProperties_usesFactoryLiveProperties() { + DefaultMediaSourceFactory defaultMediaSourceFactory = + new DefaultMediaSourceFactory((Context) ApplicationProvider.getApplicationContext()) + .setLiveTargetOffsetMs(20) + .setLiveMinOffsetMs(2222) + .setLiveMaxOffsetMs(4444) + .setLiveMinSpeed(.1f) + .setLiveMaxSpeed(2.0f); + MediaItem mediaItem = new MediaItem.Builder().setUri(URI_MEDIA + "/file.mp4").build(); + MediaSource mediaSource = defaultMediaSourceFactory.createMediaSource(mediaItem); + + MediaItem mediaItemFromSource = mediaSource.getMediaItem(); + + assertThat(mediaItemFromSource.liveConfiguration.targetOffsetMs).isEqualTo(20); + assertThat(mediaItemFromSource.liveConfiguration.minOffsetMs).isEqualTo(2222); + assertThat(mediaItemFromSource.liveConfiguration.maxOffsetMs).isEqualTo(4444); + assertThat(mediaItemFromSource.liveConfiguration.minPlaybackSpeed).isEqualTo(.1f); + assertThat(mediaItemFromSource.liveConfiguration.maxPlaybackSpeed).isEqualTo(2.0f); + } + + @Test + public void createMediaSource_withMediaItemLiveProperties_overridesFactoryLiveProperties() { + DefaultMediaSourceFactory defaultMediaSourceFactory = + new DefaultMediaSourceFactory((Context) ApplicationProvider.getApplicationContext()) + .setLiveTargetOffsetMs(20) + .setLiveMinOffsetMs(2222) + .setLiveMinOffsetMs(4444) + .setLiveMinSpeed(.1f) + .setLiveMaxSpeed(2.0f); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(URI_MEDIA + "/file.mp4") + .setLiveTargetOffsetMs(10) + .setLiveMinOffsetMs(1111) + .setLiveMinOffsetMs(3333) + .setLiveMinPlaybackSpeed(20.0f) + .setLiveMaxPlaybackSpeed(20.0f) + .build(); + MediaSource mediaSource = defaultMediaSourceFactory.createMediaSource(mediaItem); + + MediaItem mediaItemFromSource = mediaSource.getMediaItem(); + + assertThat(mediaItemFromSource).isEqualTo(mediaItem); + } } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/MergingMediaPeriodTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/MergingMediaPeriodTest.java index e28af160c38..4a756ccf9fa 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/MergingMediaPeriodTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/MergingMediaPeriodTest.java @@ -29,8 +29,9 @@ import com.google.android.exoplayer2.source.MediaSource.MediaPeriodId; import com.google.android.exoplayer2.source.MediaSourceEventListener.EventDispatcher; import com.google.android.exoplayer2.testutil.FakeMediaPeriod; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.trackselection.FixedTrackSelection; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.upstream.DefaultAllocator; import com.google.common.collect.ImmutableList; import java.util.concurrent.CountDownLatch; import org.checkerframework.checker.nullness.compatqual.NullableType; @@ -71,17 +72,20 @@ public void selectTracks_createsSampleStreamsFromChildPeriods() throws Exception new MergingPeriodDefinition( /* timeOffsetUs= */ 0, /* singleSampleTimeUs= */ 0, childFormat21, childFormat22)); - TrackSelection selectionForChild1 = + ExoTrackSelection selectionForChild1 = new FixedTrackSelection(mergingMediaPeriod.getTrackGroups().get(1), /* track= */ 0); - TrackSelection selectionForChild2 = + ExoTrackSelection selectionForChild2 = new FixedTrackSelection(mergingMediaPeriod.getTrackGroups().get(2), /* track= */ 0); SampleStream[] streams = new SampleStream[4]; mergingMediaPeriod.selectTracks( - /* selections= */ new TrackSelection[] {null, selectionForChild1, selectionForChild2, null}, + /* selections= */ new ExoTrackSelection[] { + null, selectionForChild1, selectionForChild2, null + }, /* mayRetainStreamFlags= */ new boolean[] {false, false, false, false}, streams, /* streamResetFlags= */ new boolean[] {false, false, false, false}, /* positionUs= */ 0); + mergingMediaPeriod.continueLoading(/* positionUs= */ 0); assertThat(streams[0]).isNull(); assertThat(streams[3]).isNull(); @@ -115,17 +119,18 @@ public void selectTracks_createsSampleStreamsFromChildPeriods() throws Exception childFormat21, childFormat22)); - TrackSelection selectionForChild1 = + ExoTrackSelection selectionForChild1 = new FixedTrackSelection(mergingMediaPeriod.getTrackGroups().get(0), /* track= */ 0); - TrackSelection selectionForChild2 = + ExoTrackSelection selectionForChild2 = new FixedTrackSelection(mergingMediaPeriod.getTrackGroups().get(2), /* track= */ 0); SampleStream[] streams = new SampleStream[2]; mergingMediaPeriod.selectTracks( - /* selections= */ new TrackSelection[] {selectionForChild1, selectionForChild2}, + /* selections= */ new ExoTrackSelection[] {selectionForChild1, selectionForChild2}, /* mayRetainStreamFlags= */ new boolean[] {false, false}, streams, /* streamResetFlags= */ new boolean[] {false, false}, /* positionUs= */ 0); + mergingMediaPeriod.continueLoading(/* positionUs= */ 0); FormatHolder formatHolder = new FormatHolder(); DecoderInputBuffer inputBuffer = new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); @@ -168,7 +173,8 @@ private MergingMediaPeriod prepareMergingPeriod(MergingPeriodDefinition... defin /* mediaTimeOffsetMs= */ 0), /* trackDataFactory= */ (unusedFormat, unusedMediaPeriodId) -> ImmutableList.of( - oneByteSample(definition.singleSampleTimeUs), END_OF_STREAM_ITEM)); + oneByteSample(definition.singleSampleTimeUs, C.BUFFER_FLAG_KEY_FRAME), + END_OF_STREAM_ITEM)); } MergingMediaPeriod mergingMediaPeriod = new MergingMediaPeriod( @@ -203,9 +209,10 @@ public FakeMediaPeriodWithSelectTracksPosition( TrackDataFactory trackDataFactory) { super( trackGroupArray, + new DefaultAllocator(/* trimOnReset= */ false, /* individualAllocationSize= */ 1024), trackDataFactory, mediaSourceEventDispatcher, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* deferOnPrepared= */ false); selectTracksPositionUs = C.TIME_UNSET; @@ -213,7 +220,7 @@ public FakeMediaPeriodWithSelectTracksPosition( @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/MergingMediaSourceTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/MergingMediaSourceTest.java index c66a5cff741..b0f54dd8606 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/MergingMediaSourceTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/MergingMediaSourceTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.source; import static com.google.common.truth.Truth.assertThat; -import static org.junit.Assert.fail; +import static org.junit.Assert.assertThrows; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; @@ -35,35 +35,65 @@ public class MergingMediaSourceTest { @Test - public void mergingDynamicTimelines() throws IOException { - FakeTimeline firstTimeline = - new FakeTimeline(new TimelineWindowDefinition(true, true, C.TIME_UNSET)); - FakeTimeline secondTimeline = - new FakeTimeline(new TimelineWindowDefinition(true, true, C.TIME_UNSET)); - testMergingMediaSourcePrepare(firstTimeline, secondTimeline); + public void prepare_withoutDurationClipping_usesTimelineOfFirstSource() throws IOException { + FakeTimeline timeline1 = + new FakeTimeline( + new TimelineWindowDefinition( + /* isSeekable= */ true, /* isDynamic= */ true, /* durationUs= */ 30)); + FakeTimeline timeline2 = + new FakeTimeline( + new TimelineWindowDefinition( + /* isSeekable= */ true, /* isDynamic= */ true, /* durationUs= */ C.TIME_UNSET)); + FakeTimeline timeline3 = + new FakeTimeline( + new TimelineWindowDefinition( + /* isSeekable= */ true, /* isDynamic= */ true, /* durationUs= */ 10)); + + Timeline mergedTimeline = + prepareMergingMediaSource(/* clipDurations= */ false, timeline1, timeline2, timeline3); + + assertThat(mergedTimeline).isEqualTo(timeline1); } @Test - public void mergingStaticTimelines() throws IOException { - FakeTimeline firstTimeline = new FakeTimeline(new TimelineWindowDefinition(true, false, 20)); - FakeTimeline secondTimeline = new FakeTimeline(new TimelineWindowDefinition(true, false, 10)); - testMergingMediaSourcePrepare(firstTimeline, secondTimeline); + public void prepare_withDurationClipping_usesDurationOfShortestSource() throws IOException { + FakeTimeline timeline1 = + new FakeTimeline( + new TimelineWindowDefinition( + /* isSeekable= */ true, /* isDynamic= */ true, /* durationUs= */ 30)); + FakeTimeline timeline2 = + new FakeTimeline( + new TimelineWindowDefinition( + /* isSeekable= */ true, /* isDynamic= */ true, /* durationUs= */ C.TIME_UNSET)); + FakeTimeline timeline3 = + new FakeTimeline( + new TimelineWindowDefinition( + /* isSeekable= */ true, /* isDynamic= */ true, /* durationUs= */ 10)); + + Timeline mergedTimeline = + prepareMergingMediaSource(/* clipDurations= */ true, timeline1, timeline2, timeline3); + + assertThat(mergedTimeline).isEqualTo(timeline3); } @Test - public void mergingTimelinesWithDifferentPeriodCounts() throws IOException { - FakeTimeline firstTimeline = new FakeTimeline(new TimelineWindowDefinition(1, null)); - FakeTimeline secondTimeline = new FakeTimeline(new TimelineWindowDefinition(2, null)); - try { - testMergingMediaSourcePrepare(firstTimeline, secondTimeline); - fail("Expected merging to fail."); - } catch (IllegalMergeException e) { - assertThat(e.reason).isEqualTo(IllegalMergeException.REASON_PERIOD_COUNT_MISMATCH); - } + public void prepare_differentPeriodCounts_fails() throws IOException { + FakeTimeline firstTimeline = + new FakeTimeline(new TimelineWindowDefinition(/* periodCount= */ 1, /* id= */ 1)); + FakeTimeline secondTimeline = + new FakeTimeline(new TimelineWindowDefinition(/* periodCount= */ 2, /* id= */ 2)); + + IllegalMergeException exception = + assertThrows( + IllegalMergeException.class, + () -> + prepareMergingMediaSource( + /* clipDurations= */ false, firstTimeline, secondTimeline)); + assertThat(exception.reason).isEqualTo(IllegalMergeException.REASON_PERIOD_COUNT_MISMATCH); } @Test - public void mergingMediaSourcePeriodCreation() throws Exception { + public void createPeriod_createsChildPeriods() throws Exception { FakeMediaSource[] mediaSources = new FakeMediaSource[2]; for (int i = 0; i < mediaSources.length; i++) { mediaSources[i] = new FakeMediaSource(new FakeTimeline(/* windowCount= */ 2)); @@ -83,24 +113,26 @@ public void mergingMediaSourcePeriodCreation() throws Exception { } /** - * Wraps the specified timelines in a {@link MergingMediaSource}, prepares it and checks that it - * forwards the first of the wrapped timelines. + * Wraps the specified timelines in a {@link MergingMediaSource}, prepares it and returns the + * merged timeline. */ - private static void testMergingMediaSourcePrepare(Timeline... timelines) throws IOException { + private static Timeline prepareMergingMediaSource(boolean clipDurations, Timeline... timelines) + throws IOException { FakeMediaSource[] mediaSources = new FakeMediaSource[timelines.length]; for (int i = 0; i < timelines.length; i++) { mediaSources[i] = new FakeMediaSource(timelines[i]); } - MergingMediaSource mergingMediaSource = new MergingMediaSource(mediaSources); - MediaSourceTestRunner testRunner = new MediaSourceTestRunner(mergingMediaSource, null); + MergingMediaSource mergingMediaSource = + new MergingMediaSource(/* adjustPeriodTimeOffsets= */ false, clipDurations, mediaSources); + MediaSourceTestRunner testRunner = + new MediaSourceTestRunner(mergingMediaSource, /* allocator= */ null); try { Timeline timeline = testRunner.prepareSource(); - // The merged timeline should always be the one from the first child. - assertThat(timeline).isEqualTo(timelines[0]); testRunner.releaseSource(); for (FakeMediaSource mediaSource : mediaSources) { mediaSource.assertReleased(); } + return timeline; } finally { testRunner.release(); } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/ProgressiveMediaPeriodTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/ProgressiveMediaPeriodTest.java index ecdb43f1509..aaf00388f6e 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/ProgressiveMediaPeriodTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/ProgressiveMediaPeriodTest.java @@ -15,7 +15,7 @@ */ package com.google.android.exoplayer2.source; -import static com.google.android.exoplayer2.testutil.TestUtil.runMainLooperUntil; +import static com.google.android.exoplayer2.robolectric.RobolectricUtil.runMainLooperUntil; import static com.google.common.truth.Truth.assertThat; import android.net.Uri; @@ -49,7 +49,7 @@ public void prepare_updatesSourceInfoBeforeOnPreparedCallback() throws Exception Uri.parse("asset://android_asset/media/mp4/sample.mp4"), new AssetDataSource(ApplicationProvider.getApplicationContext()), () -> new Extractor[] {new Mp4Extractor()}, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher() .withParameters(/* windowIndex= */ 0, mediaPeriodId), new DefaultLoadErrorHandlingPolicy(), diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/SampleQueueTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/SampleQueueTest.java index 11a2204f81c..db9eee2ba2a 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/SampleQueueTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/SampleQueueTest.java @@ -203,6 +203,22 @@ public void readWithoutWrite() { assertNoSamplesToRead(null); } + @Test + public void peekConsumesDownstreamFormat() { + sampleQueue.format(FORMAT_1); + clearFormatHolderAndInputBuffer(); + int result = + sampleQueue.peek( + formatHolder, inputBuffer, /* formatRequired= */ false, /* loadingFinished= */ false); + assertThat(result).isEqualTo(RESULT_FORMAT_READ); + // formatHolder should be populated. + assertThat(formatHolder.format).isEqualTo(FORMAT_1); + result = + sampleQueue.peek( + formatHolder, inputBuffer, /* formatRequired= */ false, /* loadingFinished= */ false); + assertThat(result).isEqualTo(RESULT_NOTHING_READ); + } + @Test public void equalFormatsDeduplicated() { sampleQueue.format(FORMAT_1); @@ -1625,10 +1641,32 @@ private void assertReadSample( byte[] sampleData, int offset, int length) { + // Check that peeks yields the expected values. clearFormatHolderAndInputBuffer(); int result = + sampleQueue.peek( + formatHolder, inputBuffer, /* formatRequired= */ false, /* loadingFinished= */ false); + assertBufferReadResult( + result, timeUs, isKeyFrame, isDecodeOnly, isEncrypted, sampleData, offset, length); + + // Check that read yields the expected values. + clearFormatHolderAndInputBuffer(); + result = sampleQueue.read( formatHolder, inputBuffer, /* formatRequired= */ false, /* loadingFinished= */ false); + assertBufferReadResult( + result, timeUs, isKeyFrame, isDecodeOnly, isEncrypted, sampleData, offset, length); + } + + private void assertBufferReadResult( + int result, + long timeUs, + boolean isKeyFrame, + boolean isDecodeOnly, + boolean isEncrypted, + byte[] sampleData, + int offset, + int length) { assertThat(result).isEqualTo(RESULT_BUFFER_READ); // formatHolder should not be populated. assertThat(formatHolder.format).isNull(); diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/SinglePeriodTimelineTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/SinglePeriodTimelineTest.java index 4fce17e336b..09ac9b6df27 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/SinglePeriodTimelineTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/SinglePeriodTimelineTest.java @@ -71,7 +71,7 @@ public void getPeriodPositionDynamicWindowKnownDuration() { /* windowDefaultStartPositionUs= */ 0, /* isSeekable= */ false, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); // Should return null with a positive position projection beyond window duration. diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/ads/AdPlaybackStateTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/ads/AdPlaybackStateTest.java index 3a253b29761..de998bb8b16 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/ads/AdPlaybackStateTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/ads/AdPlaybackStateTest.java @@ -31,12 +31,13 @@ public final class AdPlaybackStateTest { private static final long[] TEST_AD_GROUP_TMES_US = new long[] {0, C.msToUs(10_000)}; private static final Uri TEST_URI = Uri.EMPTY; + private static final Object TEST_ADS_ID = new Object(); private AdPlaybackState state; @Before public void setUp() { - state = new AdPlaybackState(TEST_AD_GROUP_TMES_US); + state = new AdPlaybackState(TEST_ADS_ID, TEST_AD_GROUP_TMES_US); } @Test diff --git a/library/core/src/test/java/com/google/android/exoplayer2/source/ads/AdsMediaSourceTest.java b/library/core/src/test/java/com/google/android/exoplayer2/source/ads/AdsMediaSourceTest.java index 8395fcb1f4e..b5748aaa423 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/source/ads/AdsMediaSourceTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/source/ads/AdsMediaSourceTest.java @@ -38,6 +38,7 @@ import com.google.android.exoplayer2.source.ads.AdsLoader.EventListener; import com.google.android.exoplayer2.testutil.FakeMediaSource; import com.google.android.exoplayer2.upstream.Allocator; +import com.google.android.exoplayer2.upstream.DataSpec; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -57,7 +58,7 @@ public final class AdsMediaSourceTest { PREROLL_AD_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); private static final Object PREROLL_AD_PERIOD_UID = @@ -69,20 +70,23 @@ public final class AdsMediaSourceTest { CONTENT_DURATION_US, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, /* manifest= */ null, MediaItem.fromUri(Uri.EMPTY)); private static final Object CONTENT_PERIOD_UID = CONTENT_TIMELINE.getUidOfPeriod(/* periodIndex= */ 0); private static final AdPlaybackState AD_PLAYBACK_STATE = - new AdPlaybackState(/* adGroupTimesUs...= */ 0) + new AdPlaybackState(/* adsId= */ new Object(), /* adGroupTimesUs...= */ 0) .withContentDurationUs(CONTENT_DURATION_US) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdUri(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, Uri.EMPTY) .withPlayedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0) .withAdResumePositionUs(/* adResumePositionUs= */ 0); + private static final DataSpec TEST_ADS_DATA_SPEC = new DataSpec(Uri.EMPTY); + private static final Object TEST_ADS_ID = new Object(); + @Rule public final MockitoRule mockito = MockitoJUnit.rule(); private FakeMediaSource contentMediaSource; @@ -107,10 +111,21 @@ public void setUp() { ArgumentCaptor.forClass(AdsLoader.EventListener.class); adsMediaSource = new AdsMediaSource( - contentMediaSource, adMediaSourceFactory, mockAdsLoader, mockAdViewProvider); + contentMediaSource, + TEST_ADS_DATA_SPEC, + TEST_ADS_ID, + adMediaSourceFactory, + mockAdsLoader, + mockAdViewProvider); adsMediaSource.prepareSource(mockMediaSourceCaller, /* mediaTransferListener= */ null); shadowOf(Looper.getMainLooper()).idle(); - verify(mockAdsLoader).start(eventListenerArgumentCaptor.capture(), eq(mockAdViewProvider)); + verify(mockAdsLoader) + .start( + eq(adsMediaSource), + eq(TEST_ADS_DATA_SPEC), + eq(TEST_ADS_ID), + eq(mockAdViewProvider), + eventListenerArgumentCaptor.capture()); // Simulate loading a preroll ad. AdsLoader.EventListener adsLoaderEventListener = eventListenerArgumentCaptor.getValue(); diff --git a/library/core/src/test/java/com/google/android/exoplayer2/text/ssa/SsaDecoderTest.java b/library/core/src/test/java/com/google/android/exoplayer2/text/ssa/SsaDecoderTest.java index c7833fab04a..a734019d09b 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/text/ssa/SsaDecoderTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/text/ssa/SsaDecoderTest.java @@ -18,10 +18,13 @@ import static com.google.common.truth.Truth.assertThat; import static com.google.common.truth.Truth.assertWithMessage; +import android.graphics.Color; import android.text.Layout; +import android.text.Spanned; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.android.exoplayer2.testutil.truth.SpannedSubject; import com.google.android.exoplayer2.text.Cue; import com.google.android.exoplayer2.text.Subtitle; import com.google.common.collect.Iterables; @@ -44,6 +47,7 @@ public final class SsaDecoderTest { private static final String INVALID_TIMECODES = "media/ssa/invalid_timecodes"; private static final String INVALID_POSITIONS = "media/ssa/invalid_positioning"; private static final String POSITIONS_WITHOUT_PLAYRES = "media/ssa/positioning_without_playres"; + private static final String COLORS = "media/ssa/colors"; @Test public void decodeEmpty() throws IOException { @@ -267,6 +271,54 @@ public void decodeInvalidTimecodes() throws IOException { assertTypicalCue3(subtitle, 0); } + @Test + public void decodeColors() throws IOException { + SsaDecoder decoder = new SsaDecoder(); + byte[] bytes = TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), COLORS); + Subtitle subtitle = decoder.decode(bytes, bytes.length, false); + assertThat(subtitle.getEventTimeCount()).isEqualTo(14); + // &H000000FF (AABBGGRR) -> #FFFF0000 (AARRGGBB) + Spanned firstCueText = + (Spanned) Iterables.getOnlyElement(subtitle.getCues(subtitle.getEventTime(0))).text; + SpannedSubject.assertThat(firstCueText) + .hasForegroundColorSpanBetween(0, firstCueText.length()) + .withColor(Color.RED); + // &H0000FFFF (AABBGGRR) -> #FFFFFF00 (AARRGGBB) + Spanned secondCueText = + (Spanned) Iterables.getOnlyElement(subtitle.getCues(subtitle.getEventTime(2))).text; + SpannedSubject.assertThat(secondCueText) + .hasForegroundColorSpanBetween(0, secondCueText.length()) + .withColor(Color.YELLOW); + // &HFF00 (GGRR) -> #FF00FF00 (AARRGGBB) + Spanned thirdCueText = + (Spanned) Iterables.getOnlyElement(subtitle.getCues(subtitle.getEventTime(4))).text; + SpannedSubject.assertThat(thirdCueText) + .hasForegroundColorSpanBetween(0, thirdCueText.length()) + .withColor(Color.GREEN); + // &HA00000FF (AABBGGRR) -> #5FFF0000 (AARRGGBB) + Spanned fourthCueText = + (Spanned) Iterables.getOnlyElement(subtitle.getCues(subtitle.getEventTime(6))).text; + SpannedSubject.assertThat(fourthCueText) + .hasForegroundColorSpanBetween(0, fourthCueText.length()) + .withColor(0x5FFF0000); + // 16711680 (AABBGGRR) -> &H00FF0000 (AABBGGRR) -> #FF0000FF (AARRGGBB) + Spanned fifthCueText = + (Spanned) Iterables.getOnlyElement(subtitle.getCues(subtitle.getEventTime(8))).text; + SpannedSubject.assertThat(fifthCueText) + .hasForegroundColorSpanBetween(0, fifthCueText.length()) + .withColor(0xFF0000FF); + // 2164195328 (AABBGGRR) -> &H80FF0000 (AABBGGRR) -> #7F0000FF (AARRGGBB) + Spanned sixthCueText = + (Spanned) Iterables.getOnlyElement(subtitle.getCues(subtitle.getEventTime(10))).text; + SpannedSubject.assertThat(sixthCueText) + .hasForegroundColorSpanBetween(0, sixthCueText.length()) + .withColor(0x7F0000FF); + Spanned seventhCueText = + (Spanned) Iterables.getOnlyElement(subtitle.getCues(subtitle.getEventTime(12))).text; + SpannedSubject.assertThat(seventhCueText) + .hasNoForegroundColorSpanBetween(0, seventhCueText.length()); + } + private static void assertTypicalCue1(Subtitle subtitle, int eventIndex) { assertThat(subtitle.getEventTime(eventIndex)).isEqualTo(0); assertThat(subtitle.getCues(subtitle.getEventTime(eventIndex)).get(0).text.toString()) diff --git a/library/core/src/test/java/com/google/android/exoplayer2/trackselection/AdaptiveTrackSelectionTest.java b/library/core/src/test/java/com/google/android/exoplayer2/trackselection/AdaptiveTrackSelectionTest.java index a7a8e5a4c1c..aa6f420c3e2 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/trackselection/AdaptiveTrackSelectionTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/trackselection/AdaptiveTrackSelectionTest.java @@ -22,10 +22,15 @@ import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.source.MediaSource; import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.chunk.MediaChunkIterator; import com.google.android.exoplayer2.testutil.FakeClock; import com.google.android.exoplayer2.testutil.FakeMediaChunk; +import com.google.android.exoplayer2.testutil.FakeTimeline; +import com.google.android.exoplayer2.trackselection.AdaptiveTrackSelection.AdaptationCheckpoint; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection.Definition; import com.google.android.exoplayer2.upstream.BandwidthMeter; import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.collect.ImmutableList; @@ -312,7 +317,7 @@ public void updateSelectedTrack_usesFormatOfLastChunkInTheQueueForSelection() { trackGroup, mockBandwidthMeter, /* tracks= */ new int[] {0, 1}, - /* totalFixedTrackBandwidth= */ 0); + /* adaptationCheckpoints= */ ImmutableList.of()); // Make initial selection. when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(1000L); @@ -380,6 +385,199 @@ public void updateSelectedTrack_withQueueOfUnknownFormats_doesntThrow() { assertThat(adaptiveTrackSelection.getSelectedFormat()).isAnyOf(format1, format2); } + @Test + public void updateSelectedTrack_withAdaptationCheckpoints_usesOnlyAllocatedBandwidth() { + Format format0 = videoFormat(/* bitrate= */ 100, /* width= */ 160, /* height= */ 120); + Format format1 = videoFormat(/* bitrate= */ 500, /* width= */ 320, /* height= */ 240); + Format format2 = videoFormat(/* bitrate= */ 1000, /* width= */ 640, /* height= */ 480); + Format format3 = videoFormat(/* bitrate= */ 1500, /* width= */ 1024, /* height= */ 768); + TrackGroup trackGroup = new TrackGroup(format0, format1, format2, format3); + // Choose checkpoints relative to formats so that one is in the first range, one somewhere in + // the middle, and one needs to extrapolate beyond the last checkpoint. + List checkpoints = + ImmutableList.of( + new AdaptationCheckpoint(/* totalBandwidth= */ 0, /* allocatedBandwidth= */ 0), + new AdaptationCheckpoint(/* totalBandwidth= */ 1500, /* allocatedBandwidth= */ 750), + new AdaptationCheckpoint(/* totalBandwidth= */ 3000, /* allocatedBandwidth= */ 750), + new AdaptationCheckpoint(/* totalBandwidth= */ 4000, /* allocatedBandwidth= */ 1250), + new AdaptationCheckpoint(/* totalBandwidth= */ 5000, /* allocatedBandwidth= */ 1300)); + AdaptiveTrackSelection adaptiveTrackSelection = + prepareTrackSelection( + adaptiveTrackSelectionWithAdaptationCheckpoints(trackGroup, checkpoints)); + + // Ensure format0 is selected initially so that we can assert the upswitches. + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(1L); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 999_999_999_999L, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + /* mediaChunkIterators= */ THREE_EMPTY_MEDIA_CHUNK_ITERATORS); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format0); + + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(999L); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 999_999_999_999L, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + /* mediaChunkIterators= */ THREE_EMPTY_MEDIA_CHUNK_ITERATORS); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format0); + + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(1000L); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 999_999_999_999L, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + /* mediaChunkIterators= */ THREE_EMPTY_MEDIA_CHUNK_ITERATORS); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format1); + + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(2499L); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 999_999_999_999L, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + /* mediaChunkIterators= */ THREE_EMPTY_MEDIA_CHUNK_ITERATORS); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format1); + + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(3500L); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 999_999_999_999L, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + /* mediaChunkIterators= */ THREE_EMPTY_MEDIA_CHUNK_ITERATORS); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format2); + + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(8999L); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 999_999_999_999L, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + /* mediaChunkIterators= */ THREE_EMPTY_MEDIA_CHUNK_ITERATORS); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format2); + + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(9000L); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 999_999_999_999L, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + /* mediaChunkIterators= */ THREE_EMPTY_MEDIA_CHUNK_ITERATORS); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format3); + } + + @Test + public void + builderCreateTrackSelections_withSingleAdaptiveGroup_usesCorrectAdaptationCheckpoints() { + Format formatFixed1 = new Format.Builder().setAverageBitrate(500).build(); + Format formatFixed2 = new Format.Builder().setAverageBitrate(1000).build(); + Format formatAdaptive1 = new Format.Builder().setAverageBitrate(2000).build(); + Format formatAdaptive2 = new Format.Builder().setAverageBitrate(3000).build(); + Format formatAdaptive3 = new Format.Builder().setAverageBitrate(4000).build(); + Format formatAdaptive4 = new Format.Builder().setAverageBitrate(5000).build(); + TrackGroup trackGroupMultipleFixed = new TrackGroup(formatFixed1, formatFixed2); + TrackGroup trackGroupAdaptive = + new TrackGroup(formatAdaptive1, formatAdaptive2, formatAdaptive3, formatAdaptive4); + Definition definitionFixed1 = new Definition(trackGroupMultipleFixed, /* tracks...= */ 0); + Definition definitionFixed2 = new Definition(trackGroupMultipleFixed, /* tracks...= */ 1); + Definition definitionAdaptive = new Definition(trackGroupAdaptive, /* tracks...= */ 1, 2, 3); + List> checkPoints = new ArrayList<>(); + AdaptiveTrackSelection.Factory factory = + new AdaptiveTrackSelection.Factory() { + @Override + protected AdaptiveTrackSelection createAdaptiveTrackSelection( + TrackGroup group, + BandwidthMeter bandwidthMeter, + int[] tracks, + ImmutableList adaptationCheckpoints) { + checkPoints.add(adaptationCheckpoints); + return super.createAdaptiveTrackSelection( + group, bandwidthMeter, tracks, adaptationCheckpoints); + } + }; + + Timeline timeline = new FakeTimeline(); + factory.createTrackSelections( + new Definition[] {null, definitionFixed1, null, definitionFixed2, definitionAdaptive}, + mockBandwidthMeter, + new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(/* periodIndex= */ 0)), + timeline); + + assertThat(checkPoints).hasSize(1); + assertThat(checkPoints.get(0)) + .containsExactly( + new AdaptationCheckpoint(/* totalBandwidth= */ 0, /* allocatedBandwidth= */ 0), + new AdaptationCheckpoint(/* totalBandwidth= */ 4500, /* allocatedBandwidth= */ 3000), + new AdaptationCheckpoint(/* totalBandwidth= */ 5500, /* allocatedBandwidth= */ 4000), + new AdaptationCheckpoint(/* totalBandwidth= */ 6500, /* allocatedBandwidth= */ 5000), + new AdaptationCheckpoint(/* totalBandwidth= */ 11500, /* allocatedBandwidth= */ 10000)) + .inOrder(); + } + + @Test + public void + builderCreateTrackSelections_withMultipleAdaptiveGroups_usesCorrectAdaptationCheckpoints() { + Format group1Format1 = new Format.Builder().setAverageBitrate(500).build(); + Format group1Format2 = new Format.Builder().setAverageBitrate(1000).build(); + Format group2Format1 = new Format.Builder().setAverageBitrate(250).build(); + Format group2Format2 = new Format.Builder().setAverageBitrate(500).build(); + Format group2Format3 = new Format.Builder().setAverageBitrate(1250).build(); + Format group2UnusedFormat = new Format.Builder().setAverageBitrate(2000).build(); + Format fixedFormat = new Format.Builder().setAverageBitrate(5000).build(); + TrackGroup trackGroup1 = new TrackGroup(group1Format1, group1Format2); + TrackGroup trackGroup2 = + new TrackGroup(group2Format1, group2Format2, group2Format3, group2UnusedFormat); + TrackGroup fixedGroup = new TrackGroup(fixedFormat); + Definition definition1 = new Definition(trackGroup1, /* tracks...= */ 0, 1); + Definition definition2 = new Definition(trackGroup2, /* tracks...= */ 0, 1, 2); + Definition fixedDefinition = new Definition(fixedGroup, /* tracks...= */ 0); + List> checkPoints = new ArrayList<>(); + AdaptiveTrackSelection.Factory factory = + new AdaptiveTrackSelection.Factory() { + @Override + protected AdaptiveTrackSelection createAdaptiveTrackSelection( + TrackGroup group, + BandwidthMeter bandwidthMeter, + int[] tracks, + ImmutableList adaptationCheckpoints) { + checkPoints.add(adaptationCheckpoints); + return super.createAdaptiveTrackSelection( + group, bandwidthMeter, tracks, adaptationCheckpoints); + } + }; + + Timeline timeline = new FakeTimeline(); + factory.createTrackSelections( + new Definition[] {null, definition1, fixedDefinition, definition2, null}, + mockBandwidthMeter, + new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(/* periodIndex= */ 0)), + timeline); + + assertThat(checkPoints).hasSize(2); + assertThat(checkPoints.get(0)) + .containsExactly( + new AdaptationCheckpoint(/* totalBandwidth= */ 0, /* allocatedBandwidth= */ 0), + new AdaptationCheckpoint(/* totalBandwidth= */ 5750, /* allocatedBandwidth= */ 500), + new AdaptationCheckpoint(/* totalBandwidth= */ 6000, /* allocatedBandwidth= */ 500), + new AdaptationCheckpoint(/* totalBandwidth= */ 6500, /* allocatedBandwidth= */ 1000), + new AdaptationCheckpoint(/* totalBandwidth= */ 7250, /* allocatedBandwidth= */ 1000), + new AdaptationCheckpoint(/* totalBandwidth= */ 9500, /* allocatedBandwidth= */ 2000)) + .inOrder(); + assertThat(checkPoints.get(1)) + .containsExactly( + new AdaptationCheckpoint(/* totalBandwidth= */ 0, /* allocatedBandwidth= */ 0), + new AdaptationCheckpoint(/* totalBandwidth= */ 5750, /* allocatedBandwidth= */ 250), + new AdaptationCheckpoint(/* totalBandwidth= */ 6000, /* allocatedBandwidth= */ 500), + new AdaptationCheckpoint(/* totalBandwidth= */ 6500, /* allocatedBandwidth= */ 500), + new AdaptationCheckpoint(/* totalBandwidth= */ 7250, /* allocatedBandwidth= */ 1250), + new AdaptationCheckpoint(/* totalBandwidth= */ 9500, /* allocatedBandwidth= */ 2500)) + .inOrder(); + } + private AdaptiveTrackSelection adaptiveTrackSelection(TrackGroup trackGroup) { return adaptiveTrackSelectionWithMinDurationForQualityIncreaseMs( trackGroup, AdaptiveTrackSelection.DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS); @@ -392,12 +590,12 @@ private AdaptiveTrackSelection adaptiveTrackSelectionWithMinDurationForQualityIn trackGroup, selectedAllTracksInGroup(trackGroup), mockBandwidthMeter, - /* reservedBandwidth= */ 0, minDurationForQualityIncreaseMs, AdaptiveTrackSelection.DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, AdaptiveTrackSelection.DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, /* bandwidthFraction= */ 1.0f, AdaptiveTrackSelection.DEFAULT_BUFFERED_FRACTION_TO_LIVE_EDGE_FOR_QUALITY_INCREASE, + /* adaptationCheckpoints= */ ImmutableList.of(), fakeClock)); } @@ -408,12 +606,12 @@ private AdaptiveTrackSelection adaptiveTrackSelectionWithMaxDurationForQualityDe trackGroup, selectedAllTracksInGroup(trackGroup), mockBandwidthMeter, - /* reservedBandwidth= */ 0, AdaptiveTrackSelection.DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, maxDurationForQualityDecreaseMs, AdaptiveTrackSelection.DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, /* bandwidthFraction= */ 1.0f, AdaptiveTrackSelection.DEFAULT_BUFFERED_FRACTION_TO_LIVE_EDGE_FOR_QUALITY_INCREASE, + /* adaptationCheckpoints= */ ImmutableList.of(), fakeClock)); } @@ -424,12 +622,28 @@ private AdaptiveTrackSelection adaptiveTrackSelectionWithMinTimeBetweenBufferRee trackGroup, selectedAllTracksInGroup(trackGroup), mockBandwidthMeter, - /* reservedBandwidth= */ 0, AdaptiveTrackSelection.DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, AdaptiveTrackSelection.DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, durationToRetainAfterDiscardMs, /* bandwidthFraction= */ 1.0f, AdaptiveTrackSelection.DEFAULT_BUFFERED_FRACTION_TO_LIVE_EDGE_FOR_QUALITY_INCREASE, + /* adaptationCheckpoints= */ ImmutableList.of(), + fakeClock)); + } + + private AdaptiveTrackSelection adaptiveTrackSelectionWithAdaptationCheckpoints( + TrackGroup trackGroup, List adaptationCheckpoints) { + return prepareTrackSelection( + new AdaptiveTrackSelection( + trackGroup, + selectedAllTracksInGroup(trackGroup), + mockBandwidthMeter, + AdaptiveTrackSelection.DEFAULT_MIN_DURATION_FOR_QUALITY_INCREASE_MS, + AdaptiveTrackSelection.DEFAULT_MAX_DURATION_FOR_QUALITY_DECREASE_MS, + AdaptiveTrackSelection.DEFAULT_MIN_DURATION_TO_RETAIN_AFTER_DISCARD_MS, + /* bandwidthFraction= */ 1.0f, + AdaptiveTrackSelection.DEFAULT_BUFFERED_FRACTION_TO_LIVE_EDGE_FOR_QUALITY_INCREASE, + adaptationCheckpoints, fakeClock)); } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelectorTest.java b/library/core/src/test/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelectorTest.java index 89dd62e9a6d..9aebfb7718e 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelectorTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/trackselection/DefaultTrackSelectorTest.java @@ -15,10 +15,11 @@ */ package com.google.android.exoplayer2.trackselection; +import static com.google.android.exoplayer2.C.FORMAT_EXCEEDS_CAPABILITIES; +import static com.google.android.exoplayer2.C.FORMAT_HANDLED; +import static com.google.android.exoplayer2.C.FORMAT_UNSUPPORTED_SUBTYPE; import static com.google.android.exoplayer2.RendererCapabilities.ADAPTIVE_NOT_SEAMLESS; -import static com.google.android.exoplayer2.RendererCapabilities.FORMAT_EXCEEDS_CAPABILITIES; -import static com.google.android.exoplayer2.RendererCapabilities.FORMAT_HANDLED; -import static com.google.android.exoplayer2.RendererCapabilities.FORMAT_UNSUPPORTED_SUBTYPE; +import static com.google.android.exoplayer2.RendererCapabilities.TUNNELING_NOT_SUPPORTED; import static com.google.android.exoplayer2.RendererConfiguration.DEFAULT; import static com.google.common.truth.Truth.assertThat; import static org.mockito.Mockito.never; @@ -50,6 +51,7 @@ import com.google.android.exoplayer2.upstream.BandwidthMeter; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.ImmutableList; import java.util.HashMap; import java.util.Map; import org.junit.Before; @@ -67,7 +69,8 @@ public final class DefaultTrackSelectorTest { private static final RendererCapabilities ALL_TEXT_FORMAT_SUPPORTED_RENDERER_CAPABILITIES = new FakeRendererCapabilities(C.TRACK_TYPE_TEXT); private static final RendererCapabilities ALL_AUDIO_FORMAT_EXCEEDED_RENDERER_CAPABILITIES = - new FakeRendererCapabilities(C.TRACK_TYPE_AUDIO, FORMAT_EXCEEDS_CAPABILITIES); + new FakeRendererCapabilities( + C.TRACK_TYPE_AUDIO, RendererCapabilities.create(FORMAT_EXCEEDS_CAPABILITIES)); private static final RendererCapabilities VIDEO_CAPABILITIES = new FakeRendererCapabilities(C.TRACK_TYPE_VIDEO); @@ -91,6 +94,7 @@ public final class DefaultTrackSelectorTest { .setSampleMimeType(MimeTypes.AUDIO_AAC) .setChannelCount(2) .setSampleRate(44100) + .setAverageBitrate(128000) .build(); private static final Format TEXT_FORMAT = new Format.Builder().setSampleMimeType(MimeTypes.TEXT_VTT).build(); @@ -106,7 +110,7 @@ public final class DefaultTrackSelectorTest { private static final TrackSelection[] TRACK_SELECTIONS_WITH_NO_SAMPLE_RENDERER = new TrackSelection[] {new FixedTrackSelection(VIDEO_TRACK_GROUP, 0), null}; - private static final Timeline TIMELINE = new FakeTimeline(/* windowCount= */ 1); + private static final Timeline TIMELINE = new FakeTimeline(); private static MediaPeriodId periodId; @@ -319,7 +323,7 @@ public void selectTracksSelectTrackWithSelectionFlag() throws Exception { trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, formatWithSelectionFlag); + assertFixedSelection(result.selections[0], trackGroups, formatWithSelectionFlag); } /** Tests that adaptive audio track selections respect the maximum audio bitrate. */ @@ -337,25 +341,25 @@ public void selectAdaptiveAudioTrackGroupWithMaxBitrate() throws ExoPlaybackExce TrackSelectorResult result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 2, 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 2, 0, 1); trackSelector.setParameters( trackSelector.buildUponParameters().setMaxAudioBitrate(256 * 1024 - 1)); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 1); trackSelector.setParameters(trackSelector.buildUponParameters().setMaxAudioBitrate(192 * 1024)); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 1); trackSelector.setParameters( trackSelector.buildUponParameters().setMaxAudioBitrate(192 * 1024 - 1)); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups.get(0), 1); + assertFixedSelection(result.selections[0], trackGroups.get(0), 1); trackSelector.setParameters(trackSelector.buildUponParameters().setMaxAudioBitrate(10)); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups.get(0), 1); + assertFixedSelection(result.selections[0], trackGroups.get(0), 1); } /** @@ -376,7 +380,76 @@ public void selectTracksSelectPreferredAudioLanguage() throws Exception { wrapFormats(frAudioFormat, enAudioFormat), periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, enAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, enAudioFormat); + } + + /** + * Tests that track selector will select audio track with the highest number of matching role + * flags given by {@link Parameters}. + */ + @Test + public void selectTracks_withPreferredAudioRoleFlags_selectPreferredTrack() throws Exception { + Format.Builder formatBuilder = AUDIO_FORMAT.buildUpon(); + Format noRoleFlags = formatBuilder.build(); + Format lessRoleFlags = formatBuilder.setRoleFlags(C.ROLE_FLAG_CAPTION).build(); + Format moreRoleFlags = + formatBuilder + .setRoleFlags(C.ROLE_FLAG_CAPTION | C.ROLE_FLAG_COMMENTARY | C.ROLE_FLAG_DUB) + .build(); + TrackGroupArray trackGroups = wrapFormats(noRoleFlags, moreRoleFlags, lessRoleFlags); + + trackSelector.setParameters( + defaultParameters + .buildUpon() + .setPreferredAudioRoleFlags(C.ROLE_FLAG_CAPTION | C.ROLE_FLAG_COMMENTARY)); + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {ALL_AUDIO_FORMAT_SUPPORTED_RENDERER_CAPABILITIES}, + trackGroups, + periodId, + TIMELINE); + assertFixedSelection(result.selections[0], trackGroups, moreRoleFlags); + } + + /** + * Tests that track selector with select default audio track if no role flag preference is + * specified by {@link Parameters}. + */ + @Test + public void selectTracks_withoutPreferredAudioRoleFlags_selectsDefaultTrack() throws Exception { + Format firstFormat = AUDIO_FORMAT; + Format defaultFormat = + AUDIO_FORMAT.buildUpon().setSelectionFlags(C.SELECTION_FLAG_DEFAULT).build(); + Format roleFlagFormat = AUDIO_FORMAT.buildUpon().setRoleFlags(C.ROLE_FLAG_CAPTION).build(); + TrackGroupArray trackGroups = wrapFormats(firstFormat, defaultFormat, roleFlagFormat); + + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {ALL_AUDIO_FORMAT_SUPPORTED_RENDERER_CAPABILITIES}, + trackGroups, + periodId, + TIMELINE); + assertFixedSelection(result.selections[0], trackGroups, defaultFormat); + } + + /** + * Tests that track selector with select the first audio track if no role flag preference is + * specified by {@link Parameters} and no default track exists. + */ + @Test + public void selectTracks_withoutPreferredAudioRoleFlagsOrDefaultTrack_selectsFirstTrack() + throws Exception { + Format firstFormat = AUDIO_FORMAT; + Format roleFlagFormat = AUDIO_FORMAT.buildUpon().setRoleFlags(C.ROLE_FLAG_CAPTION).build(); + TrackGroupArray trackGroups = wrapFormats(firstFormat, roleFlagFormat); + + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {ALL_AUDIO_FORMAT_SUPPORTED_RENDERER_CAPABILITIES}, + trackGroups, + periodId, + TIMELINE); + assertFixedSelection(result.selections[0], trackGroups, firstFormat); } /** @@ -398,7 +471,7 @@ public void selectTracksSelectPreferredAudioLanguageOverSelectionFlag() throws E trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, enNonDefaultFormat); + assertFixedSelection(result.selections[0], trackGroups, enNonDefaultFormat); } /** @@ -424,7 +497,7 @@ public void selectTracksPreferTrackWithinCapabilities() throws Exception { trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, supportedFormat); + assertFixedSelection(result.selections[0], trackGroups, supportedFormat); } /** @@ -442,7 +515,7 @@ public void selectTracksWithNoTrackWithinCapabilitiesSelectExceededCapabilityTra trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, AUDIO_FORMAT); + assertFixedSelection(result.selections[0], trackGroups, AUDIO_FORMAT); } /** @@ -463,7 +536,7 @@ public void selectTracksWithNoTrackWithinCapabilitiesAndSetByParamsReturnNoSelec trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); + assertNoSelection(result.selections[0]); } /** @@ -490,7 +563,7 @@ public void selectTracksPreferTrackWithinCapabilitiesOverSelectionFlag() throws trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, supportedFormat); + assertFixedSelection(result.selections[0], trackGroups, supportedFormat); } /** @@ -518,7 +591,7 @@ public void selectTracksPreferTrackWithinCapabilitiesOverPreferredLanguage() thr trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, supportedFrFormat); + assertFixedSelection(result.selections[0], trackGroups, supportedFrFormat); } /** @@ -553,7 +626,7 @@ public void selectTracksPreferTrackWithinCapabilitiesOverSelectionFlagAndPreferr trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, supportedFrFormat); + assertFixedSelection(result.selections[0], trackGroups, supportedFrFormat); } /** @@ -573,7 +646,7 @@ public void selectTracksWithinCapabilitiesSelectHigherNumChannel() throws Except trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, higherChannelFormat); + assertFixedSelection(result.selections[0], trackGroups, higherChannelFormat); } /** @@ -593,7 +666,7 @@ public void selectTracksWithinCapabilitiesSelectHigherSampleRate() throws Except trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, higherSampleRateFormat); + assertFixedSelection(result.selections[0], trackGroups, higherSampleRateFormat); } /** @@ -614,7 +687,7 @@ public void selectAudioTracks_withinCapabilities_andSameLanguage_selectsHigherBi trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, higherBitrateFormat); + assertFixedSelection(result.selections[0], trackGroups, higherBitrateFormat); } /** @@ -636,7 +709,7 @@ public void selectAudioTracks_withinCapabilities_andDifferentLanguage_selectsFir trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, firstLanguageFormat); + assertFixedSelection(result.selections[0], trackGroups, firstLanguageFormat); } /** @@ -660,7 +733,7 @@ public void selectTracksPreferHigherNumChannelBeforeSampleRate() throws Exceptio trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, higherChannelLowerSampleRateFormat); + assertFixedSelection(result.selections[0], trackGroups, higherChannelLowerSampleRateFormat); } /** @@ -683,7 +756,7 @@ public void selectTracksPreferHigherSampleRateBeforeBitrate() throws Exception { trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, higherSampleRateLowerBitrateFormat); + assertFixedSelection(result.selections[0], trackGroups, higherSampleRateLowerBitrateFormat); } /** @@ -703,7 +776,7 @@ public void selectTracksExceedingCapabilitiesSelectLowerNumChannel() throws Exce trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, lowerChannelFormat); + assertFixedSelection(result.selections[0], trackGroups, lowerChannelFormat); } /** @@ -723,7 +796,7 @@ public void selectTracksExceedingCapabilitiesSelectLowerSampleRate() throws Exce trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, lowerSampleRateFormat); + assertFixedSelection(result.selections[0], trackGroups, lowerSampleRateFormat); } /** @@ -743,7 +816,7 @@ public void selectTracksExceedingCapabilitiesSelectLowerBitrate() throws Excepti trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, lowerBitrateFormat); + assertFixedSelection(result.selections[0], trackGroups, lowerBitrateFormat); } /** @@ -768,7 +841,7 @@ public void selectTracksExceedingCapabilitiesPreferLowerNumChannelBeforeSampleRa trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, lowerChannelHigherSampleRateFormat); + assertFixedSelection(result.selections[0], trackGroups, lowerChannelHigherSampleRateFormat); } /** @@ -792,7 +865,7 @@ public void selectTracksExceedingCapabilitiesPreferLowerSampleRateBeforeBitrate( trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, lowerSampleRateHigherBitrateFormat); + assertFixedSelection(result.selections[0], trackGroups, lowerSampleRateHigherBitrateFormat); } /** Tests text track selection flags. */ @@ -812,12 +885,12 @@ public void textTrackSelectionFlags() throws ExoPlaybackException { TrackGroupArray trackGroups = wrapFormats(forcedOnly, forcedDefault, defaultOnly, noFlag); TrackSelectorResult result = trackSelector.selectTracks(textRendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, forcedDefault); + assertFixedSelection(result.selections[0], trackGroups, forcedDefault); // Ditto. trackGroups = wrapFormats(forcedOnly, noFlag, defaultOnly); result = trackSelector.selectTracks(textRendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, defaultOnly); + assertFixedSelection(result.selections[0], trackGroups, defaultOnly); // Default flags are disabled and no language preference is provided, so no text track is // selected. @@ -825,7 +898,7 @@ public void textTrackSelectionFlags() throws ExoPlaybackException { trackSelector.setParameters( defaultParameters.buildUpon().setDisabledTextTrackSelectionFlags(C.SELECTION_FLAG_DEFAULT)); result = trackSelector.selectTracks(textRendererCapabilities, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); + assertNoSelection(result.selections[0]); // All selection flags are disabled and there is no language preference, so nothing should be // selected. @@ -837,13 +910,13 @@ public void textTrackSelectionFlags() throws ExoPlaybackException { .setDisabledTextTrackSelectionFlags( C.SELECTION_FLAG_DEFAULT | C.SELECTION_FLAG_FORCED)); result = trackSelector.selectTracks(textRendererCapabilities, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); + assertNoSelection(result.selections[0]); // There is a preferred language, so a language-matching track flagged as default should // be selected, and the one without forced flag should be preferred. trackSelector.setParameters(defaultParameters.buildUpon().setPreferredTextLanguage("eng")); result = trackSelector.selectTracks(textRendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, defaultOnly); + assertFixedSelection(result.selections[0], trackGroups, defaultOnly); // Same as above, but the default flag is disabled. If multiple tracks match the preferred // language, those not flagged as forced are preferred, as they likely include the contents of @@ -855,7 +928,7 @@ public void textTrackSelectionFlags() throws ExoPlaybackException { .buildUpon() .setDisabledTextTrackSelectionFlags(C.SELECTION_FLAG_DEFAULT)); result = trackSelector.selectTracks(textRendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, noFlag); + assertFixedSelection(result.selections[0], trackGroups, noFlag); } /** @@ -884,23 +957,23 @@ public void selectingForcedTextTrackMatchesAudioLanguage() throws ExoPlaybackExc TrackGroupArray trackGroups = wrapFormats(noLanguageAudio, forcedNoLanguage); TrackSelectorResult result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(1), trackGroups, forcedNoLanguage); + assertFixedSelection(result.selections[1], trackGroups, forcedNoLanguage); // No forced text track should be selected because none of the forced text tracks' languages // matches the selected audio language. trackGroups = wrapFormats(noLanguageAudio, forcedEnglish, forcedGerman); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(1)); + assertNoSelection(result.selections[1]); // The audio declares german. The german forced track should be selected. trackGroups = wrapFormats(germanAudio, forcedGerman, forcedEnglish); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(1), trackGroups, forcedGerman); + assertFixedSelection(result.selections[1], trackGroups, forcedGerman); // Ditto trackGroups = wrapFormats(germanAudio, forcedEnglish, forcedGerman); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(1), trackGroups, forcedGerman); + assertFixedSelection(result.selections[1], trackGroups, forcedGerman); } /** @@ -922,34 +995,34 @@ public void selectUndeterminedTextLanguageAsFallback() throws ExoPlaybackExcepti TrackGroupArray trackGroups = wrapFormats(spanish, german, undeterminedUnd, undeterminedNull); TrackSelectorResult result = trackSelector.selectTracks(textRendererCapabilites, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); + assertNoSelection(result.selections[0]); trackSelector.setParameters( defaultParameters.buildUpon().setSelectUndeterminedTextLanguage(true)); result = trackSelector.selectTracks(textRendererCapabilites, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, undeterminedUnd); + assertFixedSelection(result.selections[0], trackGroups, undeterminedUnd); ParametersBuilder builder = defaultParameters.buildUpon().setPreferredTextLanguage("spa"); trackSelector.setParameters(builder); result = trackSelector.selectTracks(textRendererCapabilites, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, spanish); + assertFixedSelection(result.selections[0], trackGroups, spanish); trackGroups = wrapFormats(german, undeterminedUnd, undeterminedNull); result = trackSelector.selectTracks(textRendererCapabilites, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); + assertNoSelection(result.selections[0]); trackSelector.setParameters(builder.setSelectUndeterminedTextLanguage(true)); result = trackSelector.selectTracks(textRendererCapabilites, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, undeterminedUnd); + assertFixedSelection(result.selections[0], trackGroups, undeterminedUnd); trackGroups = wrapFormats(german, undeterminedNull); result = trackSelector.selectTracks(textRendererCapabilites, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, undeterminedNull); + assertFixedSelection(result.selections[0], trackGroups, undeterminedNull); trackGroups = wrapFormats(german); result = trackSelector.selectTracks(textRendererCapabilites, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); + assertNoSelection(result.selections[0]); } /** Tests audio track selection when there are multiple audio renderers. */ @@ -980,20 +1053,20 @@ public void selectPreferredTextTrackMultipleRenderers() throws Exception { // Without an explicit language preference, nothing should be selected. TrackSelectorResult result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); - assertNoSelection(result.selections.get(1)); + assertNoSelection(result.selections[0]); + assertNoSelection(result.selections[1]); // Explicit language preference for english. First renderer should be used. trackSelector.setParameters(defaultParameters.buildUpon().setPreferredTextLanguage("en")); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, english); - assertNoSelection(result.selections.get(1)); + assertFixedSelection(result.selections[0], trackGroups, english); + assertNoSelection(result.selections[1]); // Explicit language preference for German. Second renderer should be used. trackSelector.setParameters(defaultParameters.buildUpon().setPreferredTextLanguage("de")); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); - assertFixedSelection(result.selections.get(1), trackGroups, german); + assertNoSelection(result.selections[0]); + assertFixedSelection(result.selections[1], trackGroups, german); } /** @@ -1025,7 +1098,7 @@ public void selectTracksWithinCapabilitiesAndForceLowestBitrateSelectLowerBitrat trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, lowerBitrateFormat); + assertFixedSelection(result.selections[0], trackGroups, lowerBitrateFormat); } /** @@ -1057,7 +1130,7 @@ public void selectTracksWithinCapabilitiesAndForceHighestBitrateSelectHigherBitr trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, higherBitrateFormat); + assertFixedSelection(result.selections[0], trackGroups, higherBitrateFormat); } @Test @@ -1070,7 +1143,7 @@ public void selectTracksWithMultipleAudioTracks() throws Exception { new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 1); } @Test @@ -1101,7 +1174,22 @@ public void selectTracks_multipleAudioTracks_selectsAllTracksInBestConfiguration new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 6); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 6); + } + + @Test + public void selectTracks_multipleAudioTracksWithoutBitrate_onlySelectsSingleTrack() + throws Exception { + TrackGroupArray trackGroups = + singleTrackGroup( + AUDIO_FORMAT.buildUpon().setId("0").setAverageBitrate(Format.NO_VALUE).build(), + AUDIO_FORMAT.buildUpon().setId("1").setAverageBitrate(Format.NO_VALUE).build()); + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups.get(0), /* expectedTrack= */ 0); } @Test @@ -1118,7 +1206,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedSampleRates() throws Exc trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, highSampleRateAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, highSampleRateAudioFormat); // The same applies if the tracks are provided in the opposite order. trackGroups = singleTrackGroup(lowSampleRateAudioFormat, highSampleRateAudioFormat); @@ -1126,7 +1214,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedSampleRates() throws Exc trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, highSampleRateAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, highSampleRateAudioFormat); // If we explicitly enable mixed sample rate adaptiveness, expect an adaptive selection. trackSelector.setParameters( @@ -1135,7 +1223,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedSampleRates() throws Exc trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 1); } @Test @@ -1151,7 +1239,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedMimeTypes() throws Excep trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, aacAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, aacAudioFormat); // The same applies if the tracks are provided in the opposite order. trackGroups = singleTrackGroup(opusAudioFormat, aacAudioFormat); @@ -1159,7 +1247,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedMimeTypes() throws Excep trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, opusAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, opusAudioFormat); // If we explicitly enable mixed mime type adaptiveness, expect an adaptive selection. trackSelector.setParameters( @@ -1168,7 +1256,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedMimeTypes() throws Excep trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 1); } @Test @@ -1184,7 +1272,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedChannelCounts() throws E trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, surroundAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, surroundAudioFormat); // The same applies if the tracks are provided in the opposite order. trackGroups = singleTrackGroup(surroundAudioFormat, stereoAudioFormat); @@ -1192,7 +1280,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedChannelCounts() throws E trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, surroundAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, surroundAudioFormat); // If we constrain the channel count to 4 we expect a fixed selection containing the track with // fewer channels. @@ -1201,7 +1289,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedChannelCounts() throws E trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, stereoAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, stereoAudioFormat); // If we constrain the channel count to 2 we expect a fixed selection containing the track with // fewer channels. @@ -1210,7 +1298,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedChannelCounts() throws E trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, stereoAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, stereoAudioFormat); // If we constrain the channel count to 1 we expect a fixed selection containing the track with // fewer channels. @@ -1219,7 +1307,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedChannelCounts() throws E trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, stereoAudioFormat); + assertFixedSelection(result.selections[0], trackGroups, stereoAudioFormat); // If we disable exceeding of constraints we expect no selection. trackSelector.setParameters( @@ -1231,7 +1319,7 @@ public void selectTracksWithMultipleAudioTracksWithMixedChannelCounts() throws E trackSelector.selectTracks( new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertNoSelection(result.selections.get(0)); + assertNoSelection(result.selections[0]); } @Test @@ -1255,7 +1343,7 @@ public void selectTracksWithMultipleAudioTracksOverrideReturnsAdaptiveTrackSelec new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 1, 2); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 1, 2); } /** Tests audio track selection when there are multiple audio renderers. */ @@ -1286,20 +1374,20 @@ public void selectPreferredAudioTrackMultipleRenderers() throws Exception { TrackGroupArray trackGroups = wrapFormats(english, german); TrackSelectorResult result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, english); - assertNoSelection(result.selections.get(1)); + assertFixedSelection(result.selections[0], trackGroups, english); + assertNoSelection(result.selections[1]); // Explicit language preference for english. First renderer should be used. trackSelector.setParameters(defaultParameters.buildUpon().setPreferredAudioLanguage("en")); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertFixedSelection(result.selections.get(0), trackGroups, english); - assertNoSelection(result.selections.get(1)); + assertFixedSelection(result.selections[0], trackGroups, english); + assertNoSelection(result.selections[1]); // Explicit language preference for German. Second renderer should be used. trackSelector.setParameters(defaultParameters.buildUpon().setPreferredAudioLanguage("de")); result = trackSelector.selectTracks(rendererCapabilities, trackGroups, periodId, TIMELINE); - assertNoSelection(result.selections.get(0)); - assertFixedSelection(result.selections.get(1), trackGroups, german); + assertNoSelection(result.selections[0]); + assertFixedSelection(result.selections[1], trackGroups, german); } @Test @@ -1312,13 +1400,16 @@ public void selectTracksWithMultipleVideoTracks() throws Exception { new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 1); } @Test public void selectTracksWithMultipleVideoTracksWithNonSeamlessAdaptiveness() throws Exception { FakeRendererCapabilities nonSeamlessVideoCapabilities = - new FakeRendererCapabilities(C.TRACK_TYPE_VIDEO, FORMAT_HANDLED | ADAPTIVE_NOT_SEAMLESS); + new FakeRendererCapabilities( + C.TRACK_TYPE_VIDEO, + RendererCapabilities.create( + FORMAT_HANDLED, ADAPTIVE_NOT_SEAMLESS, TUNNELING_NOT_SUPPORTED)); // Should do non-seamless adaptiveness by default, so expect an adaptive selection. Format.Builder formatBuilder = VIDEO_FORMAT.buildUpon(); @@ -1333,7 +1424,7 @@ public void selectTracksWithMultipleVideoTracksWithNonSeamlessAdaptiveness() thr periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 1); // If we explicitly disable non-seamless adaptiveness, expect a fixed selection. trackSelector.setParameters( @@ -1345,7 +1436,7 @@ public void selectTracksWithMultipleVideoTracksWithNonSeamlessAdaptiveness() thr periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups.get(0), 0); + assertFixedSelection(result.selections[0], trackGroups.get(0), 0); } @Test @@ -1361,7 +1452,7 @@ public void selectTracksWithMultipleVideoTracksWithMixedMimeTypes() throws Excep trackSelector.selectTracks( new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, h264VideoFormat); + assertFixedSelection(result.selections[0], trackGroups, h264VideoFormat); // The same applies if the tracks are provided in the opposite order. trackGroups = singleTrackGroup(h265VideoFormat, h264VideoFormat); @@ -1369,7 +1460,7 @@ public void selectTracksWithMultipleVideoTracksWithMixedMimeTypes() throws Excep trackSelector.selectTracks( new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertFixedSelection(result.selections.get(0), trackGroups, h265VideoFormat); + assertFixedSelection(result.selections[0], trackGroups, h265VideoFormat); // If we explicitly enable mixed mime type adaptiveness, expect an adaptive selection. trackSelector.setParameters( @@ -1378,7 +1469,7 @@ public void selectTracksWithMultipleVideoTracksWithMixedMimeTypes() throws Excep trackSelector.selectTracks( new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 0, 1); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 0, 1); } @Test @@ -1402,13 +1493,147 @@ public void selectTracksWithMultipleVideoTracksOverrideReturnsAdaptiveTrackSelec new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); assertThat(result.length).isEqualTo(1); - assertAdaptiveSelection(result.selections.get(0), trackGroups.get(0), 1, 2); + assertAdaptiveSelection(result.selections[0], trackGroups.get(0), 1, 2); + } + + @Test + public void selectTracks_multipleVideoAndAudioTracks() throws Exception { + Format videoFormat1 = VIDEO_FORMAT.buildUpon().setAverageBitrate(1000).build(); + Format videoFormat2 = VIDEO_FORMAT.buildUpon().setAverageBitrate(2000).build(); + Format audioFormat1 = AUDIO_FORMAT.buildUpon().setAverageBitrate(100).build(); + Format audioFormat2 = AUDIO_FORMAT.buildUpon().setAverageBitrate(200).build(); + TrackGroupArray trackGroups = + new TrackGroupArray( + new TrackGroup(videoFormat1, videoFormat2), new TrackGroup(audioFormat1, audioFormat2)); + + // Multiple adaptive selections allowed. + trackSelector.setParameters( + trackSelector.buildUponParameters().setAllowMultipleAdaptiveSelections(true)); + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {VIDEO_CAPABILITIES, AUDIO_CAPABILITIES}, + trackGroups, + periodId, + TIMELINE); + + assertThat(result.length).isEqualTo(2); + assertAdaptiveSelection( + result.selections[0], trackGroups.get(0), /* expectedTracks...= */ 1, 0); + assertAdaptiveSelection( + result.selections[1], trackGroups.get(1), /* expectedTracks...= */ 1, 0); + + // Multiple adaptive selection disallowed. + trackSelector.setParameters( + trackSelector.buildUponParameters().setAllowMultipleAdaptiveSelections(false)); + result = + trackSelector.selectTracks( + new RendererCapabilities[] {VIDEO_CAPABILITIES, AUDIO_CAPABILITIES}, + trackGroups, + periodId, + TIMELINE); + + assertThat(result.length).isEqualTo(2); + assertAdaptiveSelection( + result.selections[0], trackGroups.get(0), /* expectedTracks...= */ 1, 0); + assertFixedSelection(result.selections[1], trackGroups.get(1), /* expectedTrack= */ 1); + } + + @Test + public void selectTracks_withPreferredVideoMimeTypes_selectsTrackWithPreferredMimeType() + throws Exception { + Format formatAv1 = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_AV1).build(); + Format formatVp9 = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_VP9).build(); + Format formatH264 = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(); + TrackGroupArray trackGroups = wrapFormats(formatAv1, formatVp9, formatH264); + + trackSelector.setParameters( + trackSelector.buildUponParameters().setPreferredVideoMimeType(MimeTypes.VIDEO_VP9)); + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups, formatVp9); + + trackSelector.setParameters( + trackSelector + .buildUponParameters() + .setPreferredVideoMimeTypes(MimeTypes.VIDEO_VP9, MimeTypes.VIDEO_AV1)); + result = + trackSelector.selectTracks( + new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups, formatVp9); + + trackSelector.setParameters( + trackSelector + .buildUponParameters() + .setPreferredVideoMimeTypes(MimeTypes.VIDEO_DIVX, MimeTypes.VIDEO_H264)); + result = + trackSelector.selectTracks( + new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups, formatH264); + + // Select first in the list if no preference is specified. + trackSelector.setParameters( + trackSelector.buildUponParameters().setPreferredVideoMimeType(null)); + result = + trackSelector.selectTracks( + new RendererCapabilities[] {VIDEO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups, formatAv1); + } + + @Test + public void selectTracks_withPreferredAudioMimeTypes_selectsTrackWithPreferredMimeType() + throws Exception { + Format formatAac = new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build(); + Format formatAc4 = new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AC4).build(); + Format formatEAc3 = new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_E_AC3).build(); + TrackGroupArray trackGroups = wrapFormats(formatAac, formatAc4, formatEAc3); + + trackSelector.setParameters( + trackSelector.buildUponParameters().setPreferredAudioMimeType(MimeTypes.AUDIO_AC4)); + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups, formatAc4); + + trackSelector.setParameters( + trackSelector + .buildUponParameters() + .setPreferredAudioMimeTypes(MimeTypes.AUDIO_AC4, MimeTypes.AUDIO_AAC)); + result = + trackSelector.selectTracks( + new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups, formatAc4); + + trackSelector.setParameters( + trackSelector + .buildUponParameters() + .setPreferredAudioMimeTypes(MimeTypes.AUDIO_AMR, MimeTypes.AUDIO_E_AC3)); + result = + trackSelector.selectTracks( + new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups, formatEAc3); + + // Select first in the list if no preference is specified. + trackSelector.setParameters( + trackSelector.buildUponParameters().setPreferredAudioMimeType(null)); + result = + trackSelector.selectTracks( + new RendererCapabilities[] {AUDIO_CAPABILITIES}, trackGroups, periodId, TIMELINE); + assertThat(result.length).isEqualTo(1); + assertFixedSelection(result.selections[0], trackGroups, formatAac); } private static void assertSelections(TrackSelectorResult result, TrackSelection[] expected) { assertThat(result.length).isEqualTo(expected.length); for (int i = 0; i < expected.length; i++) { - assertThat(result.selections.get(i)).isEqualTo(expected[i]); + assertThat(result.selections[i]).isEqualTo(expected[i]); } } @@ -1472,6 +1697,7 @@ private static Format buildAudioFormatWithConfiguration( .setSampleMimeType(mimeType) .setChannelCount(channelCount) .setSampleRate(sampleRate) + .setAverageBitrate(128000) .build(); } @@ -1507,33 +1733,37 @@ private static Parameters buildParametersForEqualsTest() { /* viewportWidth= */ 8, /* viewportHeight= */ 9, /* viewportOrientationMayChange= */ true, + /* preferredVideoMimeTypes= */ ImmutableList.of(MimeTypes.VIDEO_AV1, MimeTypes.VIDEO_H264), // Audio - /* preferredAudioLanguage= */ "en", + /* preferredAudioLanguages= */ ImmutableList.of("zh", "jp"), + /* preferredAudioRoleFlags= */ C.ROLE_FLAG_COMMENTARY, /* maxAudioChannelCount= */ 10, /* maxAudioBitrate= */ 11, /* exceedAudioConstraintsIfNecessary= */ false, /* allowAudioMixedMimeTypeAdaptiveness= */ true, /* allowAudioMixedSampleRateAdaptiveness= */ false, /* allowAudioMixedChannelCountAdaptiveness= */ true, + /* preferredAudioMimeTypes= */ ImmutableList.of(MimeTypes.AUDIO_AC3, MimeTypes.AUDIO_E_AC3), // Text - /* preferredTextLanguage= */ "de", + /* preferredTextLanguages= */ ImmutableList.of("de", "en"), /* preferredTextRoleFlags= */ C.ROLE_FLAG_CAPTION, /* selectUndeterminedTextLanguage= */ true, - /* disabledTextTrackSelectionFlags= */ 12, + /* disabledTextTrackSelectionFlags= */ C.SELECTION_FLAG_AUTOSELECT, // General /* forceLowestBitrate= */ false, /* forceHighestSupportedBitrate= */ true, /* exceedRendererCapabilitiesIfNecessary= */ false, - /* tunnelingAudioSessionId= */ 13, + /* tunnelingEnabled= */ true, + /* allowMultipleAdaptiveSelections= */ true, // Overrides selectionOverrides, rendererDisabledFlags); } /** - * A {@link RendererCapabilities} that advertises support for all formats of a given type using - * a provided support value. For any format that does not have the given track type, - * {@link #supportsFormat(Format)} will return {@link #FORMAT_UNSUPPORTED_TYPE}. + * A {@link RendererCapabilities} that advertises support for all formats of a given type using a + * provided support value. For any format that does not have the given track type, {@link + * #supportsFormat(Format)} will return {@link C#FORMAT_UNSUPPORTED_TYPE}. */ private static final class FakeRendererCapabilities implements RendererCapabilities { @@ -1541,11 +1771,11 @@ private static final class FakeRendererCapabilities implements RendererCapabilit @Capabilities private final int supportValue; /** - * Returns {@link FakeRendererCapabilities} that advertises adaptive support for all - * tracks of the given type. + * Returns {@link FakeRendererCapabilities} that advertises adaptive support for all tracks of + * the given type. * * @param trackType the track type of all formats that this renderer capabilities advertises - * support for. + * support for. */ FakeRendererCapabilities(int trackType) { this( @@ -1582,7 +1812,7 @@ public int getTrackType() { public int supportsFormat(Format format) { return MimeTypes.getTrackType(format.sampleMimeType) == trackType ? supportValue - : RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + : RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } @Override @@ -1590,7 +1820,6 @@ public int supportsFormat(Format format) { public int supportsMixedMimeTypeAdaptation() { return ADAPTIVE_SEAMLESS; } - } /** @@ -1608,8 +1837,8 @@ private static final class FakeMappedRendererCapabilities implements RendererCap * * @param trackType the track type to be returned for {@link #getTrackType()} * @param formatToCapability a map of (format id, support level) that will be used to return - * support level for any given format. For any format that's not in the map, - * {@link #supportsFormat(Format)} will return {@link #FORMAT_UNSUPPORTED_TYPE}. + * support level for any given format. For any format that's not in the map, {@link + * #supportsFormat(Format)} will return {@link C#FORMAT_UNSUPPORTED_TYPE}. */ FakeMappedRendererCapabilities(int trackType, Map formatToCapability) { this.trackType = trackType; @@ -1631,7 +1860,7 @@ public int getTrackType() { public int supportsFormat(Format format) { return format.id != null && formatToCapability.containsKey(format.id) ? formatToCapability.get(format.id) - : RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + : RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } @Override @@ -1639,7 +1868,5 @@ public int supportsFormat(Format format) { public int supportsMixedMimeTypeAdaptation() { return ADAPTIVE_SEAMLESS; } - } - } diff --git a/library/core/src/test/java/com/google/android/exoplayer2/trackselection/MappingTrackSelectorTest.java b/library/core/src/test/java/com/google/android/exoplayer2/trackselection/MappingTrackSelectorTest.java index 5d5508f3cd1..9a01f85aa9c 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/trackselection/MappingTrackSelectorTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/trackselection/MappingTrackSelectorTest.java @@ -52,7 +52,7 @@ public final class MappingTrackSelectorTest { private static final TrackGroup AUDIO_TRACK_GROUP = buildTrackGroup(MimeTypes.AUDIO_AAC); private static final TrackGroup METADATA_TRACK_GROUP = buildTrackGroup(MimeTypes.APPLICATION_ID3); - private static final Timeline TIMELINE = new FakeTimeline(/* windowCount= */ 1); + private static final Timeline TIMELINE = new FakeTimeline(); private static MediaPeriodId periodId; @@ -131,22 +131,23 @@ private static TrackGroup buildTrackGroup(String sampleMimeType) { /** * A {@link MappingTrackSelector} that stashes the {@link MappedTrackInfo} passed to {@link - * #selectTracks(MappedTrackInfo, int[][][], int[])}. + * #selectTracks(MappedTrackInfo, int[][][], int[], MediaPeriodId, Timeline)}. */ private static final class FakeMappingTrackSelector extends MappingTrackSelector { private MappedTrackInfo lastMappedTrackInfo; @Override - protected Pair selectTracks( + protected Pair selectTracks( MappedTrackInfo mappedTrackInfo, @Capabilities int[][][] rendererFormatSupports, - @AdaptiveSupport int[] rendererMixedMimeTypeAdaptationSupports) - throws ExoPlaybackException { + @AdaptiveSupport int[] rendererMixedMimeTypeAdaptationSupports, + MediaPeriodId mediaPeriodId, + Timeline timeline) { int rendererCount = mappedTrackInfo.getRendererCount(); lastMappedTrackInfo = mappedTrackInfo; return Pair.create( - new RendererConfiguration[rendererCount], new TrackSelection[rendererCount]); + new RendererConfiguration[rendererCount], new ExoTrackSelection[rendererCount]); } public void assertMappedTrackGroups(int rendererIndex, TrackGroup... expected) { @@ -156,7 +157,6 @@ public void assertMappedTrackGroups(int rendererIndex, TrackGroup... expected) { assertThat(rendererTrackGroupArray.get(i)).isEqualTo(expected[i]); } } - } /** @@ -184,8 +184,9 @@ public int getTrackType() { @Capabilities public int supportsFormat(Format format) throws ExoPlaybackException { return MimeTypes.getTrackType(format.sampleMimeType) == trackType - ? RendererCapabilities.create(FORMAT_HANDLED, ADAPTIVE_SEAMLESS, TUNNELING_NOT_SUPPORTED) - : RendererCapabilities.create(FORMAT_UNSUPPORTED_TYPE); + ? RendererCapabilities.create( + C.FORMAT_HANDLED, ADAPTIVE_SEAMLESS, TUNNELING_NOT_SUPPORTED) + : RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); } @Override diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/AssetDataSourceContractTest.java b/library/core/src/test/java/com/google/android/exoplayer2/upstream/AssetDataSourceContractTest.java new file mode 100644 index 00000000000..9df41556168 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/upstream/AssetDataSourceContractTest.java @@ -0,0 +1,63 @@ +/* + * Copyright (C) 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.upstream; + +import android.net.Uri; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.common.collect.ImmutableList; +import java.io.IOException; +import org.junit.Before; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link AssetDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class AssetDataSourceContractTest extends DataSourceContractTest { + + // We pick an arbitrary file from the assets. The selected file has a convenient size of 1024 + // bytes. + private static final String ASSET_PATH = "media/mp3/1024_incrementing_bytes.mp3"; + private static final Uri ASSET_URI = Uri.parse("asset:///" + ASSET_PATH); + + private byte[] data; + + @Before + public void setUp() throws IOException { + data = TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), ASSET_PATH); + } + + @Override + protected ImmutableList getTestResources() { + return ImmutableList.of( + new TestResource.Builder() + .setName("simple") + .setUri(ASSET_URI) + .setExpectedBytes(data) + .build()); + } + + @Override + protected Uri getNotFoundUri() { + return Uri.parse("asset:///nonexistentdir/nonexistentfile"); + } + + @Override + protected DataSource createDataSource() { + return new AssetDataSource(ApplicationProvider.getApplicationContext()); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/ByteArrayDataSourceContractTest.java b/library/core/src/test/java/com/google/android/exoplayer2/upstream/ByteArrayDataSourceContractTest.java new file mode 100644 index 00000000000..1aa2198fc51 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/upstream/ByteArrayDataSourceContractTest.java @@ -0,0 +1,57 @@ +/* + * Copyright (C) 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.upstream; + +import android.net.Uri; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.common.collect.ImmutableList; +import org.junit.Ignore; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link ByteArrayDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class ByteArrayDataSourceContractTest extends DataSourceContractTest { + + private static final byte[] DATA = TestUtil.buildTestData(20); + + @Override + protected ImmutableList getTestResources() { + return ImmutableList.of( + new TestResource.Builder() + .setName("simple") + .setUri(Uri.EMPTY) + .setExpectedBytes(DATA) + .build()); + } + + @Override + protected Uri getNotFoundUri() { + throw new UnsupportedOperationException(); + } + + @Override + protected DataSource createDataSource() { + return new ByteArrayDataSource(DATA); + } + + @Override + @Test + @Ignore + public void resourceNotFound() {} +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/CacheDataSourceContractTest.java b/library/core/src/test/java/com/google/android/exoplayer2/upstream/CacheDataSourceContractTest.java new file mode 100644 index 00000000000..b75ff45f13f --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/upstream/CacheDataSourceContractTest.java @@ -0,0 +1,76 @@ +/* + * Copyright (C) 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.upstream; + +import android.net.Uri; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.android.exoplayer2.upstream.cache.CacheDataSource; +import com.google.android.exoplayer2.upstream.cache.NoOpCacheEvictor; +import com.google.android.exoplayer2.upstream.cache.SimpleCache; +import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.ImmutableList; +import java.io.File; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Paths; +import org.junit.Before; +import org.junit.Rule; +import org.junit.rules.TemporaryFolder; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link CacheDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class CacheDataSourceContractTest extends DataSourceContractTest { + private static final byte[] DATA = TestUtil.buildTestData(20); + + @Rule public final TemporaryFolder tempFolder = new TemporaryFolder(); + + private Uri simpleUri; + + @Before + public void setUp() throws IOException { + File file = tempFolder.newFile(); + Files.write(Paths.get(file.getAbsolutePath()), DATA); + simpleUri = Uri.fromFile(file); + } + + @Override + protected ImmutableList getTestResources() { + return ImmutableList.of( + new TestResource.Builder() + .setName("simple") + .setUri(simpleUri) + .setExpectedBytes(DATA) + .build()); + } + + @Override + protected Uri getNotFoundUri() { + return Uri.fromFile(tempFolder.getRoot().toPath().resolve("nonexistent").toFile()); + } + + @Override + protected DataSource createDataSource() throws IOException { + File tempFolder = + Util.createTempDirectory(ApplicationProvider.getApplicationContext(), "ExoPlayerTest"); + SimpleCache cache = + new SimpleCache(tempFolder, new NoOpCacheEvictor(), TestUtil.getInMemoryDatabaseProvider()); + return new CacheDataSource(cache, new FileDataSource()); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/DataSchemeDataSourceContractTest.java b/library/core/src/test/java/com/google/android/exoplayer2/upstream/DataSchemeDataSourceContractTest.java new file mode 100644 index 00000000000..97bd7018658 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/upstream/DataSchemeDataSourceContractTest.java @@ -0,0 +1,61 @@ +/* + * Copyright (C) 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.upstream; + +import static java.nio.charset.StandardCharsets.UTF_8; + +import android.net.Uri; +import android.util.Base64; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.common.collect.ImmutableList; +import java.util.Random; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link ByteArrayDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class DataSchemeDataSourceContractTest extends DataSourceContractTest { + + private static final String DATA = TestUtil.buildTestString(20, new Random(0)); + private static final String BASE64_ENCODED_DATA = + Base64.encodeToString(TestUtil.buildTestData(20), Base64.DEFAULT); + + @Override + protected ImmutableList getTestResources() { + return ImmutableList.of( + new TestResource.Builder() + .setName("plain text") + .setUri(Uri.parse("data:text/plain," + DATA)) + .setExpectedBytes(DATA.getBytes(UTF_8)) + .build(), + new TestResource.Builder() + .setName("base64 encoded text") + .setUri(Uri.parse("data:text/plain;base64," + BASE64_ENCODED_DATA)) + .setExpectedBytes(Base64.decode(BASE64_ENCODED_DATA, Base64.DEFAULT)) + .build()); + } + + @Override + protected Uri getNotFoundUri() { + return Uri.parse("data:"); + } + + @Override + protected DataSource createDataSource() { + return new DataSchemeDataSource(); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/FileDataSourceContractTest.java b/library/core/src/test/java/com/google/android/exoplayer2/upstream/FileDataSourceContractTest.java new file mode 100644 index 00000000000..671235738ce --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/upstream/FileDataSourceContractTest.java @@ -0,0 +1,72 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.upstream; + +import android.net.Uri; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.common.collect.ImmutableList; +import java.io.File; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Paths; +import org.junit.Before; +import org.junit.Rule; +import org.junit.rules.TemporaryFolder; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link FileDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class FileDataSourceContractTest extends DataSourceContractTest { + + private static final byte[] DATA = TestUtil.buildTestData(20); + + @Rule public final TemporaryFolder tempFolder = new TemporaryFolder(); + + private Uri simpleUri; + + @Before + public void writeFiles() throws Exception { + simpleUri = writeFile(DATA); + } + + @Override + protected ImmutableList getTestResources() { + return ImmutableList.of( + new TestResource.Builder() + .setName("simple") + .setUri(simpleUri) + .setExpectedBytes(DATA) + .build()); + } + + @Override + protected Uri getNotFoundUri() { + return Uri.fromFile(tempFolder.getRoot().toPath().resolve("nonexistent").toFile()); + } + + @Override + protected DataSource createDataSource() { + return new FileDataSource(); + } + + private Uri writeFile(byte[] data) throws IOException { + File file = tempFolder.newFile(); + Files.write(Paths.get(file.getAbsolutePath()), data); + return Uri.fromFile(file); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/UdpDataSourceContractTest.java b/library/core/src/test/java/com/google/android/exoplayer2/upstream/UdpDataSourceContractTest.java new file mode 100644 index 00000000000..c8dd082e674 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/upstream/UdpDataSourceContractTest.java @@ -0,0 +1,145 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.upstream; + +import static java.lang.Math.min; + +import android.net.Uri; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.testutil.DataSourceContractTest; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.common.collect.ImmutableList; +import java.io.IOException; +import java.net.DatagramPacket; +import java.net.DatagramSocket; +import java.net.InetAddress; +import java.net.SocketException; +import org.junit.Before; +import org.junit.Ignore; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** {@link DataSource} contract tests for {@link UdpDataSource}. */ +@RunWith(AndroidJUnit4.class) +public class UdpDataSourceContractTest extends DataSourceContractTest { + + private UdpDataSource udpDataSource; + private byte[] data; + + @Before + public void setUp() { + udpDataSource = new UdpDataSource(); + data = TestUtil.buildTestData(/* length= */ 256); + PacketTrasmitterTransferListener transferListener = new PacketTrasmitterTransferListener(data); + udpDataSource.addTransferListener(transferListener); + } + + @Override + protected DataSource createDataSource() { + return udpDataSource; + } + + @Override + protected ImmutableList getTestResources() { + return ImmutableList.of( + new TestResource.Builder() + .setName("local-udp-unicast-socket") + .setUri(Uri.parse("udp://localhost:" + findFreeUdpPort())) + .setExpectedBytes(data) + .setEndOfInputExpected(false) + .build()); + } + + @Override + protected Uri getNotFoundUri() { + return Uri.parse("udp://notfound.invalid:12345"); + } + + @Test + @Ignore("UdpDataSource doesn't support DataSpec's position or length [internal: b/175856954]") + @Override + public void dataSpecWithPosition_readUntilEnd() {} + + @Test + @Ignore("UdpDataSource doesn't support DataSpec's position or length [internal: b/175856954]") + @Override + public void dataSpecWithLength_readExpectedRange() {} + + @Test + @Ignore("UdpDataSource doesn't support DataSpec's position or length [internal: b/175856954]") + @Override + public void dataSpecWithPositionAndLength_readExpectedRange() {} + + /** + * Finds a free UDP port in the range of unreserved ports 50000-60000 that can be used from the + * test or throws an {@link IllegalStateException} if no port is available. + * + *

    There is no guarantee that the port returned will still be available as another process may + * occupy it in the mean time. + */ + private static int findFreeUdpPort() { + for (int i = 50000; i <= 60000; i++) { + try { + new DatagramSocket(i).close(); + return i; + } catch (SocketException e) { + // Port is occupied, continue to next port. + } + } + throw new IllegalStateException(); + } + + /** + * A {@link TransferListener} that triggers UDP packet transmissions back to the UDP data source. + */ + private static class PacketTrasmitterTransferListener implements TransferListener { + private final byte[] data; + + public PacketTrasmitterTransferListener(byte[] data) { + this.data = data; + } + + @Override + public void onTransferInitializing(DataSource source, DataSpec dataSpec, boolean isNetwork) {} + + @Override + public void onTransferStart(DataSource source, DataSpec dataSpec, boolean isNetwork) { + String host = dataSpec.uri.getHost(); + int port = dataSpec.uri.getPort(); + try (DatagramSocket socket = new DatagramSocket()) { + // Split data in packets of up to 64 bytes: UDP is unreliable, it may lose, duplicate or + // re-order packets. However, we want to transmit more than one UDP packets to thoroughly + // test the UDP data source. We assume that UDP delivery within the same host is reliable. + for (int offset = 0; offset < data.length; offset += 64) { + int packetLength = min(64, data.length - offset); + DatagramPacket packet = + new DatagramPacket(data, offset, packetLength, InetAddress.getByName(host), port); + socket.send(packet); + } + } catch (IOException e) { + throw new IllegalStateException(e); + } + } + + @Override + public void onBytesTransferred( + DataSource source, DataSpec dataSpec, boolean isNetwork, int bytesTransferred) {} + + @Override + public void onTransferEnd(DataSource source, DataSpec dataSpec, boolean isNetwork) {} + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/upstream/cache/SimpleCacheTest.java b/library/core/src/test/java/com/google/android/exoplayer2/upstream/cache/SimpleCacheTest.java index 482c95bfd21..9fea31b5d57 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/upstream/cache/SimpleCacheTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/upstream/cache/SimpleCacheTest.java @@ -643,7 +643,7 @@ public void getCachedBytes_withMultipleNonAdjacentSpans_returnsCachedBytes() thr .isEqualTo(10); } - /* Tests https://github.com/google/ExoPlayer/issues/3260 case. */ + // Regression test for https://github.com/google/ExoPlayer/issues/3260. @Test public void exceptionDuringIndexStore_doesNotPreventEviction() throws Exception { CachedContentIndex contentIndex = diff --git a/library/core/src/test/java/com/google/android/exoplayer2/util/MutableFlagsTest.java b/library/core/src/test/java/com/google/android/exoplayer2/util/MutableFlagsTest.java new file mode 100644 index 00000000000..1e68f804764 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/util/MutableFlagsTest.java @@ -0,0 +1,140 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.util; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.ArrayList; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link MutableFlags}. */ +@RunWith(AndroidJUnit4.class) +public final class MutableFlagsTest { + + @Test + public void contains_withoutAdd_returnsFalseForAllValues() { + MutableFlags flags = new MutableFlags(); + + assertThat(flags.contains(/* flag= */ -1234)).isFalse(); + assertThat(flags.contains(/* flag= */ 0)).isFalse(); + assertThat(flags.contains(/* flag= */ 2)).isFalse(); + assertThat(flags.contains(/* flag= */ Integer.MAX_VALUE)).isFalse(); + } + + @Test + public void contains_afterAdd_returnsTrueForAddedValues() { + MutableFlags flags = new MutableFlags(); + + flags.add(/* flag= */ -1234); + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 2); + flags.add(/* flag= */ Integer.MAX_VALUE); + + assertThat(flags.contains(/* flag= */ -1235)).isFalse(); + assertThat(flags.contains(/* flag= */ -1234)).isTrue(); + assertThat(flags.contains(/* flag= */ 0)).isTrue(); + assertThat(flags.contains(/* flag= */ 1)).isFalse(); + assertThat(flags.contains(/* flag= */ 2)).isTrue(); + assertThat(flags.contains(/* flag= */ Integer.MAX_VALUE - 1)).isFalse(); + assertThat(flags.contains(/* flag= */ Integer.MAX_VALUE)).isTrue(); + } + + @Test + public void contains_afterClear_returnsFalseForAllValues() { + MutableFlags flags = new MutableFlags(); + flags.add(/* flag= */ -1234); + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 2); + flags.add(/* flag= */ Integer.MAX_VALUE); + + flags.clear(); + + assertThat(flags.contains(/* flag= */ -1234)).isFalse(); + assertThat(flags.contains(/* flag= */ 0)).isFalse(); + assertThat(flags.contains(/* flag= */ 2)).isFalse(); + assertThat(flags.contains(/* flag= */ Integer.MAX_VALUE)).isFalse(); + } + + @Test + public void size_withoutAdd_returnsZero() { + MutableFlags flags = new MutableFlags(); + + assertThat(flags.size()).isEqualTo(0); + } + + @Test + public void size_afterAdd_returnsNumberUniqueOfElements() { + MutableFlags flags = new MutableFlags(); + + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 123); + flags.add(/* flag= */ 123); + + assertThat(flags.size()).isEqualTo(2); + } + + @Test + public void size_afterClear_returnsZero() { + MutableFlags flags = new MutableFlags(); + + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 123); + flags.clear(); + + assertThat(flags.size()).isEqualTo(0); + } + + @Test + public void get_withNegativeIndex_throwsIllegalArgumentException() { + MutableFlags flags = new MutableFlags(); + + assertThrows(IllegalArgumentException.class, () -> flags.get(/* index= */ -1)); + } + + @Test + public void get_withIndexExceedingSize_throwsIllegalArgumentException() { + MutableFlags flags = new MutableFlags(); + + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 123); + + assertThrows(IllegalArgumentException.class, () -> flags.get(/* index= */ 2)); + } + + @Test + public void get_afterAdd_returnsAllUniqueValues() { + MutableFlags flags = new MutableFlags(); + + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 0); + flags.add(/* flag= */ 123); + flags.add(/* flag= */ 123); + flags.add(/* flag= */ 456); + + List values = new ArrayList<>(); + for (int i = 0; i < flags.size(); i++) { + values.add(flags.get(i)); + } + assertThat(values).containsExactly(0, 123, 456); + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/video/DecoderVideoRendererTest.java b/library/core/src/test/java/com/google/android/exoplayer2/video/DecoderVideoRendererTest.java index 57cc7cb9b0a..848b0ce410d 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/video/DecoderVideoRendererTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/video/DecoderVideoRendererTest.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.video; +import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.oneByteSample; import static org.mockito.ArgumentMatchers.any; import static org.mockito.Mockito.never; @@ -39,7 +40,7 @@ import com.google.android.exoplayer2.drm.DrmSessionManager; import com.google.android.exoplayer2.drm.ExoMediaCrypto; import com.google.android.exoplayer2.testutil.FakeSampleStream; -import com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem; +import com.google.android.exoplayer2.upstream.DefaultAllocator; import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.collect.ImmutableList; import java.util.concurrent.Phaser; @@ -89,7 +90,7 @@ public String getName() { @Override @Capabilities public int supportsFormat(Format format) { - return RendererCapabilities.create(FORMAT_HANDLED); + return RendererCapabilities.create(C.FORMAT_HANDLED); } @Override @@ -184,11 +185,13 @@ public void shutDown() throws Exception { public void enable_withMayRenderStartOfStream_rendersFirstFrameBeforeStart() throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ H264_FORMAT, - ImmutableList.of(oneByteSample(/* timeUs= */ 0))); + ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + fakeSampleStream.writeData(/* startPositionUs= */ 0); renderer.enable( RendererConfiguration.DEFAULT, @@ -213,11 +216,13 @@ public void enable_withoutMayRenderStartOfStream_doesNotRenderFirstFrameBeforeSt throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ H264_FORMAT, - ImmutableList.of(oneByteSample(/* timeUs= */ 0))); + ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + fakeSampleStream.writeData(/* startPositionUs= */ 0); renderer.enable( RendererConfiguration.DEFAULT, @@ -241,11 +246,13 @@ public void enable_withoutMayRenderStartOfStream_doesNotRenderFirstFrameBeforeSt public void enable_withoutMayRenderStartOfStream_rendersFirstFrameAfterStart() throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ H264_FORMAT, - ImmutableList.of(oneByteSample(/* timeUs= */ 0))); + ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + fakeSampleStream.writeData(/* startPositionUs= */ 0); renderer.enable( RendererConfiguration.DEFAULT, @@ -272,20 +279,23 @@ public void enable_withoutMayRenderStartOfStream_rendersFirstFrameAfterStart() t public void replaceStream_whenStarted_rendersFirstFrameOfNewStream() throws Exception { FakeSampleStream fakeSampleStream1 = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ H264_FORMAT, ImmutableList.of( - oneByteSample(/* timeUs= */ 0), FakeSampleStreamItem.END_OF_STREAM_ITEM)); + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream1.writeData(/* startPositionUs= */ 0); FakeSampleStream fakeSampleStream2 = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ H264_FORMAT, - ImmutableList.of( - oneByteSample(/* timeUs= */ 0), FakeSampleStreamItem.END_OF_STREAM_ITEM)); + ImmutableList.of(oneByteSample(/* timeUs= */ 0), END_OF_STREAM_ITEM)); + fakeSampleStream2.writeData(/* startPositionUs= */ 0); renderer.enable( RendererConfiguration.DEFAULT, new Format[] {H264_FORMAT}, @@ -321,20 +331,23 @@ public void replaceStream_whenStarted_rendersFirstFrameOfNewStream() throws Exce public void replaceStream_whenNotStarted_doesNotRenderFirstFrameOfNewStream() throws Exception { FakeSampleStream fakeSampleStream1 = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ H264_FORMAT, ImmutableList.of( - oneByteSample(/* timeUs= */ 0), FakeSampleStreamItem.END_OF_STREAM_ITEM)); + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream1.writeData(/* startPositionUs= */ 0); FakeSampleStream fakeSampleStream2 = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ H264_FORMAT, - ImmutableList.of( - oneByteSample(/* timeUs= */ 0), FakeSampleStreamItem.END_OF_STREAM_ITEM)); + ImmutableList.of(oneByteSample(/* timeUs= */ 0), END_OF_STREAM_ITEM)); + fakeSampleStream2.writeData(/* startPositionUs= */ 0); renderer.enable( RendererConfiguration.DEFAULT, new Format[] {H264_FORMAT}, diff --git a/library/core/src/test/java/com/google/android/exoplayer2/video/FixedFrameRateEstimatorTest.java b/library/core/src/test/java/com/google/android/exoplayer2/video/FixedFrameRateEstimatorTest.java new file mode 100644 index 00000000000..dbe2c6900e1 --- /dev/null +++ b/library/core/src/test/java/com/google/android/exoplayer2/video/FixedFrameRateEstimatorTest.java @@ -0,0 +1,208 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.video; + +import static com.google.android.exoplayer2.video.FixedFrameRateEstimator.CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC; +import static com.google.android.exoplayer2.video.FixedFrameRateEstimator.MAX_MATCHING_FRAME_DIFFERENCE_NS; +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link FixedFrameRateEstimator}. */ +@RunWith(AndroidJUnit4.class) +public final class FixedFrameRateEstimatorTest { + + @Test + public void fixedFrameRate_withSingleOutlier_syncsAndResyncs() { + long frameDurationNs = 33_333_333; + FixedFrameRateEstimator estimator = new FixedFrameRateEstimator(); + + // Initial frame. + long framePresentationTimestampNs = 0; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + + // Frames with consistent durations, working toward establishing sync. + for (int i = 0; i < CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC - 1; i++) { + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + } + + // This frame should establish sync. + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isTrue(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(frameDurationNs); + + framePresentationTimestampNs += frameDurationNs; + // Make the frame duration just shorter enough to lose sync. + framePresentationTimestampNs -= MAX_MATCHING_FRAME_DIFFERENCE_NS + 1; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + + // Frames with consistent durations, working toward re-establishing sync. + for (int i = 0; i < CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC - 1; i++) { + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + } + + // This frame should re-establish sync. + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isTrue(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(frameDurationNs); + } + + @Test + public void fixedFrameRate_withOutlierFirstFrameDuration_syncs() { + long frameDurationNs = 33_333_333; + FixedFrameRateEstimator estimator = new FixedFrameRateEstimator(); + + // Initial frame with double duration. + long framePresentationTimestampNs = 0; + estimator.onNextFrame(framePresentationTimestampNs); + + framePresentationTimestampNs += frameDurationNs * 2; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + + // Frames with consistent durations, working toward establishing sync. + for (int i = 0; i < CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC - 1; i++) { + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + } + + // This frame should establish sync. + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(framePresentationTimestampNs); + } + + @Test + public void newFixedFrameRate_resyncs() { + long frameDurationNs = 33_333_333; + FixedFrameRateEstimator estimator = new FixedFrameRateEstimator(); + + long framePresentationTimestampNs = 0; + estimator.onNextFrame(framePresentationTimestampNs); + for (int i = 0; i < CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC; i++) { + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(framePresentationTimestampNs); + } + + assertThat(estimator.isSynced()).isTrue(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(frameDurationNs); + + // Frames durations are halved from this point. + long halfFrameRateDuration = frameDurationNs / 2; + + // Frames with consistent durations, working toward establishing new sync. + for (int i = 0; i < CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC - 1; i++) { + framePresentationTimestampNs += halfFrameRateDuration; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + } + + // This frame should establish sync. + framePresentationTimestampNs += halfFrameRateDuration; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isTrue(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(halfFrameRateDuration); + } + + @Test + public void fixedFrameRate_withMillisecondPrecision_syncs() { + long frameDurationNs = 33_333_333; + FixedFrameRateEstimator estimator = new FixedFrameRateEstimator(); + + // Initial frame. + long framePresentationTimestampNs = 0; + estimator.onNextFrame(getNsWithMsPrecision(framePresentationTimestampNs)); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + + // Frames with consistent durations, working toward establishing sync. + for (int i = 0; i < CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC - 1; i++) { + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(getNsWithMsPrecision(framePresentationTimestampNs)); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + } + + // This frame should establish sync. + framePresentationTimestampNs += frameDurationNs; + estimator.onNextFrame(getNsWithMsPrecision(framePresentationTimestampNs)); + + assertThat(estimator.isSynced()).isTrue(); + // The estimated frame duration should be strictly better than millisecond precision. + long estimatedFrameDurationNs = estimator.getFrameDurationNs(); + long estimatedFrameDurationErrorNs = Math.abs(estimatedFrameDurationNs - frameDurationNs); + assertThat(estimatedFrameDurationErrorNs).isLessThan(1000000); + } + + @Test + public void variableFrameRate_doesNotSync() { + long frameDurationNs = 33_333_333; + FixedFrameRateEstimator estimator = new FixedFrameRateEstimator(); + + // Initial frame. + long framePresentationTimestampNs = 0; + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + + for (int i = 0; i < CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC * 10; i++) { + framePresentationTimestampNs += frameDurationNs; + // Adjust a frame that's just different enough, just often enough to prevent sync. + if ((i % CONSECUTIVE_MATCHING_FRAME_DURATIONS_FOR_SYNC) == 0) { + framePresentationTimestampNs += MAX_MATCHING_FRAME_DIFFERENCE_NS + 1; + } + estimator.onNextFrame(framePresentationTimestampNs); + + assertThat(estimator.isSynced()).isFalse(); + assertThat(estimator.getFrameDurationNs()).isEqualTo(C.TIME_UNSET); + } + } + + private static final long getNsWithMsPrecision(long presentationTimeNs) { + return (presentationTimeNs / 1000000) * 1000000; + } +} diff --git a/library/core/src/test/java/com/google/android/exoplayer2/video/MediaCodecVideoRendererTest.java b/library/core/src/test/java/com/google/android/exoplayer2/video/MediaCodecVideoRendererTest.java index 74d110516b1..ccc4e89d58b 100644 --- a/library/core/src/test/java/com/google/android/exoplayer2/video/MediaCodecVideoRendererTest.java +++ b/library/core/src/test/java/com/google/android/exoplayer2/video/MediaCodecVideoRendererTest.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.video; +import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.format; import static com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem.oneByteSample; import static com.google.common.truth.Truth.assertThat; @@ -47,7 +48,7 @@ import com.google.android.exoplayer2.mediacodec.MediaCodecInfo; import com.google.android.exoplayer2.mediacodec.MediaCodecSelector; import com.google.android.exoplayer2.testutil.FakeSampleStream; -import com.google.android.exoplayer2.testutil.FakeSampleStream.FakeSampleStreamItem; +import com.google.android.exoplayer2.upstream.DefaultAllocator; import com.google.android.exoplayer2.util.MimeTypes; import com.google.common.collect.ImmutableList; import java.util.Collections; @@ -107,7 +108,7 @@ public void setUp() throws Exception { @Override @Capabilities protected int supportsFormat(MediaCodecSelector mediaCodecSelector, Format format) { - return RendererCapabilities.create(FORMAT_HANDLED); + return RendererCapabilities.create(C.FORMAT_HANDLED); } @Override @@ -125,15 +126,17 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF public void render_dropsLateBuffer() throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of( oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), // First buffer. oneByteSample(/* timeUs= */ 50_000), // Late buffer. oneByteSample(/* timeUs= */ 100_000), // Last buffer. - FakeSampleStreamItem.END_OF_STREAM_ITEM)); + END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); mediaCodecVideoRenderer.enable( RendererConfiguration.DEFAULT, new Format[] {VIDEO_H264}, @@ -160,17 +163,20 @@ public void render_dropsLateBuffer() throws Exception { @Test public void render_sendsVideoSizeChangeWithCurrentFormatValues() throws Exception { - mediaCodecVideoRenderer.enable( - RendererConfiguration.DEFAULT, - new Format[] {VIDEO_H264}, + FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of( - oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), - FakeSampleStreamItem.END_OF_STREAM_ITEM)), + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); + mediaCodecVideoRenderer.enable( + RendererConfiguration.DEFAULT, + new Format[] {VIDEO_H264}, + fakeSampleStream, /* positionUs= */ 0, /* joining= */ false, /* mayRenderStartOfStream= */ true, @@ -204,11 +210,13 @@ public void render_sendsVideoSizeChangeWithCurrentFormatValues() throws Exceptio FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ pAsp1, ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + fakeSampleStream.writeData(/* startPositionUs= */ 0); mediaCodecVideoRenderer.enable( RendererConfiguration.DEFAULT, @@ -223,13 +231,16 @@ public void render_sendsVideoSizeChangeWithCurrentFormatValues() throws Exceptio mediaCodecVideoRenderer.render(/* positionUs= */ 0, SystemClock.elapsedRealtime() * 1000); mediaCodecVideoRenderer.render(/* positionUs= */ 250, SystemClock.elapsedRealtime() * 1000); - fakeSampleStream.addFakeSampleStreamItem(format(pAsp2)); - fakeSampleStream.addFakeSampleStreamItem(oneByteSample(/* timeUs= */ 5_000)); - fakeSampleStream.addFakeSampleStreamItem(oneByteSample(/* timeUs= */ 10_000)); - fakeSampleStream.addFakeSampleStreamItem(format(pAsp3)); - fakeSampleStream.addFakeSampleStreamItem(oneByteSample(/* timeUs= */ 15_000)); - fakeSampleStream.addFakeSampleStreamItem(oneByteSample(/* timeUs= */ 20_000)); - fakeSampleStream.addFakeSampleStreamItem(FakeSampleStreamItem.END_OF_STREAM_ITEM); + fakeSampleStream.append( + ImmutableList.of( + format(pAsp2), + oneByteSample(/* timeUs= */ 5_000), + oneByteSample(/* timeUs= */ 10_000), + format(pAsp3), + oneByteSample(/* timeUs= */ 15_000), + oneByteSample(/* timeUs= */ 20_000), + END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 5_000); mediaCodecVideoRenderer.setCurrentStreamFinal(); int pos = 500; @@ -251,11 +262,13 @@ public void render_includingResetPosition_keepsOutputFormatInVideoFrameMetadataL throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + fakeSampleStream.writeData(/* startPositionUs= */ 0); mediaCodecVideoRenderer.enable( RendererConfiguration.DEFAULT, new Format[] {VIDEO_H264}, @@ -270,9 +283,10 @@ public void render_includingResetPosition_keepsOutputFormatInVideoFrameMetadataL mediaCodecVideoRenderer.render(/* positionUs= */ 0, SystemClock.elapsedRealtime() * 1000); mediaCodecVideoRenderer.resetPosition(0); mediaCodecVideoRenderer.setCurrentStreamFinal(); - fakeSampleStream.addFakeSampleStreamItem( - oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME)); - fakeSampleStream.addFakeSampleStreamItem(FakeSampleStreamItem.END_OF_STREAM_ITEM); + fakeSampleStream.append( + ImmutableList.of( + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); int positionUs = 10; do { mediaCodecVideoRenderer.render(positionUs, SystemClock.elapsedRealtime() * 1000); @@ -287,11 +301,13 @@ public void render_includingResetPosition_keepsOutputFormatInVideoFrameMetadataL public void enable_withMayRenderStartOfStream_rendersFirstFrameBeforeStart() throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + fakeSampleStream.writeData(/* startPositionUs= */ 0); mediaCodecVideoRenderer.enable( RendererConfiguration.DEFAULT, @@ -315,11 +331,13 @@ public void enable_withoutMayRenderStartOfStream_doesNotRenderFirstFrameBeforeSt throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, - ImmutableList.of(oneByteSample(/* timeUs= */ 0))); + ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + fakeSampleStream.writeData(/* startPositionUs= */ 0); mediaCodecVideoRenderer.enable( RendererConfiguration.DEFAULT, @@ -342,11 +360,13 @@ public void enable_withoutMayRenderStartOfStream_doesNotRenderFirstFrameBeforeSt public void enable_withoutMayRenderStartOfStream_rendersFirstFrameAfterStart() throws Exception { FakeSampleStream fakeSampleStream = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + fakeSampleStream.writeData(/* startPositionUs= */ 0); mediaCodecVideoRenderer.enable( RendererConfiguration.DEFAULT, @@ -371,22 +391,25 @@ public void replaceStream_rendersFirstFrameOnlyAfterStartPosition() throws Excep ShadowLooper shadowLooper = shadowOf(testMainLooper); FakeSampleStream fakeSampleStream1 = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of( - oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), - FakeSampleStreamItem.END_OF_STREAM_ITEM)); + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream1.writeData(/* startPositionUs= */ 0); FakeSampleStream fakeSampleStream2 = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of( oneByteSample(/* timeUs= */ 1_000_000, C.BUFFER_FLAG_KEY_FRAME), - FakeSampleStreamItem.END_OF_STREAM_ITEM)); + END_OF_STREAM_ITEM)); + fakeSampleStream2.writeData(/* startPositionUs= */ 0); mediaCodecVideoRenderer.enable( RendererConfiguration.DEFAULT, new Format[] {VIDEO_H264}, @@ -422,22 +445,24 @@ public void replaceStream_whenNotStarted_doesNotRenderFirstFrameOfNewStream() th ShadowLooper shadowLooper = shadowOf(testMainLooper); FakeSampleStream fakeSampleStream1 = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of( - oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), - FakeSampleStreamItem.END_OF_STREAM_ITEM)); + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream1.writeData(/* startPositionUs= */ 0); FakeSampleStream fakeSampleStream2 = new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, - DrmSessionManager.DUMMY, + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), /* initialFormat= */ VIDEO_H264, ImmutableList.of( - oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), - FakeSampleStreamItem.END_OF_STREAM_ITEM)); + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream2.writeData(/* startPositionUs= */ 0); mediaCodecVideoRenderer.enable( RendererConfiguration.DEFAULT, new Format[] {VIDEO_H264}, diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashChunkSource.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashChunkSource.java index e12a67a7547..d93915f761c 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashChunkSource.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashChunkSource.java @@ -21,14 +21,12 @@ import com.google.android.exoplayer2.source.chunk.ChunkSource; import com.google.android.exoplayer2.source.dash.PlayerEmsgHandler.PlayerTrackEmsgHandler; import com.google.android.exoplayer2.source.dash.manifest.DashManifest; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.LoaderErrorThrower; import com.google.android.exoplayer2.upstream.TransferListener; import java.util.List; -/** - * An {@link ChunkSource} for DASH streams. - */ +/** A {@link ChunkSource} for DASH streams. */ public interface DashChunkSource extends ChunkSource { /** Factory for {@link DashChunkSource}s. */ @@ -55,7 +53,7 @@ DashChunkSource createDashChunkSource( DashManifest manifest, int periodIndex, int[] adaptationSetIndices, - TrackSelection trackSelection, + ExoTrackSelection trackSelection, int type, long elapsedRealtimeOffsetMs, boolean enableEventMessageTrack, @@ -76,5 +74,5 @@ DashChunkSource createDashChunkSource( * * @param trackSelection The new track selection instance. Must be equivalent to the previous one. */ - void updateTrackSelection(TrackSelection trackSelection); + void updateTrackSelection(ExoTrackSelection trackSelection); } diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashMediaPeriod.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashMediaPeriod.java index 81d72b61f3c..6cf10b35787 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashMediaPeriod.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashMediaPeriod.java @@ -47,7 +47,7 @@ import com.google.android.exoplayer2.source.dash.manifest.EventStream; import com.google.android.exoplayer2.source.dash.manifest.Period; import com.google.android.exoplayer2.source.dash.manifest.Representation; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy; import com.google.android.exoplayer2.upstream.LoaderErrorThrower; @@ -213,10 +213,10 @@ public TrackGroupArray getTrackGroups() { } @Override - public List getStreamKeys(List trackSelections) { + public List getStreamKeys(List trackSelections) { List manifestAdaptationSets = manifest.getPeriod(periodIndex).adaptationSets; List streamKeys = new ArrayList<>(); - for (TrackSelection trackSelection : trackSelections) { + for (ExoTrackSelection trackSelection : trackSelections) { int trackGroupIndex = trackGroups.indexOf(trackSelection.getTrackGroup()); TrackGroupInfo trackGroupInfo = trackGroupInfos[trackGroupIndex]; if (trackGroupInfo.trackGroupCategory != TrackGroupInfo.CATEGORY_PRIMARY) { @@ -256,7 +256,7 @@ public List getStreamKeys(List trackSelections) { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, @@ -356,7 +356,7 @@ public void onContinueLoadingRequested(ChunkSampleStream sample // Internal methods. - private int[] getStreamIndexToTrackGroupIndex(TrackSelection[] selections) { + private int[] getStreamIndexToTrackGroupIndex(ExoTrackSelection[] selections) { int[] streamIndexToTrackGroupIndex = new int[selections.length]; for (int i = 0; i < selections.length; i++) { if (selections[i] != null) { @@ -369,7 +369,7 @@ private int[] getStreamIndexToTrackGroupIndex(TrackSelection[] selections) { } private void releaseDisabledStreams( - TrackSelection[] selections, boolean[] mayRetainStreamFlags, SampleStream[] streams) { + ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, SampleStream[] streams) { for (int i = 0; i < selections.length; i++) { if (selections[i] == null || !mayRetainStreamFlags[i]) { if (streams[i] instanceof ChunkSampleStream) { @@ -386,7 +386,7 @@ private void releaseDisabledStreams( } private void releaseOrphanEmbeddedStreams( - TrackSelection[] selections, SampleStream[] streams, int[] streamIndexToTrackGroupIndex) { + ExoTrackSelection[] selections, SampleStream[] streams, int[] streamIndexToTrackGroupIndex) { for (int i = 0; i < selections.length; i++) { if (streams[i] instanceof EmptySampleStream || streams[i] instanceof EmbeddedSampleStream) { // We need to release an embedded stream if the corresponding primary stream is released. @@ -414,14 +414,14 @@ private void releaseOrphanEmbeddedStreams( } private void selectNewStreams( - TrackSelection[] selections, + ExoTrackSelection[] selections, SampleStream[] streams, boolean[] streamResetFlags, long positionUs, int[] streamIndexToTrackGroupIndex) { // Create newly selected primary and event streams. for (int i = 0; i < selections.length; i++) { - TrackSelection selection = selections[i]; + ExoTrackSelection selection = selections[i]; if (selection == null) { continue; } @@ -703,8 +703,11 @@ private static int buildPrimaryAndEmbeddedTrackGroupInfos( return trackGroupCount; } - private static void buildManifestEventTrackGroupInfos(List eventStreams, - TrackGroup[] trackGroups, TrackGroupInfo[] trackGroupInfos, int existingTrackGroupCount) { + private static void buildManifestEventTrackGroupInfos( + List eventStreams, + TrackGroup[] trackGroups, + TrackGroupInfo[] trackGroupInfos, + int existingTrackGroupCount) { for (int i = 0; i < eventStreams.size(); i++) { EventStream eventStream = eventStreams.get(i); Format format = @@ -717,8 +720,8 @@ private static void buildManifestEventTrackGroupInfos(List eventStr } } - private ChunkSampleStream buildSampleStream(TrackGroupInfo trackGroupInfo, - TrackSelection selection, long positionUs) { + private ChunkSampleStream buildSampleStream( + TrackGroupInfo trackGroupInfo, ExoTrackSelection selection, long positionUs) { int embeddedTrackCount = 0; boolean enableEventMessageTrack = trackGroupInfo.embeddedEventMessageTrackGroupIndex != C.INDEX_UNSET; @@ -813,8 +816,8 @@ private static Descriptor findDescriptor(List descriptors, String sc return null; } - private static boolean hasEventMessageTrack(List adaptationSets, - int[] adaptationSetIndices) { + private static boolean hasEventMessageTrack( + List adaptationSets, int[] adaptationSetIndices) { for (int i : adaptationSetIndices) { List representations = adaptationSets.get(i).representations; for (int j = 0; j < representations.size(); j++) { @@ -897,8 +900,8 @@ private static final class TrackGroupInfo { public @interface TrackGroupCategory {} /** - * A normal track group that has its samples drawn from the stream. - * For example: a video Track Group or an audio Track Group. + * A normal track group that has its samples drawn from the stream. For example: a video Track + * Group or an audio Track Group. */ private static final int CATEGORY_PRIMARY = 0; @@ -909,9 +912,8 @@ private static final class TrackGroupInfo { private static final int CATEGORY_EMBEDDED = 1; /** - * A track group that has its samples listed explicitly in the DASH manifest file. - * For example: an EventStream track has its sample (Events) included directly in the DASH - * manifest file. + * A track group that has its samples listed explicitly in the DASH manifest file. For example: + * an EventStream track has its sample (Events) included directly in the DASH manifest file. */ private static final int CATEGORY_MANIFEST_EVENTS = 2; @@ -940,8 +942,8 @@ public static TrackGroupInfo primaryTrack( /* eventStreamGroupIndex= */ -1); } - public static TrackGroupInfo embeddedEmsgTrack(int[] adaptationSetIndices, - int primaryTrackGroupIndex) { + public static TrackGroupInfo embeddedEmsgTrack( + int[] adaptationSetIndices, int primaryTrackGroupIndex) { return new TrackGroupInfo( C.TRACK_TYPE_METADATA, CATEGORY_EMBEDDED, @@ -992,5 +994,4 @@ private TrackGroupInfo( this.eventStreamGroupIndex = eventStreamGroupIndex; } } - } diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashMediaSource.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashMediaSource.java index 2f5b169e30d..258ebf32704 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashMediaSource.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashMediaSource.java @@ -16,6 +16,8 @@ package com.google.android.exoplayer2.source.dash; import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static com.google.android.exoplayer2.util.Util.castNonNull; import static java.lang.Math.max; import static java.lang.Math.min; @@ -29,9 +31,12 @@ import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.ParserException; +import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.drm.DefaultDrmSessionManagerProvider; import com.google.android.exoplayer2.drm.DrmSessionEventListener; import com.google.android.exoplayer2.drm.DrmSessionManager; +import com.google.android.exoplayer2.drm.DrmSessionManagerProvider; import com.google.android.exoplayer2.offline.FilteringManifestParser; import com.google.android.exoplayer2.offline.StreamKey; import com.google.android.exoplayer2.source.BaseMediaSource; @@ -41,7 +46,6 @@ import com.google.android.exoplayer2.source.MediaLoadData; import com.google.android.exoplayer2.source.MediaPeriod; import com.google.android.exoplayer2.source.MediaSource; -import com.google.android.exoplayer2.source.MediaSourceDrmHelper; import com.google.android.exoplayer2.source.MediaSourceEventListener; import com.google.android.exoplayer2.source.MediaSourceEventListener.EventDispatcher; import com.google.android.exoplayer2.source.MediaSourceFactory; @@ -50,6 +54,8 @@ import com.google.android.exoplayer2.source.dash.manifest.AdaptationSet; import com.google.android.exoplayer2.source.dash.manifest.DashManifest; import com.google.android.exoplayer2.source.dash.manifest.DashManifestParser; +import com.google.android.exoplayer2.source.dash.manifest.Period; +import com.google.android.exoplayer2.source.dash.manifest.Representation; import com.google.android.exoplayer2.source.dash.manifest.UtcTimingElement; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.DataSource; @@ -68,10 +74,12 @@ import com.google.android.exoplayer2.util.SntpClient; import com.google.android.exoplayer2.util.Util; import com.google.common.base.Charsets; +import com.google.common.math.LongMath; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; +import java.math.RoundingMode; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Collections; @@ -92,14 +100,14 @@ public final class DashMediaSource extends BaseMediaSource { public static final class Factory implements MediaSourceFactory { private final DashChunkSource.Factory chunkSourceFactory; - private final MediaSourceDrmHelper mediaSourceDrmHelper; @Nullable private final DataSource.Factory manifestDataSourceFactory; - @Nullable private DrmSessionManager drmSessionManager; + private boolean usingCustomDrmSessionManagerProvider; + private DrmSessionManagerProvider drmSessionManagerProvider; private CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory; private LoadErrorHandlingPolicy loadErrorHandlingPolicy; - private long livePresentationDelayMs; - private boolean livePresentationDelayOverridesManifest; + private long targetLiveOffsetOverrideMs; + private long fallbackTargetLiveOffsetMs; @Nullable private ParsingLoadable.Parser manifestParser; private List streamKeys; @Nullable private Object tag; @@ -121,16 +129,17 @@ public Factory(DataSource.Factory dataSourceFactory) { * @param manifestDataSourceFactory A factory for {@link DataSource} instances that will be used * to load (and refresh) the manifest. May be {@code null} if the factory will only ever be * used to create create media sources with sideloaded manifests via {@link - * #createMediaSource(DashManifest, Handler, MediaSourceEventListener)}. + * #createMediaSource(DashManifest, MediaItem)}. */ public Factory( DashChunkSource.Factory chunkSourceFactory, @Nullable DataSource.Factory manifestDataSourceFactory) { this.chunkSourceFactory = checkNotNull(chunkSourceFactory); this.manifestDataSourceFactory = manifestDataSourceFactory; - mediaSourceDrmHelper = new MediaSourceDrmHelper(); + drmSessionManagerProvider = new DefaultDrmSessionManagerProvider(); loadErrorHandlingPolicy = new DefaultLoadErrorHandlingPolicy(); - livePresentationDelayMs = DEFAULT_LIVE_PRESENTATION_DELAY_MS; + targetLiveOffsetOverrideMs = C.TIME_UNSET; + fallbackTargetLiveOffsetMs = DEFAULT_FALLBACK_TARGET_LIVE_OFFSET_MS; compositeSequenceableLoaderFactory = new DefaultCompositeSequenceableLoaderFactory(); streamKeys = Collections.emptyList(); } @@ -157,37 +166,51 @@ public Factory setStreamKeys(@Nullable List streamKeys) { return this; } + @Override + public Factory setDrmSessionManagerProvider( + @Nullable DrmSessionManagerProvider drmSessionManagerProvider) { + if (drmSessionManagerProvider != null) { + this.drmSessionManagerProvider = drmSessionManagerProvider; + this.usingCustomDrmSessionManagerProvider = true; + } else { + this.drmSessionManagerProvider = new DefaultDrmSessionManagerProvider(); + this.usingCustomDrmSessionManagerProvider = false; + } + return this; + } + @Override public Factory setDrmSessionManager(@Nullable DrmSessionManager drmSessionManager) { - this.drmSessionManager = drmSessionManager; + if (drmSessionManager == null) { + setDrmSessionManagerProvider(null); + } else { + setDrmSessionManagerProvider(unusedMediaItem -> drmSessionManager); + } return this; } @Override public Factory setDrmHttpDataSourceFactory( @Nullable HttpDataSource.Factory drmHttpDataSourceFactory) { - mediaSourceDrmHelper.setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + if (!usingCustomDrmSessionManagerProvider) { + ((DefaultDrmSessionManagerProvider) drmSessionManagerProvider) + .setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + } return this; } @Override public Factory setDrmUserAgent(@Nullable String userAgent) { - mediaSourceDrmHelper.setDrmUserAgent(userAgent); + if (!usingCustomDrmSessionManagerProvider) { + ((DefaultDrmSessionManagerProvider) drmSessionManagerProvider).setDrmUserAgent(userAgent); + } return this; } - /** @deprecated Use {@link #setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)} instead. */ - @Deprecated - public Factory setMinLoadableRetryCount(int minLoadableRetryCount) { - return setLoadErrorHandlingPolicy(new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount)); - } - /** * Sets the {@link LoadErrorHandlingPolicy}. The default value is created by calling {@link * DefaultLoadErrorHandlingPolicy#DefaultLoadErrorHandlingPolicy()}. * - *

    Calling this method overrides any calls to {@link #setMinLoadableRetryCount(int)}. - * * @param loadErrorHandlingPolicy A {@link LoadErrorHandlingPolicy}. * @return This factory, for convenience. */ @@ -200,34 +223,31 @@ public Factory setLoadErrorHandlingPolicy( return this; } - /** @deprecated Use {@link #setLivePresentationDelayMs(long, boolean)} instead. */ + /** + * @deprecated Use {@link MediaItem.Builder#setLiveTargetOffsetMs(long)} to override the + * manifest, or {@link #setFallbackTargetLiveOffsetMs(long)} to provide a fallback value. + */ @Deprecated - @SuppressWarnings("deprecation") - public Factory setLivePresentationDelayMs(long livePresentationDelayMs) { - if (livePresentationDelayMs == DEFAULT_LIVE_PRESENTATION_DELAY_PREFER_MANIFEST_MS) { - return setLivePresentationDelayMs(DEFAULT_LIVE_PRESENTATION_DELAY_MS, false); - } else { - return setLivePresentationDelayMs(livePresentationDelayMs, true); + public Factory setLivePresentationDelayMs( + long livePresentationDelayMs, boolean overridesManifest) { + targetLiveOffsetOverrideMs = overridesManifest ? livePresentationDelayMs : C.TIME_UNSET; + if (!overridesManifest) { + setFallbackTargetLiveOffsetMs(livePresentationDelayMs); } + return this; } /** - * Sets the duration in milliseconds by which the default start position should precede the end - * of the live window for live playbacks. The {@code overridesManifest} parameter specifies - * whether the value is used in preference to one in the manifest, if present. The default value - * is {@link #DEFAULT_LIVE_PRESENTATION_DELAY_MS}, and by default {@code overridesManifest} is - * false. + * Sets the target {@link Player#getCurrentLiveOffset() offset for live streams} that is used if + * no value is defined in the {@link MediaItem} or the manifest. + * + *

    The default value is {@link #DEFAULT_FALLBACK_TARGET_LIVE_OFFSET_MS}. * - * @param livePresentationDelayMs For live playbacks, the duration in milliseconds by which the - * default start position should precede the end of the live window. - * @param overridesManifest Whether the value is used in preference to one in the manifest, if - * present. + * @param fallbackTargetLiveOffsetMs The fallback live target offset in milliseconds. * @return This factory, for convenience. */ - public Factory setLivePresentationDelayMs( - long livePresentationDelayMs, boolean overridesManifest) { - this.livePresentationDelayMs = livePresentationDelayMs; - this.livePresentationDelayOverridesManifest = overridesManifest; + public Factory setFallbackTargetLiveOffsetMs(long fallbackTargetLiveOffsetMs) { + this.fallbackTargetLiveOffsetMs = fallbackTargetLiveOffsetMs; return this; } @@ -275,7 +295,7 @@ public DashMediaSource createMediaSource(DashManifest manifest) { manifest, new MediaItem.Builder() .setUri(Uri.EMPTY) - .setMediaId(DUMMY_MEDIA_ID) + .setMediaId(DEFAULT_MEDIA_ID) .setMimeType(MimeTypes.APPLICATION_MPD) .setStreamKeys(streamKeys) .setTag(tag) @@ -302,12 +322,17 @@ public DashMediaSource createMediaSource(DashManifest manifest, MediaItem mediaI } boolean hasUri = mediaItem.playbackProperties != null; boolean hasTag = hasUri && mediaItem.playbackProperties.tag != null; + boolean hasTargetLiveOffset = mediaItem.liveConfiguration.targetOffsetMs != C.TIME_UNSET; mediaItem = mediaItem .buildUpon() .setMimeType(MimeTypes.APPLICATION_MPD) .setUri(hasUri ? mediaItem.playbackProperties.uri : Uri.EMPTY) .setTag(hasTag ? mediaItem.playbackProperties.tag : tag) + .setLiveTargetOffsetMs( + hasTargetLiveOffset + ? mediaItem.liveConfiguration.targetOffsetMs + : targetLiveOffsetOverrideMs) .setStreamKeys(streamKeys) .build(); return new DashMediaSource( @@ -317,43 +342,9 @@ public DashMediaSource createMediaSource(DashManifest manifest, MediaItem mediaI /* manifestParser= */ null, chunkSourceFactory, compositeSequenceableLoaderFactory, - drmSessionManager != null ? drmSessionManager : mediaSourceDrmHelper.create(mediaItem), + drmSessionManagerProvider.get(mediaItem), loadErrorHandlingPolicy, - livePresentationDelayMs, - livePresentationDelayOverridesManifest); - } - - /** - * @deprecated Use {@link #createMediaSource(DashManifest)} and {@link - * #addEventListener(Handler, MediaSourceEventListener)} instead. - */ - @Deprecated - public DashMediaSource createMediaSource( - DashManifest manifest, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - DashMediaSource mediaSource = createMediaSource(manifest); - if (eventHandler != null && eventListener != null) { - mediaSource.addEventListener(eventHandler, eventListener); - } - return mediaSource; - } - - /** - * @deprecated Use {@link #createMediaSource(MediaItem)} and {@link #addEventListener(Handler, - * MediaSourceEventListener)} instead. - */ - @SuppressWarnings("deprecation") - @Deprecated - public DashMediaSource createMediaSource( - Uri manifestUri, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - DashMediaSource mediaSource = createMediaSource(manifestUri); - if (eventHandler != null && eventListener != null) { - mediaSource.addEventListener(eventHandler, eventListener); - } - return mediaSource; + fallbackTargetLiveOffsetMs); } /** @deprecated Use {@link #createMediaSource(MediaItem)} instead. */ @@ -394,12 +385,21 @@ public DashMediaSource createMediaSource(MediaItem mediaItem) { boolean needsTag = mediaItem.playbackProperties.tag == null && tag != null; boolean needsStreamKeys = mediaItem.playbackProperties.streamKeys.isEmpty() && !streamKeys.isEmpty(); - if (needsTag && needsStreamKeys) { - mediaItem = mediaItem.buildUpon().setTag(tag).setStreamKeys(streamKeys).build(); - } else if (needsTag) { - mediaItem = mediaItem.buildUpon().setTag(tag).build(); - } else if (needsStreamKeys) { - mediaItem = mediaItem.buildUpon().setStreamKeys(streamKeys).build(); + boolean needsTargetLiveOffset = + mediaItem.liveConfiguration.targetOffsetMs == C.TIME_UNSET + && targetLiveOffsetOverrideMs != C.TIME_UNSET; + if (needsTag || needsStreamKeys || needsTargetLiveOffset) { + MediaItem.Builder builder = mediaItem.buildUpon(); + if (needsTag) { + builder.setTag(tag); + } + if (needsStreamKeys) { + builder.setStreamKeys(streamKeys); + } + if (needsTargetLiveOffset) { + builder.setLiveTargetOffsetMs(targetLiveOffsetOverrideMs); + } + mediaItem = builder.build(); } return new DashMediaSource( mediaItem, @@ -408,10 +408,9 @@ public DashMediaSource createMediaSource(MediaItem mediaItem) { manifestParser, chunkSourceFactory, compositeSequenceableLoaderFactory, - drmSessionManager != null ? drmSessionManager : mediaSourceDrmHelper.create(mediaItem), + drmSessionManagerProvider.get(mediaItem), loadErrorHandlingPolicy, - livePresentationDelayMs, - livePresentationDelayOverridesManifest); + fallbackTargetLiveOffsetMs); } @Override @@ -421,27 +420,21 @@ public int[] getSupportedTypes() { } /** - * The default presentation delay for live streams. The presentation delay is the duration by - * which the default start position precedes the end of the live window. + * The default target {@link Player#getCurrentLiveOffset() offset for live streams} that is used + * if no value is defined in the {@link MediaItem} or the manifest. */ - public static final long DEFAULT_LIVE_PRESENTATION_DELAY_MS = 30_000; - /** @deprecated Use {@link #DEFAULT_LIVE_PRESENTATION_DELAY_MS}. */ - @Deprecated - public static final long DEFAULT_LIVE_PRESENTATION_DELAY_FIXED_MS = - DEFAULT_LIVE_PRESENTATION_DELAY_MS; - /** @deprecated Use of this parameter is no longer necessary. */ - @Deprecated public static final long DEFAULT_LIVE_PRESENTATION_DELAY_PREFER_MANIFEST_MS = -1; - + public static final long DEFAULT_FALLBACK_TARGET_LIVE_OFFSET_MS = 30_000; + /** @deprecated Use {@link #DEFAULT_FALLBACK_TARGET_LIVE_OFFSET_MS} instead. */ + @Deprecated public static final long DEFAULT_LIVE_PRESENTATION_DELAY_MS = 30_000; /** The media id used by media items of dash media sources without a manifest URI. */ - public static final String DUMMY_MEDIA_ID = - "com.google.android.exoplayer2.source.dash.DashMediaSource"; + public static final String DEFAULT_MEDIA_ID = "DashMediaSource"; /** * The interval in milliseconds between invocations of {@link * MediaSourceCaller#onSourceInfoRefreshed(MediaSource, Timeline)} when the source's {@link * Timeline} is changing dynamically (for example, for incomplete live streams). */ - private static final int NOTIFY_MANIFEST_INTERVAL_MS = 5000; + private static final long DEFAULT_NOTIFY_MANIFEST_INTERVAL_MS = 5000; /** * The minimum default start position for live streams, relative to the start of the live window. */ @@ -449,14 +442,14 @@ public int[] getSupportedTypes() { private static final String TAG = "DashMediaSource"; + private final MediaItem mediaItem; private final boolean sideloadedManifest; private final DataSource.Factory manifestDataSourceFactory; private final DashChunkSource.Factory chunkSourceFactory; private final CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory; private final DrmSessionManager drmSessionManager; private final LoadErrorHandlingPolicy loadErrorHandlingPolicy; - private final long livePresentationDelayMs; - private final boolean livePresentationDelayOverridesManifest; + private final long fallbackTargetLiveOffsetMs; private final EventDispatcher manifestEventDispatcher; private final ParsingLoadable.Parser manifestParser; private final ManifestCallback manifestCallback; @@ -466,8 +459,6 @@ public int[] getSupportedTypes() { private final Runnable simulateManifestRefreshRunnable; private final PlayerEmsgCallback playerEmsgCallback; private final LoaderErrorThrower manifestLoadErrorThrower; - private final MediaItem mediaItem; - private final MediaItem.PlaybackProperties playbackProperties; private DataSource dataSource; private Loader loader; @@ -476,6 +467,7 @@ public int[] getSupportedTypes() { private IOException manifestFatalError; private Handler handler; + private MediaItem.LiveConfiguration liveConfiguration; private Uri manifestUri; private Uri initialManifestUri; private DashManifest manifest; @@ -489,121 +481,6 @@ public int[] getSupportedTypes() { private int firstPeriodId; - /** @deprecated Use {@link Factory} instead. */ - @Deprecated - @SuppressWarnings("deprecation") - public DashMediaSource( - DashManifest manifest, - DashChunkSource.Factory chunkSourceFactory, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - manifest, - chunkSourceFactory, - DefaultLoadErrorHandlingPolicy.DEFAULT_MIN_LOADABLE_RETRY_COUNT, - eventHandler, - eventListener); - } - - /** @deprecated Use {@link Factory} instead. */ - @Deprecated - public DashMediaSource( - DashManifest manifest, - DashChunkSource.Factory chunkSourceFactory, - int minLoadableRetryCount, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - new MediaItem.Builder() - .setMediaId(DUMMY_MEDIA_ID) - .setMimeType(MimeTypes.APPLICATION_MPD) - .setUri(Uri.EMPTY) - .build(), - manifest, - /* manifestDataSourceFactory= */ null, - /* manifestParser= */ null, - chunkSourceFactory, - new DefaultCompositeSequenceableLoaderFactory(), - DrmSessionManager.getDummyDrmSessionManager(), - new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount), - DEFAULT_LIVE_PRESENTATION_DELAY_MS, - /* livePresentationDelayOverridesManifest= */ false); - if (eventHandler != null && eventListener != null) { - addEventListener(eventHandler, eventListener); - } - } - - /** @deprecated Use {@link Factory} instead. */ - @Deprecated - @SuppressWarnings("deprecation") - public DashMediaSource( - Uri manifestUri, - DataSource.Factory manifestDataSourceFactory, - DashChunkSource.Factory chunkSourceFactory, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - manifestUri, - manifestDataSourceFactory, - chunkSourceFactory, - DefaultLoadErrorHandlingPolicy.DEFAULT_MIN_LOADABLE_RETRY_COUNT, - DEFAULT_LIVE_PRESENTATION_DELAY_PREFER_MANIFEST_MS, - eventHandler, - eventListener); - } - - /** @deprecated Use {@link Factory} instead. */ - @Deprecated - @SuppressWarnings("deprecation") - public DashMediaSource( - Uri manifestUri, - DataSource.Factory manifestDataSourceFactory, - DashChunkSource.Factory chunkSourceFactory, - int minLoadableRetryCount, - long livePresentationDelayMs, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - manifestUri, - manifestDataSourceFactory, - new DashManifestParser(), - chunkSourceFactory, - minLoadableRetryCount, - livePresentationDelayMs, - eventHandler, - eventListener); - } - - /** @deprecated Use {@link Factory} instead. */ - @Deprecated - @SuppressWarnings("deprecation") - public DashMediaSource( - Uri manifestUri, - DataSource.Factory manifestDataSourceFactory, - ParsingLoadable.Parser manifestParser, - DashChunkSource.Factory chunkSourceFactory, - int minLoadableRetryCount, - long livePresentationDelayMs, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - new MediaItem.Builder().setUri(manifestUri).setMimeType(MimeTypes.APPLICATION_MPD).build(), - /* manifest= */ null, - manifestDataSourceFactory, - manifestParser, - chunkSourceFactory, - new DefaultCompositeSequenceableLoaderFactory(), - DrmSessionManager.getDummyDrmSessionManager(), - new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount), - livePresentationDelayMs == DEFAULT_LIVE_PRESENTATION_DELAY_PREFER_MANIFEST_MS - ? DEFAULT_LIVE_PRESENTATION_DELAY_MS - : livePresentationDelayMs, - livePresentationDelayMs != DEFAULT_LIVE_PRESENTATION_DELAY_PREFER_MANIFEST_MS); - if (eventHandler != null && eventListener != null) { - addEventListener(eventHandler, eventListener); - } - } - private DashMediaSource( MediaItem mediaItem, @Nullable DashManifest manifest, @@ -613,20 +490,18 @@ private DashMediaSource( CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory, DrmSessionManager drmSessionManager, LoadErrorHandlingPolicy loadErrorHandlingPolicy, - long livePresentationDelayMs, - boolean livePresentationDelayOverridesManifest) { + long fallbackTargetLiveOffsetMs) { this.mediaItem = mediaItem; - this.playbackProperties = checkNotNull(mediaItem.playbackProperties); - this.manifestUri = playbackProperties.uri; - this.initialManifestUri = playbackProperties.uri; + this.liveConfiguration = mediaItem.liveConfiguration; + this.manifestUri = checkNotNull(mediaItem.playbackProperties).uri; + this.initialManifestUri = mediaItem.playbackProperties.uri; this.manifest = manifest; this.manifestDataSourceFactory = manifestDataSourceFactory; this.manifestParser = manifestParser; this.chunkSourceFactory = chunkSourceFactory; this.drmSessionManager = drmSessionManager; this.loadErrorHandlingPolicy = loadErrorHandlingPolicy; - this.livePresentationDelayMs = livePresentationDelayMs; - this.livePresentationDelayOverridesManifest = livePresentationDelayOverridesManifest; + this.fallbackTargetLiveOffsetMs = fallbackTargetLiveOffsetMs; this.compositeSequenceableLoaderFactory = compositeSequenceableLoaderFactory; sideloadedManifest = manifest != null; manifestEventDispatcher = createEventDispatcher(/* mediaPeriodId= */ null); @@ -671,7 +546,7 @@ public void replaceManifestUri(Uri manifestUri) { @Override @Nullable public Object getTag() { - return playbackProperties.tag; + return castNonNull(mediaItem.playbackProperties).tag; } @Override @@ -1028,20 +903,18 @@ private void processManifest(boolean scheduleRefresh) { // Update the window. boolean windowChangingImplicitly = false; int lastPeriodIndex = manifest.getPeriodCount() - 1; - PeriodSeekInfo firstPeriodSeekInfo = PeriodSeekInfo.createPeriodSeekInfo(manifest.getPeriod(0), - manifest.getPeriodDurationUs(0)); - PeriodSeekInfo lastPeriodSeekInfo = PeriodSeekInfo.createPeriodSeekInfo( - manifest.getPeriod(lastPeriodIndex), manifest.getPeriodDurationUs(lastPeriodIndex)); + Period lastPeriod = manifest.getPeriod(lastPeriodIndex); + long lastPeriodDurationUs = manifest.getPeriodDurationUs(lastPeriodIndex); + long nowUnixTimeUs = C.msToUs(Util.getNowUnixTimeMs(elapsedRealtimeOffsetMs)); // Get the period-relative start/end times. - long currentStartTimeUs = firstPeriodSeekInfo.availableStartTimeUs; - long currentEndTimeUs = lastPeriodSeekInfo.availableEndTimeUs; - if (manifest.dynamic && !lastPeriodSeekInfo.isIndexExplicit) { + long currentStartTimeUs = + getAvailableStartTimeUs( + manifest.getPeriod(0), manifest.getPeriodDurationUs(0), nowUnixTimeUs); + long currentEndTimeUs = getAvailableEndTimeUs(lastPeriod, lastPeriodDurationUs, nowUnixTimeUs); + if (manifest.dynamic && !isIndexExplicit(lastPeriod)) { // The manifest describes an incomplete live stream. Update the start/end times to reflect the // live stream duration and the manifest's time shift buffer depth. - long nowUnixTimeUs = C.msToUs(Util.getNowUnixTimeMs(elapsedRealtimeOffsetMs)); - long liveStreamDurationUs = nowUnixTimeUs - C.msToUs(manifest.availabilityStartTimeMs); - long liveStreamEndPositionInLastPeriodUs = liveStreamDurationUs - - C.msToUs(manifest.getPeriod(lastPeriodIndex).startMs); + long liveStreamEndPositionInLastPeriodUs = currentEndTimeUs - C.msToUs(lastPeriod.startMs); currentEndTimeUs = min(liveStreamEndPositionInLastPeriodUs, currentEndTimeUs); if (manifest.timeShiftBufferDepthMs != C.TIME_UNSET) { long timeShiftBufferDepthUs = C.msToUs(manifest.timeShiftBufferDepthMs); @@ -1064,23 +937,7 @@ private void processManifest(boolean scheduleRefresh) { for (int i = 0; i < manifest.getPeriodCount() - 1; i++) { windowDurationUs += manifest.getPeriodDurationUs(i); } - long windowDefaultStartPositionUs = 0; - if (manifest.dynamic) { - long presentationDelayForManifestMs = livePresentationDelayMs; - if (!livePresentationDelayOverridesManifest - && manifest.suggestedPresentationDelayMs != C.TIME_UNSET) { - presentationDelayForManifestMs = manifest.suggestedPresentationDelayMs; - } - // Snap the default position to the start of the segment containing it. - windowDefaultStartPositionUs = windowDurationUs - C.msToUs(presentationDelayForManifestMs); - if (windowDefaultStartPositionUs < MIN_LIVE_DEFAULT_START_POSITION_US) { - // The default start position is too close to the start of the live window. Set it to the - // minimum default start position provided the window is at least twice as big. Else set - // it to the middle of the window. - windowDefaultStartPositionUs = - min(MIN_LIVE_DEFAULT_START_POSITION_US, windowDurationUs / 2); - } - } + long windowStartTimeMs = C.TIME_UNSET; if (manifest.availabilityStartTimeMs != C.TIME_UNSET) { windowStartTimeMs = @@ -1088,17 +945,36 @@ private void processManifest(boolean scheduleRefresh) { + manifest.getPeriod(0).startMs + C.usToMs(currentStartTimeUs); } + + long windowDefaultStartPositionUs = 0; + if (manifest.dynamic) { + updateMediaItemLiveConfiguration( + /* nowPeriodTimeUs= */ currentStartTimeUs + nowUnixTimeUs - C.msToUs(windowStartTimeMs), + /* windowStartPeriodTimeUs= */ currentStartTimeUs, + /* windowEndPeriodTimeUs= */ currentEndTimeUs); + windowDefaultStartPositionUs = + nowUnixTimeUs - C.msToUs(windowStartTimeMs + liveConfiguration.targetOffsetMs); + long minimumDefaultStartPositionUs = + min(MIN_LIVE_DEFAULT_START_POSITION_US, windowDurationUs / 2); + if (windowDefaultStartPositionUs < minimumDefaultStartPositionUs) { + // The default start position is too close to the start of the live window. Set it to the + // minimum default start position provided the window is at least twice as big. Else set + // it to the middle of the window. + windowDefaultStartPositionUs = minimumDefaultStartPositionUs; + } + } DashTimeline timeline = new DashTimeline( manifest.availabilityStartTimeMs, windowStartTimeMs, elapsedRealtimeOffsetMs, firstPeriodId, - currentStartTimeUs, + /* offsetInFirstPeriodUs= */ currentStartTimeUs, windowDurationUs, windowDefaultStartPositionUs, manifest, - mediaItem); + mediaItem, + manifest.dynamic ? liveConfiguration : null); refreshSourceInfo(timeline); if (!sideloadedManifest) { @@ -1106,7 +982,10 @@ private void processManifest(boolean scheduleRefresh) { handler.removeCallbacks(simulateManifestRefreshRunnable); // If the window is changing implicitly, post a simulated manifest refresh to update it. if (windowChangingImplicitly) { - handler.postDelayed(simulateManifestRefreshRunnable, NOTIFY_MANIFEST_INTERVAL_MS); + handler.postDelayed( + simulateManifestRefreshRunnable, + getIntervalUntilNextManifestRefreshMs( + manifest, Util.getNowUnixTimeMs(elapsedRealtimeOffsetMs))); } if (manifestLoadPending) { startLoadingManifest(); @@ -1129,6 +1008,77 @@ private void processManifest(boolean scheduleRefresh) { } } + private void updateMediaItemLiveConfiguration( + long nowPeriodTimeUs, long windowStartPeriodTimeUs, long windowEndPeriodTimeUs) { + long maxLiveOffsetMs; + if (mediaItem.liveConfiguration.maxOffsetMs != C.TIME_UNSET) { + maxLiveOffsetMs = mediaItem.liveConfiguration.maxOffsetMs; + } else if (manifest.serviceDescription != null + && manifest.serviceDescription.maxOffsetMs != C.TIME_UNSET) { + maxLiveOffsetMs = manifest.serviceDescription.maxOffsetMs; + } else { + maxLiveOffsetMs = C.usToMs(nowPeriodTimeUs - windowStartPeriodTimeUs); + } + long minLiveOffsetMs; + if (mediaItem.liveConfiguration.minOffsetMs != C.TIME_UNSET) { + minLiveOffsetMs = mediaItem.liveConfiguration.minOffsetMs; + } else if (manifest.serviceDescription != null + && manifest.serviceDescription.minOffsetMs != C.TIME_UNSET) { + minLiveOffsetMs = manifest.serviceDescription.minOffsetMs; + } else { + minLiveOffsetMs = C.usToMs(nowPeriodTimeUs - windowEndPeriodTimeUs); + if (minLiveOffsetMs < 0 && maxLiveOffsetMs > 0) { + // The current time is in the window, so assume all clocks are synchronized and set the + // minimum to a live offset of zero. + minLiveOffsetMs = 0; + } + if (manifest.minBufferTimeMs != C.TIME_UNSET) { + minLiveOffsetMs = min(minLiveOffsetMs + manifest.minBufferTimeMs, maxLiveOffsetMs); + } + } + long targetOffsetMs; + if (liveConfiguration.targetOffsetMs != C.TIME_UNSET) { + // Keep existing target offset even if the media configuration changes. + targetOffsetMs = liveConfiguration.targetOffsetMs; + } else if (manifest.serviceDescription != null + && manifest.serviceDescription.targetOffsetMs != C.TIME_UNSET) { + targetOffsetMs = manifest.serviceDescription.targetOffsetMs; + } else if (manifest.suggestedPresentationDelayMs != C.TIME_UNSET) { + targetOffsetMs = manifest.suggestedPresentationDelayMs; + } else { + targetOffsetMs = fallbackTargetLiveOffsetMs; + } + if (targetOffsetMs < minLiveOffsetMs) { + targetOffsetMs = minLiveOffsetMs; + } + if (targetOffsetMs > maxLiveOffsetMs) { + long windowDurationUs = windowEndPeriodTimeUs - windowStartPeriodTimeUs; + long liveOffsetAtWindowStartUs = nowPeriodTimeUs - windowStartPeriodTimeUs; + long safeDistanceFromWindowStartUs = + min(MIN_LIVE_DEFAULT_START_POSITION_US, windowDurationUs / 2); + long maxTargetOffsetForSafeDistanceToWindowStartMs = + C.usToMs(liveOffsetAtWindowStartUs - safeDistanceFromWindowStartUs); + targetOffsetMs = + Util.constrainValue( + maxTargetOffsetForSafeDistanceToWindowStartMs, minLiveOffsetMs, maxLiveOffsetMs); + } + float minPlaybackSpeed = C.RATE_UNSET; + if (mediaItem.liveConfiguration.minPlaybackSpeed != C.RATE_UNSET) { + minPlaybackSpeed = mediaItem.liveConfiguration.minPlaybackSpeed; + } else if (manifest.serviceDescription != null) { + minPlaybackSpeed = manifest.serviceDescription.minPlaybackSpeed; + } + float maxPlaybackSpeed = C.RATE_UNSET; + if (mediaItem.liveConfiguration.maxPlaybackSpeed != C.RATE_UNSET) { + maxPlaybackSpeed = mediaItem.liveConfiguration.maxPlaybackSpeed; + } else if (manifest.serviceDescription != null) { + maxPlaybackSpeed = manifest.serviceDescription.maxPlaybackSpeed; + } + liveConfiguration = + new MediaItem.LiveConfiguration( + targetOffsetMs, minLiveOffsetMs, maxLiveOffsetMs, minPlaybackSpeed, maxPlaybackSpeed); + } + private void scheduleManifestRefresh(long delayUntilNextLoadMs) { handler.postDelayed(refreshManifestRunnable, delayUntilNextLoadMs); } @@ -1165,69 +1115,118 @@ private void startLoading(ParsingLoadable loadable, loadable.type); } - private static final class PeriodSeekInfo { - - public static PeriodSeekInfo createPeriodSeekInfo( - com.google.android.exoplayer2.source.dash.manifest.Period period, long durationUs) { - int adaptationSetCount = period.adaptationSets.size(); - long availableStartTimeUs = 0; - long availableEndTimeUs = Long.MAX_VALUE; - boolean isIndexExplicit = false; - boolean seenEmptyIndex = false; - - boolean haveAudioVideoAdaptationSets = false; - for (int i = 0; i < adaptationSetCount; i++) { - int type = period.adaptationSets.get(i).type; - if (type == C.TRACK_TYPE_AUDIO || type == C.TRACK_TYPE_VIDEO) { - haveAudioVideoAdaptationSets = true; - break; - } + private static long getIntervalUntilNextManifestRefreshMs( + DashManifest manifest, long nowUnixTimeMs) { + int periodIndex = manifest.getPeriodCount() - 1; + Period period = manifest.getPeriod(periodIndex); + long periodStartUs = C.msToUs(period.startMs); + long periodDurationUs = manifest.getPeriodDurationUs(periodIndex); + long nowUnixTimeUs = C.msToUs(nowUnixTimeMs); + long availabilityStartTimeUs = C.msToUs(manifest.availabilityStartTimeMs); + long intervalUs = C.msToUs(DEFAULT_NOTIFY_MANIFEST_INTERVAL_MS); + for (int i = 0; i < period.adaptationSets.size(); i++) { + List representations = period.adaptationSets.get(i).representations; + if (representations.isEmpty()) { + continue; } - - for (int i = 0; i < adaptationSetCount; i++) { - AdaptationSet adaptationSet = period.adaptationSets.get(i); - // Exclude text adaptation sets from duration calculations, if we have at least one audio - // or video adaptation set. See: https://github.com/google/ExoPlayer/issues/4029 - if (haveAudioVideoAdaptationSets && adaptationSet.type == C.TRACK_TYPE_TEXT) { - continue; + @Nullable DashSegmentIndex index = representations.get(0).getIndex(); + if (index != null) { + long nextSegmentShiftUnixTimeUs = + availabilityStartTimeUs + + periodStartUs + + index.getNextSegmentAvailableTimeUs(periodDurationUs, nowUnixTimeUs); + long requiredIntervalUs = nextSegmentShiftUnixTimeUs - nowUnixTimeUs; + // Avoid multiple refreshes within a very small amount of time. + if (requiredIntervalUs < intervalUs - 100_000 + || (requiredIntervalUs > intervalUs && requiredIntervalUs < intervalUs + 100_000)) { + intervalUs = requiredIntervalUs; } + } + } + // Round up to compensate for a potential loss in the us to ms conversion. + return LongMath.divide(intervalUs, 1000, RoundingMode.CEILING); + } - DashSegmentIndex index = adaptationSet.representations.get(0).getIndex(); - if (index == null) { - return new PeriodSeekInfo(true, 0, durationUs); - } - isIndexExplicit |= index.isExplicit(); - int segmentCount = index.getSegmentCount(durationUs); - if (segmentCount == 0) { - seenEmptyIndex = true; - availableStartTimeUs = 0; - availableEndTimeUs = 0; - } else if (!seenEmptyIndex) { - long firstSegmentNum = index.getFirstSegmentNum(); - long adaptationSetAvailableStartTimeUs = index.getTimeUs(firstSegmentNum); - availableStartTimeUs = max(availableStartTimeUs, adaptationSetAvailableStartTimeUs); - if (segmentCount != DashSegmentIndex.INDEX_UNBOUNDED) { - long lastSegmentNum = firstSegmentNum + segmentCount - 1; - long adaptationSetAvailableEndTimeUs = index.getTimeUs(lastSegmentNum) - + index.getDurationUs(lastSegmentNum, durationUs); - availableEndTimeUs = min(availableEndTimeUs, adaptationSetAvailableEndTimeUs); - } - } + private static long getAvailableStartTimeUs( + Period period, long periodDurationUs, long nowUnixTimeUs) { + long availableStartTimeUs = 0; + boolean haveAudioVideoAdaptationSets = hasVideoOrAudioAdaptationSets(period); + for (int i = 0; i < period.adaptationSets.size(); i++) { + AdaptationSet adaptationSet = period.adaptationSets.get(i); + List representations = adaptationSet.representations; + // Exclude text adaptation sets from duration calculations, if we have at least one audio + // or video adaptation set. See: https://github.com/google/ExoPlayer/issues/4029 + if ((haveAudioVideoAdaptationSets && adaptationSet.type == C.TRACK_TYPE_TEXT) + || representations.isEmpty()) { + continue; + } + @Nullable DashSegmentIndex index = representations.get(0).getIndex(); + if (index == null) { + return 0; } - return new PeriodSeekInfo(isIndexExplicit, availableStartTimeUs, availableEndTimeUs); + int availableSegmentCount = index.getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs); + if (availableSegmentCount == 0) { + return 0; + } + long firstAvailableSegmentNum = + index.getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs); + long adaptationSetAvailableStartTimeUs = index.getTimeUs(firstAvailableSegmentNum); + availableStartTimeUs = max(availableStartTimeUs, adaptationSetAvailableStartTimeUs); } + return availableStartTimeUs; + } - public final boolean isIndexExplicit; - public final long availableStartTimeUs; - public final long availableEndTimeUs; + private static long getAvailableEndTimeUs( + Period period, long periodDurationUs, long nowUnixTimeUs) { + long availableEndTimeUs = Long.MAX_VALUE; + boolean haveAudioVideoAdaptationSets = hasVideoOrAudioAdaptationSets(period); + for (int i = 0; i < period.adaptationSets.size(); i++) { + AdaptationSet adaptationSet = period.adaptationSets.get(i); + List representations = adaptationSet.representations; + // Exclude text adaptation sets from duration calculations, if we have at least one audio + // or video adaptation set. See: https://github.com/google/ExoPlayer/issues/4029 + if ((haveAudioVideoAdaptationSets && adaptationSet.type == C.TRACK_TYPE_TEXT) + || representations.isEmpty()) { + continue; + } + @Nullable DashSegmentIndex index = representations.get(0).getIndex(); + if (index == null) { + return periodDurationUs; + } + int availableSegmentCount = index.getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs); + if (availableSegmentCount == 0) { + return 0; + } + long firstAvailableSegmentNum = + index.getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs); + long lastAvailableSegmentNum = firstAvailableSegmentNum + availableSegmentCount - 1; + long adaptationSetAvailableEndTimeUs = + index.getTimeUs(lastAvailableSegmentNum) + + index.getDurationUs(lastAvailableSegmentNum, periodDurationUs); + availableEndTimeUs = min(availableEndTimeUs, adaptationSetAvailableEndTimeUs); + } + return availableEndTimeUs; + } - private PeriodSeekInfo(boolean isIndexExplicit, long availableStartTimeUs, - long availableEndTimeUs) { - this.isIndexExplicit = isIndexExplicit; - this.availableStartTimeUs = availableStartTimeUs; - this.availableEndTimeUs = availableEndTimeUs; + private static boolean isIndexExplicit(Period period) { + for (int i = 0; i < period.adaptationSets.size(); i++) { + @Nullable + DashSegmentIndex index = period.adaptationSets.get(i).representations.get(0).getIndex(); + if (index == null || index.isExplicit()) { + return true; + } } + return false; + } + private static boolean hasVideoOrAudioAdaptationSets(Period period) { + for (int i = 0; i < period.adaptationSets.size(); i++) { + int type = period.adaptationSets.get(i).type; + if (type == C.TRACK_TYPE_AUDIO || type == C.TRACK_TYPE_VIDEO) { + return true; + } + } + return false; } private static final class DashTimeline extends Timeline { @@ -1242,6 +1241,7 @@ private static final class DashTimeline extends Timeline { private final long windowDefaultStartPositionUs; private final DashManifest manifest; private final MediaItem mediaItem; + @Nullable private final MediaItem.LiveConfiguration liveConfiguration; public DashTimeline( long presentationStartTimeMs, @@ -1252,7 +1252,9 @@ public DashTimeline( long windowDurationUs, long windowDefaultStartPositionUs, DashManifest manifest, - MediaItem mediaItem) { + MediaItem mediaItem, + @Nullable MediaItem.LiveConfiguration liveConfiguration) { + checkState(manifest.dynamic == (liveConfiguration != null)); this.presentationStartTimeMs = presentationStartTimeMs; this.windowStartTimeMs = windowStartTimeMs; this.elapsedRealtimeEpochOffsetMs = elapsedRealtimeEpochOffsetMs; @@ -1262,6 +1264,7 @@ public DashTimeline( this.windowDefaultStartPositionUs = windowDefaultStartPositionUs; this.manifest = manifest; this.mediaItem = mediaItem; + this.liveConfiguration = liveConfiguration; } @Override @@ -1298,7 +1301,7 @@ public Window getWindow(int windowIndex, Window window, long defaultPositionProj elapsedRealtimeEpochOffsetMs, /* isSeekable= */ true, /* isDynamic= */ isMovingLiveWindow(manifest), - /* isLive= */ manifest.dynamic, + liveConfiguration, windowDefaultStartPositionUs, windowDurationUs, /* firstPeriodIndex= */ 0, @@ -1347,8 +1350,9 @@ private long getAdjustedWindowDefaultStartPositionUs(long defaultPositionProject } // If there are multiple video adaptation sets with unaligned segments, the initial time may // not correspond to the start of a segment in both, but this is an edge case. - DashSegmentIndex snapIndex = period.adaptationSets.get(videoAdaptationSetIndex) - .representations.get(0).getIndex(); + @Nullable + DashSegmentIndex snapIndex = + period.adaptationSets.get(videoAdaptationSetIndex).representations.get(0).getIndex(); if (snapIndex == null || snapIndex.getSegmentCount(periodDurationUs) == 0) { // Video adaptation set does not include a non-empty index for snapping. return windowDefaultStartPositionUs; diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashSegmentIndex.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashSegmentIndex.java index 9d45bc726ee..527ed6ce82b 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashSegmentIndex.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashSegmentIndex.java @@ -64,38 +64,66 @@ public interface DashSegmentIndex { */ RangedUri getSegmentUrl(long segmentNum); + /** Returns the segment number of the first defined segment in the index. */ + long getFirstSegmentNum(); + /** - * Returns the segment number of the first segment. + * Returns the segment number of the first available segment in the index. * - * @return The segment number of the first segment. + * @param periodDurationUs The duration of the enclosing period in microseconds, or {@link + * C#TIME_UNSET} if the period's duration is not yet known. + * @param nowUnixTimeUs The current time in milliseconds since the Unix epoch. + * @return The number of the first available segment. */ - long getFirstSegmentNum(); + long getFirstAvailableSegmentNum(long periodDurationUs, long nowUnixTimeUs); /** - * Returns the number of segments in the index, or {@link #INDEX_UNBOUNDED}. - *

    - * An unbounded index occurs if a dynamic manifest uses SegmentTemplate elements without a + * Returns the number of segments defined in the index, or {@link #INDEX_UNBOUNDED}. + * + *

    An unbounded index occurs if a dynamic manifest uses SegmentTemplate elements without a * SegmentTimeline element, and if the period duration is not yet known. In this case the caller - * must manually determine the window of currently available segments. + * can query the available segment using {@link #getFirstAvailableSegmentNum(long, long)} and + * {@link #getAvailableSegmentCount(long, long)}. * - * @param periodDurationUs The duration of the enclosing period in microseconds, or - * {@link C#TIME_UNSET} if the period's duration is not yet known. + * @param periodDurationUs The duration of the enclosing period in microseconds, or {@link + * C#TIME_UNSET} if the period's duration is not yet known. * @return The number of segments in the index, or {@link #INDEX_UNBOUNDED}. */ int getSegmentCount(long periodDurationUs); + /** + * Returns the number of available segments in the index. + * + * @param periodDurationUs The duration of the enclosing period in microseconds, or {@link + * C#TIME_UNSET} if the period's duration is not yet known. + * @param nowUnixTimeUs The current time in milliseconds since the Unix epoch. + * @return The number of available segments in the index. + */ + int getAvailableSegmentCount(long periodDurationUs, long nowUnixTimeUs); + + /** + * Returns the time, in microseconds, at which a new segment becomes available, or {@link + * C#TIME_UNSET} if not applicable. + * + * @param periodDurationUs The duration of the enclosing period in microseconds, or {@link + * C#TIME_UNSET} if the period's duration is not yet known. + * @param nowUnixTimeUs The current time in milliseconds since the Unix epoch. + * @return The time, in microseconds, at which a new segment becomes available, or {@link + * C#TIME_UNSET} if not applicable. + */ + long getNextSegmentAvailableTimeUs(long periodDurationUs, long nowUnixTimeUs); + /** * Returns true if segments are defined explicitly by the index. - *

    - * If true is returned, each segment is defined explicitly by the index data, and all of the + * + *

    If true is returned, each segment is defined explicitly by the index data, and all of the * listed segments are guaranteed to be available at the time when the index was obtained. - *

    - * If false is returned then segment information was derived from properties such as a fixed + * + *

    If false is returned then segment information was derived from properties such as a fixed * segment duration. If the presentation is dynamic, it's possible that only a subset of the * segments are available. * * @return Whether segments are defined explicitly by the index. */ boolean isExplicit(); - } diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashUtil.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashUtil.java index 5dc6662d4f5..1aee832a37f 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashUtil.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashUtil.java @@ -50,14 +50,18 @@ public final class DashUtil { * * @param representation The {@link Representation} to which the request belongs. * @param requestUri The {@link RangedUri} of the data to request. + * @param flags Flags to be set on the returned {@link DataSpec}. See {@link + * DataSpec.Builder#setFlags(int)}. * @return The {@link DataSpec}. */ - public static DataSpec buildDataSpec(Representation representation, RangedUri requestUri) { + public static DataSpec buildDataSpec( + Representation representation, RangedUri requestUri, int flags) { return new DataSpec.Builder() .setUri(requestUri.resolveUri(representation.baseUrl)) .setPosition(requestUri.start) .setLength(requestUri.length) .setKey(representation.getCacheKey()) + .setFlags(flags) .build(); } @@ -194,7 +198,7 @@ private static void loadInitializationData( ChunkExtractor chunkExtractor, RangedUri requestUri) throws IOException { - DataSpec dataSpec = DashUtil.buildDataSpec(representation, requestUri); + DataSpec dataSpec = DashUtil.buildDataSpec(representation, requestUri, /* flags= */ 0); InitializationChunk initializationChunk = new InitializationChunk( dataSource, diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashWrappingSegmentIndex.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashWrappingSegmentIndex.java index 3eca7892c45..4c771cdcbf8 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashWrappingSegmentIndex.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DashWrappingSegmentIndex.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.source.dash; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.extractor.ChunkIndex; import com.google.android.exoplayer2.source.dash.manifest.RangedUri; @@ -41,11 +42,26 @@ public long getFirstSegmentNum() { return 0; } + @Override + public long getFirstAvailableSegmentNum(long periodDurationUs, long nowUnixTimeUs) { + return 0; + } + @Override public int getSegmentCount(long periodDurationUs) { return chunkIndex.length; } + @Override + public int getAvailableSegmentCount(long periodDurationUs, long nowUnixTimeUs) { + return chunkIndex.length; + } + + @Override + public long getNextSegmentAvailableTimeUs(long periodDurationUs, long nowUnixTimeUs) { + return C.TIME_UNSET; + } + @Override public long getTimeUs(long segmentNum) { return chunkIndex.timesUs[(int) segmentNum] - timeOffsetUs; diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DefaultDashChunkSource.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DefaultDashChunkSource.java index 01e51c3f6ce..22255899506 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DefaultDashChunkSource.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/DefaultDashChunkSource.java @@ -47,7 +47,7 @@ import com.google.android.exoplayer2.source.dash.manifest.DashManifest; import com.google.android.exoplayer2.source.dash.manifest.RangedUri; import com.google.android.exoplayer2.source.dash.manifest.Representation; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DataSpec; import com.google.android.exoplayer2.upstream.HttpDataSource.InvalidResponseCodeException; @@ -84,7 +84,7 @@ public DashChunkSource createDashChunkSource( DashManifest manifest, int periodIndex, int[] adaptationSetIndices, - TrackSelection trackSelection, + ExoTrackSelection trackSelection, int trackType, long elapsedRealtimeOffsetMs, boolean enableEventMessageTrack, @@ -122,12 +122,11 @@ public DashChunkSource createDashChunkSource( protected final RepresentationHolder[] representationHolders; - private TrackSelection trackSelection; + private ExoTrackSelection trackSelection; private DashManifest manifest; private int periodIndex; - private IOException fatalError; + @Nullable private IOException fatalError; private boolean missingLastSegment; - private long liveEdgeTimeUs; /** * @param manifestLoaderErrorThrower Throws errors affecting loading of manifests. @@ -153,7 +152,7 @@ public DefaultDashChunkSource( DashManifest manifest, int periodIndex, int[] adaptationSetIndices, - TrackSelection trackSelection, + ExoTrackSelection trackSelection, int trackType, DataSource dataSource, long elapsedRealtimeOffsetMs, @@ -173,7 +172,6 @@ public DefaultDashChunkSource( this.playerTrackEmsgHandler = playerTrackEmsgHandler; long periodDurationUs = manifest.getPeriodDurationUs(periodIndex); - liveEdgeTimeUs = C.TIME_UNSET; List representations = getRepresentations(); representationHolders = new RepresentationHolder[trackSelection.length()]; @@ -197,8 +195,12 @@ public long getAdjustedSeekPositionUs(long positionUs, SeekParameters seekParame if (representationHolder.segmentIndex != null) { long segmentNum = representationHolder.getSegmentNum(positionUs); long firstSyncUs = representationHolder.getSegmentStartTimeUs(segmentNum); + int segmentCount = representationHolder.getSegmentCount(); long secondSyncUs = - firstSyncUs < positionUs && segmentNum < representationHolder.getSegmentCount() - 1 + firstSyncUs < positionUs + && (segmentCount == DashSegmentIndex.INDEX_UNBOUNDED + || segmentNum + < representationHolder.getFirstSegmentNum() + segmentCount - 1) ? representationHolder.getSegmentStartTimeUs(segmentNum + 1) : firstSyncUs; return seekParameters.resolveSeekPositionUs(positionUs, firstSyncUs, secondSyncUs); @@ -226,7 +228,7 @@ public void updateManifest(DashManifest newManifest, int newPeriodIndex) { } @Override - public void updateTrackSelection(TrackSelection trackSelection) { + public void updateTrackSelection(ExoTrackSelection trackSelection) { this.trackSelection = trackSelection; } @@ -267,7 +269,6 @@ public void getNextChunk( } long bufferedDurationUs = loadPositionUs - playbackPositionUs; - long timeToLiveEdgeUs = resolveTimeToLiveEdgeUs(playbackPositionUs); long presentationPositionUs = C.msToUs(manifest.availabilityStartTimeMs) + C.msToUs(manifest.getPeriod(periodIndex).startMs) @@ -280,6 +281,7 @@ public void getNextChunk( } long nowUnixTimeUs = C.msToUs(Util.getNowUnixTimeMs(elapsedRealtimeOffsetMs)); + long nowPeriodTimeUs = getNowPeriodTimeUs(nowUnixTimeUs); MediaChunk previous = queue.isEmpty() ? null : queue.get(queue.size() - 1); MediaChunkIterator[] chunkIterators = new MediaChunkIterator[trackSelection.length()]; for (int i = 0; i < chunkIterators.length; i++) { @@ -288,9 +290,9 @@ public void getNextChunk( chunkIterators[i] = MediaChunkIterator.EMPTY; } else { long firstAvailableSegmentNum = - representationHolder.getFirstAvailableSegmentNum(manifest, periodIndex, nowUnixTimeUs); + representationHolder.getFirstAvailableSegmentNum(nowUnixTimeUs); long lastAvailableSegmentNum = - representationHolder.getLastAvailableSegmentNum(manifest, periodIndex, nowUnixTimeUs); + representationHolder.getLastAvailableSegmentNum(nowUnixTimeUs); long segmentNum = getSegmentNum( representationHolder, @@ -303,13 +305,14 @@ public void getNextChunk( } else { chunkIterators[i] = new RepresentationSegmentIterator( - representationHolder, segmentNum, lastAvailableSegmentNum); + representationHolder, segmentNum, lastAvailableSegmentNum, nowPeriodTimeUs); } } } + long availableLiveDurationUs = getAvailableLiveDurationUs(nowUnixTimeUs, playbackPositionUs); trackSelection.updateSelectedTrack( - playbackPositionUs, bufferedDurationUs, timeToLiveEdgeUs, queue, chunkIterators); + playbackPositionUs, bufferedDurationUs, availableLiveDurationUs, queue, chunkIterators); RepresentationHolder representationHolder = representationHolders[trackSelection.getSelectedIndex()]; @@ -342,13 +345,8 @@ public void getNextChunk( return; } - long firstAvailableSegmentNum = - representationHolder.getFirstAvailableSegmentNum(manifest, periodIndex, nowUnixTimeUs); - long lastAvailableSegmentNum = - representationHolder.getLastAvailableSegmentNum(manifest, periodIndex, nowUnixTimeUs); - - updateLiveEdgeTimeUs(representationHolder, lastAvailableSegmentNum); - + long firstAvailableSegmentNum = representationHolder.getFirstAvailableSegmentNum(nowUnixTimeUs); + long lastAvailableSegmentNum = representationHolder.getLastAvailableSegmentNum(nowUnixTimeUs); long segmentNum = getSegmentNum( representationHolder, @@ -397,7 +395,8 @@ public void getNextChunk( trackSelection.getSelectionData(), segmentNum, maxSegmentCount, - seekTimeUs); + seekTimeUs, + nowPeriodTimeUs); } @Override @@ -430,8 +429,7 @@ public boolean onChunkLoadError( if (!cancelable) { return false; } - if (playerTrackEmsgHandler != null - && playerTrackEmsgHandler.maybeRefreshManifestOnLoadingError(chunk)) { + if (playerTrackEmsgHandler != null && playerTrackEmsgHandler.onChunkLoadError(chunk)) { return true; } // Workaround for missing segment at the end of the period @@ -488,15 +486,22 @@ private ArrayList getRepresentations() { return representations; } - private void updateLiveEdgeTimeUs( - RepresentationHolder representationHolder, long lastAvailableSegmentNum) { - liveEdgeTimeUs = manifest.dynamic - ? representationHolder.getSegmentEndTimeUs(lastAvailableSegmentNum) : C.TIME_UNSET; + private long getAvailableLiveDurationUs(long nowUnixTimeUs, long playbackPositionUs) { + if (!manifest.dynamic) { + return C.TIME_UNSET; + } + long lastSegmentNum = representationHolders[0].getLastAvailableSegmentNum(nowUnixTimeUs); + long lastSegmentEndTimeUs = representationHolders[0].getSegmentEndTimeUs(lastSegmentNum); + long nowPeriodTimeUs = getNowPeriodTimeUs(nowUnixTimeUs); + long availabilityEndTimeUs = min(nowPeriodTimeUs, lastSegmentEndTimeUs); + return max(0, availabilityEndTimeUs - playbackPositionUs); } - private long resolveTimeToLiveEdgeUs(long playbackPositionUs) { - boolean resolveTimeToLiveEdgePossible = manifest.dynamic && liveEdgeTimeUs != C.TIME_UNSET; - return resolveTimeToLiveEdgePossible ? liveEdgeTimeUs - playbackPositionUs : C.TIME_UNSET; + private long getNowPeriodTimeUs(long nowUnixTimeUs) { + return manifest.availabilityStartTimeMs == C.TIME_UNSET + ? C.TIME_UNSET + : nowUnixTimeUs + - C.msToUs(manifest.availabilityStartTimeMs + manifest.getPeriod(periodIndex).startMs); } protected Chunk newInitializationChunk( @@ -519,7 +524,7 @@ protected Chunk newInitializationChunk( } else { requestUri = indexUri; } - DataSpec dataSpec = DashUtil.buildDataSpec(representation, requestUri); + DataSpec dataSpec = DashUtil.buildDataSpec(representation, requestUri, /* flags= */ 0); return new InitializationChunk( dataSource, dataSpec, @@ -538,14 +543,20 @@ protected Chunk newMediaChunk( Object trackSelectionData, long firstSegmentNum, int maxSegmentCount, - long seekTimeUs) { + long seekTimeUs, + long nowPeriodTimeUs) { Representation representation = representationHolder.representation; long startTimeUs = representationHolder.getSegmentStartTimeUs(firstSegmentNum); RangedUri segmentUri = representationHolder.getSegmentUrl(firstSegmentNum); String baseUrl = representation.baseUrl; if (representationHolder.chunkExtractor == null) { long endTimeUs = representationHolder.getSegmentEndTimeUs(firstSegmentNum); - DataSpec dataSpec = DashUtil.buildDataSpec(representation, segmentUri); + int flags = + representationHolder.isSegmentAvailableAtFullNetworkSpeed( + firstSegmentNum, nowPeriodTimeUs) + ? 0 + : DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED; + DataSpec dataSpec = DashUtil.buildDataSpec(representation, segmentUri, flags); return new SingleSampleMediaChunk(dataSource, dataSpec, trackFormat, trackSelectionReason, trackSelectionData, startTimeUs, endTimeUs, firstSegmentNum, trackType, trackFormat); } else { @@ -560,13 +571,18 @@ protected Chunk newMediaChunk( segmentUri = mergedSegmentUri; segmentCount++; } - long endTimeUs = representationHolder.getSegmentEndTimeUs(firstSegmentNum + segmentCount - 1); + long segmentNum = firstSegmentNum + segmentCount - 1; + long endTimeUs = representationHolder.getSegmentEndTimeUs(segmentNum); long periodDurationUs = representationHolder.periodDurationUs; long clippedEndTimeUs = periodDurationUs != C.TIME_UNSET && periodDurationUs <= endTimeUs ? periodDurationUs : C.TIME_UNSET; - DataSpec dataSpec = DashUtil.buildDataSpec(representation, segmentUri); + int flags = + representationHolder.isSegmentAvailableAtFullNetworkSpeed(segmentNum, nowPeriodTimeUs) + ? 0 + : DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED; + DataSpec dataSpec = DashUtil.buildDataSpec(representation, segmentUri, flags); long sampleOffsetUs = -representation.presentationTimeOffsetUs; return new ContainerMediaChunk( dataSource, @@ -591,6 +607,7 @@ protected Chunk newMediaChunk( protected static final class RepresentationSegmentIterator extends BaseMediaChunkIterator { private final RepresentationHolder representationHolder; + private final long nowPeriodTimeUs; /** * Creates iterator. @@ -598,20 +615,29 @@ protected static final class RepresentationSegmentIterator extends BaseMediaChun * @param representation The {@link RepresentationHolder} to wrap. * @param firstAvailableSegmentNum The number of the first available segment. * @param lastAvailableSegmentNum The number of the last available segment. + * @param nowPeriodTimeUs The current time in microseconds since the start of the period used + * for calculating if segments are available at full network speed. */ public RepresentationSegmentIterator( RepresentationHolder representation, long firstAvailableSegmentNum, - long lastAvailableSegmentNum) { + long lastAvailableSegmentNum, + long nowPeriodTimeUs) { super(/* fromIndex= */ firstAvailableSegmentNum, /* toIndex= */ lastAvailableSegmentNum); this.representationHolder = representation; + this.nowPeriodTimeUs = nowPeriodTimeUs; } @Override public DataSpec getDataSpec() { checkInBounds(); - RangedUri segmentUri = representationHolder.getSegmentUrl(getCurrentIndex()); - return DashUtil.buildDataSpec(representationHolder.representation, segmentUri); + long currentIndex = getCurrentIndex(); + RangedUri segmentUri = representationHolder.getSegmentUrl(currentIndex); + int flags = + representationHolder.isSegmentAvailableAtFullNetworkSpeed(currentIndex, nowPeriodTimeUs) + ? 0 + : DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED; + return DashUtil.buildDataSpec(representationHolder.representation, segmentUri, flags); } @Override @@ -739,6 +765,11 @@ public long getFirstSegmentNum() { return segmentIndex.getFirstSegmentNum() + segmentNumShift; } + public long getFirstAvailableSegmentNum(long nowUnixTimeUs) { + return segmentIndex.getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs) + + segmentNumShift; + } + public int getSegmentCount() { return segmentIndex.getSegmentCount(periodDurationUs); } @@ -760,35 +791,14 @@ public RangedUri getSegmentUrl(long segmentNum) { return segmentIndex.getSegmentUrl(segmentNum - segmentNumShift); } - public long getFirstAvailableSegmentNum( - DashManifest manifest, int periodIndex, long nowUnixTimeUs) { - if (getSegmentCount() == DashSegmentIndex.INDEX_UNBOUNDED - && manifest.timeShiftBufferDepthMs != C.TIME_UNSET) { - // The index is itself unbounded. We need to use the current time to calculate the range of - // available segments. - long liveEdgeTimeUs = nowUnixTimeUs - C.msToUs(manifest.availabilityStartTimeMs); - long periodStartUs = C.msToUs(manifest.getPeriod(periodIndex).startMs); - long liveEdgeTimeInPeriodUs = liveEdgeTimeUs - periodStartUs; - long bufferDepthUs = C.msToUs(manifest.timeShiftBufferDepthMs); - return max(getFirstSegmentNum(), getSegmentNum(liveEdgeTimeInPeriodUs - bufferDepthUs)); - } - return getFirstSegmentNum(); - } - - public long getLastAvailableSegmentNum( - DashManifest manifest, int periodIndex, long nowUnixTimeUs) { - int availableSegmentCount = getSegmentCount(); - if (availableSegmentCount == DashSegmentIndex.INDEX_UNBOUNDED) { - // The index is itself unbounded. We need to use the current time to calculate the range of - // available segments. - long liveEdgeTimeUs = nowUnixTimeUs - C.msToUs(manifest.availabilityStartTimeMs); - long periodStartUs = C.msToUs(manifest.getPeriod(periodIndex).startMs); - long liveEdgeTimeInPeriodUs = liveEdgeTimeUs - periodStartUs; - // getSegmentNum(liveEdgeTimeInPeriodUs) will not be completed yet, so subtract one to get - // the index of the last completed segment. - return getSegmentNum(liveEdgeTimeInPeriodUs) - 1; - } - return getFirstSegmentNum() + availableSegmentCount - 1; + public long getLastAvailableSegmentNum(long nowUnixTimeUs) { + return getFirstAvailableSegmentNum(nowUnixTimeUs) + + segmentIndex.getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs) + - 1; + } + + public boolean isSegmentAvailableAtFullNetworkSpeed(long segmentNum, long nowPeriodTimeUs) { + return nowPeriodTimeUs == C.TIME_UNSET || getSegmentEndTimeUs(segmentNum) <= nowPeriodTimeUs; } @Nullable @@ -799,7 +809,6 @@ private static ChunkExtractor createChunkExtractor( List closedCaptionFormats, @Nullable TrackOutput playerEmsgTrackOutput) { String containerMimeType = representation.format.containerMimeType; - Extractor extractor; if (MimeTypes.isText(containerMimeType)) { if (MimeTypes.APPLICATION_RAWCC.equals(containerMimeType)) { diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/PlayerEmsgHandler.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/PlayerEmsgHandler.java index 2185b52f933..65a83a48294 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/PlayerEmsgHandler.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/PlayerEmsgHandler.java @@ -24,8 +24,6 @@ import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.FormatHolder; import com.google.android.exoplayer2.ParserException; -import com.google.android.exoplayer2.drm.DrmSessionEventListener; -import com.google.android.exoplayer2.drm.DrmSessionManager; import com.google.android.exoplayer2.extractor.TrackOutput; import com.google.android.exoplayer2.metadata.Metadata; import com.google.android.exoplayer2.metadata.MetadataInputBuffer; @@ -87,8 +85,7 @@ public interface PlayerEmsgCallback { private DashManifest manifest; private long expiredManifestPublishTimeUs; - private long lastLoadedChunkEndTimeUs; - private long lastLoadedChunkEndTimeBeforeRefreshUs; + private boolean chunkLoadedCompletedSinceLastManifestRefreshRequest; private boolean isWaitingForManifestRefresh; private boolean released; @@ -107,8 +104,6 @@ public PlayerEmsgHandler( manifestPublishTimeToExpiryTimeUs = new TreeMap<>(); handler = Util.createHandlerForCurrentLooper(/* callback= */ this); decoder = new EventMessageDecoder(); - lastLoadedChunkEndTimeUs = C.TIME_UNSET; - lastLoadedChunkEndTimeBeforeRefreshUs = C.TIME_UNSET; } /** @@ -123,6 +118,36 @@ public void updateManifest(DashManifest newManifest) { removePreviouslyExpiredManifestPublishTimeValues(); } + /** Returns a {@link TrackOutput} that emsg messages could be written to. */ + public PlayerTrackEmsgHandler newPlayerTrackEmsgHandler() { + return new PlayerTrackEmsgHandler(allocator); + } + + /** Release this emsg handler. It should not be reused after this call. */ + public void release() { + released = true; + handler.removeCallbacksAndMessages(null); + } + + @Override + public boolean handleMessage(Message message) { + if (released) { + return true; + } + switch (message.what) { + case (EMSG_MANIFEST_EXPIRED): + ManifestExpiryEventInfo messageObj = (ManifestExpiryEventInfo) message.obj; + handleManifestExpiredMessage( + messageObj.eventTimeUs, messageObj.manifestPublishTimeMsInEmsg); + return true; + default: + // Do nothing. + } + return false; + } + + // Internal methods. + /* package */ boolean maybeRefreshManifestBeforeLoadingNextChunk(long presentationPositionUs) { if (!manifest.dynamic) { return false; @@ -148,83 +173,27 @@ public void updateManifest(DashManifest newManifest) { return manifestRefreshNeeded; } - /** - * For live streaming with emsg event stream, forward seeking can seek pass the emsg messages that - * signals end-of-stream or Manifest expiry, which results in load error. In this case, we should - * notify the Dash media source to refresh its manifest. - * - * @param chunk The chunk whose load encountered the error. - * @return True if manifest refresh has been requested, false otherwise. - */ - /* package */ boolean maybeRefreshManifestOnLoadingError(Chunk chunk) { + /* package */ void onChunkLoadCompleted(Chunk chunk) { + chunkLoadedCompletedSinceLastManifestRefreshRequest = true; + } + + /* package */ boolean onChunkLoadError(boolean isForwardSeek) { if (!manifest.dynamic) { return false; } if (isWaitingForManifestRefresh) { return true; } - boolean isAfterForwardSeek = - lastLoadedChunkEndTimeUs != C.TIME_UNSET && lastLoadedChunkEndTimeUs < chunk.startTimeUs; - if (isAfterForwardSeek) { - // if we are after a forward seek, and the playback is dynamic with embedded emsg stream, - // there's a chance that we have seek over the emsg messages, in which case we should ask - // media source for a refresh. + if (isForwardSeek) { + // If a forward seek has occurred, there's a chance that the seek has skipped EMSGs signalling + // end-of-stream or manifest expiration. We must assume that the manifest might need to be + // refreshed. maybeNotifyDashManifestRefreshNeeded(); return true; } return false; } - /** - * Called when the a new chunk in the current media stream has been loaded. - * - * @param chunk The chunk whose load has been completed. - */ - /* package */ void onChunkLoadCompleted(Chunk chunk) { - if (lastLoadedChunkEndTimeUs != C.TIME_UNSET || chunk.endTimeUs > lastLoadedChunkEndTimeUs) { - lastLoadedChunkEndTimeUs = chunk.endTimeUs; - } - } - - /** - * Returns whether an event with given schemeIdUri and value is a DASH emsg event targeting the - * player. - */ - public static boolean isPlayerEmsgEvent(String schemeIdUri, String value) { - return "urn:mpeg:dash:event:2012".equals(schemeIdUri) - && ("1".equals(value) || "2".equals(value) || "3".equals(value)); - } - - /** Returns a {@link TrackOutput} that emsg messages could be written to. */ - public PlayerTrackEmsgHandler newPlayerTrackEmsgHandler() { - return new PlayerTrackEmsgHandler(allocator); - } - - /** Release this emsg handler. It should not be reused after this call. */ - public void release() { - released = true; - handler.removeCallbacksAndMessages(null); - } - - @Override - public boolean handleMessage(Message message) { - if (released) { - return true; - } - switch (message.what) { - case (EMSG_MANIFEST_EXPIRED): - ManifestExpiryEventInfo messageObj = (ManifestExpiryEventInfo) message.obj; - handleManifestExpiredMessage( - messageObj.eventTimeUs, messageObj.manifestPublishTimeMsInEmsg); - return true; - default: - // Do nothing. - } - return false; - } - - // Internal methods. - private void handleManifestExpiredMessage(long eventTimeUs, long manifestPublishTimeMsInEmsg) { Long previousExpiryTimeUs = manifestPublishTimeToExpiryTimeUs.get(manifestPublishTimeMsInEmsg); if (previousExpiryTimeUs == null) { @@ -258,13 +227,12 @@ private void notifyManifestPublishTimeExpired() { /** Requests DASH media manifest to be refreshed if necessary. */ private void maybeNotifyDashManifestRefreshNeeded() { - if (lastLoadedChunkEndTimeBeforeRefreshUs != C.TIME_UNSET - && lastLoadedChunkEndTimeBeforeRefreshUs == lastLoadedChunkEndTimeUs) { - // Already requested manifest refresh. + if (!chunkLoadedCompletedSinceLastManifestRefreshRequest) { + // Don't request a refresh unless some progress has been made. return; } isWaitingForManifestRefresh = true; - lastLoadedChunkEndTimeBeforeRefreshUs = lastLoadedChunkEndTimeUs; + chunkLoadedCompletedSinceLastManifestRefreshRequest = false; playerEmsgCallback.onDashManifestRefreshRequested(); } @@ -277,6 +245,15 @@ private static long getManifestPublishTimeMsInEmsg(EventMessage eventMessage) { } } + /** + * Returns whether an event with given schemeIdUri and value is a DASH emsg event targeting the + * player. + */ + private static boolean isPlayerEmsgEvent(String schemeIdUri, String value) { + return "urn:mpeg:dash:event:2012".equals(schemeIdUri) + && ("1".equals(value) || "2".equals(value) || "3".equals(value)); + } + /** Handles emsg messages for a specific track for the player. */ public final class PlayerTrackEmsgHandler implements TrackOutput { @@ -284,15 +261,13 @@ public final class PlayerTrackEmsgHandler implements TrackOutput { private final FormatHolder formatHolder; private final MetadataInputBuffer buffer; + private long maxLoadedChunkEndTimeUs; + /* package */ PlayerTrackEmsgHandler(Allocator allocator) { - this.sampleQueue = - new SampleQueue( - allocator, - /* playbackLooper= */ handler.getLooper(), - DrmSessionManager.getDummyDrmSessionManager(), - new DrmSessionEventListener.EventDispatcher()); + this.sampleQueue = SampleQueue.createWithoutDrm(allocator); formatHolder = new FormatHolder(); buffer = new MetadataInputBuffer(); + maxLoadedChunkEndTimeUs = C.TIME_UNSET; } @Override @@ -332,24 +307,27 @@ public boolean maybeRefreshManifestBeforeLoadingNextChunk(long presentationPosit } /** - * Called when the a new chunk in the current media stream has been loaded. + * Called when a chunk load has been completed. * * @param chunk The chunk whose load has been completed. */ public void onChunkLoadCompleted(Chunk chunk) { + if (maxLoadedChunkEndTimeUs == C.TIME_UNSET || chunk.endTimeUs > maxLoadedChunkEndTimeUs) { + maxLoadedChunkEndTimeUs = chunk.endTimeUs; + } PlayerEmsgHandler.this.onChunkLoadCompleted(chunk); } /** - * For live streaming with emsg event stream, forward seeking can seek pass the emsg messages - * that signals end-of-stream or Manifest expiry, which results in load error. In this case, we - * should notify the Dash media source to refresh its manifest. + * Called when a chunk load has encountered an error. * - * @param chunk The chunk whose load encountered the error. - * @return True if manifest refresh has been requested, false otherwise. + * @param chunk The chunk whose load encountered an error. + * @return Whether a manifest refresh has been requested. */ - public boolean maybeRefreshManifestOnLoadingError(Chunk chunk) { - return PlayerEmsgHandler.this.maybeRefreshManifestOnLoadingError(chunk); + public boolean onChunkLoadError(Chunk chunk) { + boolean isAfterForwardSeek = + maxLoadedChunkEndTimeUs != C.TIME_UNSET && maxLoadedChunkEndTimeUs < chunk.startTimeUs; + return PlayerEmsgHandler.this.onChunkLoadError(isAfterForwardSeek); } /** Release this track emsg handler. It should not be reused after this call. */ diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifest.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifest.java index c21af45d152..36135e04546 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifest.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifest.java @@ -82,6 +82,9 @@ public class DashManifest implements FilterableManifest { */ @Nullable public final UtcTimingElement utcTiming; + /** The {@link ServiceDescriptionElement}, or null if not present. */ + @Nullable public final ServiceDescriptionElement serviceDescription; + /** The location of this manifest, or null if not present. */ @Nullable public final Uri location; @@ -92,7 +95,7 @@ public class DashManifest implements FilterableManifest { /** * @deprecated Use {@link #DashManifest(long, long, long, boolean, long, long, long, long, - * ProgramInformation, UtcTimingElement, Uri, List)}. + * ProgramInformation, UtcTimingElement, ServiceDescriptionElement, Uri, List)}. */ @Deprecated public DashManifest( @@ -118,6 +121,7 @@ public DashManifest( publishTimeMs, /* programInformation= */ null, utcTiming, + /* serviceDescription= */ null, location, periods); } @@ -133,6 +137,7 @@ public DashManifest( long publishTimeMs, @Nullable ProgramInformation programInformation, @Nullable UtcTimingElement utcTiming, + @Nullable ServiceDescriptionElement serviceDescription, @Nullable Uri location, List periods) { this.availabilityStartTimeMs = availabilityStartTimeMs; @@ -146,6 +151,7 @@ public DashManifest( this.programInformation = programInformation; this.utcTiming = utcTiming; this.location = location; + this.serviceDescription = serviceDescription; this.periods = periods == null ? Collections.emptyList() : periods; } @@ -203,6 +209,7 @@ public final DashManifest copy(List streamKeys) { publishTimeMs, programInformation, utcTiming, + serviceDescription, location, copyPeriods); } diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParser.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParser.java index 83e6556fe11..9b5efd49535 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParser.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParser.java @@ -40,11 +40,11 @@ import com.google.android.exoplayer2.util.Util; import com.google.android.exoplayer2.util.XmlPullParserUtil; import com.google.common.base.Charsets; +import com.google.common.collect.ImmutableList; import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.InputStream; import java.util.ArrayList; -import java.util.Collections; import java.util.List; import java.util.UUID; import java.util.regex.Matcher; @@ -125,6 +125,8 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, ProgramInformation programInformation = null; UtcTimingElement utcTiming = null; Uri location = null; + ServiceDescriptionElement serviceDescription = null; + long baseUrlAvailabilityTimeOffsetUs = dynamic ? 0 : C.TIME_UNSET; List periods = new ArrayList<>(); long nextPeriodStartMs = dynamic ? C.TIME_UNSET : 0; @@ -134,6 +136,8 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, xpp.next(); if (XmlPullParserUtil.isStartTag(xpp, "BaseURL")) { if (!seenFirstBaseUrl) { + baseUrlAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, baseUrlAvailabilityTimeOffsetUs); baseUrl = parseBaseUrl(xpp, baseUrl); seenFirstBaseUrl = true; } @@ -143,8 +147,17 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, utcTiming = parseUtcTiming(xpp); } else if (XmlPullParserUtil.isStartTag(xpp, "Location")) { location = Uri.parse(xpp.nextText()); + } else if (XmlPullParserUtil.isStartTag(xpp, "ServiceDescription")) { + serviceDescription = parseServiceDescription(xpp); } else if (XmlPullParserUtil.isStartTag(xpp, "Period") && !seenEarlyAccessPeriod) { - Pair periodWithDurationMs = parsePeriod(xpp, baseUrl, nextPeriodStartMs); + Pair periodWithDurationMs = + parsePeriod( + xpp, + baseUrl, + nextPeriodStartMs, + baseUrlAvailabilityTimeOffsetUs, + availabilityStartTime, + timeShiftBufferDepthMs); Period period = periodWithDurationMs.first; if (period.startMs == C.TIME_UNSET) { if (dynamic) { @@ -189,6 +202,7 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, publishTimeMs, programInformation, utcTiming, + serviceDescription, location, periods); } @@ -204,6 +218,7 @@ protected DashManifest buildMediaPresentationDescription( long publishTimeMs, @Nullable ProgramInformation programInformation, @Nullable UtcTimingElement utcTiming, + @Nullable ServiceDescriptionElement serviceDescription, @Nullable Uri location, List periods) { return new DashManifest( @@ -217,6 +232,7 @@ protected DashManifest buildMediaPresentationDescription( publishTimeMs, programInformation, utcTiming, + serviceDescription, location, periods); } @@ -231,33 +247,96 @@ protected UtcTimingElement buildUtcTimingElement(String schemeIdUri, String valu return new UtcTimingElement(schemeIdUri, value); } - protected Pair parsePeriod(XmlPullParser xpp, String baseUrl, long defaultStartMs) + protected ServiceDescriptionElement parseServiceDescription(XmlPullParser xpp) + throws XmlPullParserException, IOException { + long targetOffsetMs = C.TIME_UNSET; + long minOffsetMs = C.TIME_UNSET; + long maxOffsetMs = C.TIME_UNSET; + float minPlaybackSpeed = C.RATE_UNSET; + float maxPlaybackSpeed = C.RATE_UNSET; + do { + xpp.next(); + if (XmlPullParserUtil.isStartTag(xpp, "Latency")) { + targetOffsetMs = parseLong(xpp, "target", C.TIME_UNSET); + minOffsetMs = parseLong(xpp, "min", C.TIME_UNSET); + maxOffsetMs = parseLong(xpp, "max", C.TIME_UNSET); + } else if (XmlPullParserUtil.isStartTag(xpp, "PlaybackRate")) { + minPlaybackSpeed = parseFloat(xpp, "min", C.RATE_UNSET); + maxPlaybackSpeed = parseFloat(xpp, "max", C.RATE_UNSET); + } + } while (!XmlPullParserUtil.isEndTag(xpp, "ServiceDescription")); + return new ServiceDescriptionElement( + targetOffsetMs, minOffsetMs, maxOffsetMs, minPlaybackSpeed, maxPlaybackSpeed); + } + + protected Pair parsePeriod( + XmlPullParser xpp, + String baseUrl, + long defaultStartMs, + long baseUrlAvailabilityTimeOffsetUs, + long availabilityStartTimeMs, + long timeShiftBufferDepthMs) throws XmlPullParserException, IOException { @Nullable String id = xpp.getAttributeValue(null, "id"); long startMs = parseDuration(xpp, "start", defaultStartMs); + long periodStartUnixTimeMs = + availabilityStartTimeMs != C.TIME_UNSET ? availabilityStartTimeMs + startMs : C.TIME_UNSET; long durationMs = parseDuration(xpp, "duration", C.TIME_UNSET); @Nullable SegmentBase segmentBase = null; @Nullable Descriptor assetIdentifier = null; List adaptationSets = new ArrayList<>(); List eventStreams = new ArrayList<>(); boolean seenFirstBaseUrl = false; + long segmentBaseAvailabilityTimeOffsetUs = C.TIME_UNSET; do { xpp.next(); if (XmlPullParserUtil.isStartTag(xpp, "BaseURL")) { if (!seenFirstBaseUrl) { + baseUrlAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, baseUrlAvailabilityTimeOffsetUs); baseUrl = parseBaseUrl(xpp, baseUrl); seenFirstBaseUrl = true; } } else if (XmlPullParserUtil.isStartTag(xpp, "AdaptationSet")) { - adaptationSets.add(parseAdaptationSet(xpp, baseUrl, segmentBase, durationMs)); + adaptationSets.add( + parseAdaptationSet( + xpp, + baseUrl, + segmentBase, + durationMs, + baseUrlAvailabilityTimeOffsetUs, + segmentBaseAvailabilityTimeOffsetUs, + periodStartUnixTimeMs, + timeShiftBufferDepthMs)); } else if (XmlPullParserUtil.isStartTag(xpp, "EventStream")) { eventStreams.add(parseEventStream(xpp)); } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentBase")) { - segmentBase = parseSegmentBase(xpp, null); + segmentBase = parseSegmentBase(xpp, /* parent= */ null); } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentList")) { - segmentBase = parseSegmentList(xpp, null, durationMs); + segmentBaseAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, /* parentAvailabilityTimeOffsetUs= */ C.TIME_UNSET); + segmentBase = + parseSegmentList( + xpp, + /* parent= */ null, + periodStartUnixTimeMs, + durationMs, + baseUrlAvailabilityTimeOffsetUs, + segmentBaseAvailabilityTimeOffsetUs, + timeShiftBufferDepthMs); } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentTemplate")) { - segmentBase = parseSegmentTemplate(xpp, null, Collections.emptyList(), durationMs); + segmentBaseAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, /* parentAvailabilityTimeOffsetUs= */ C.TIME_UNSET); + segmentBase = + parseSegmentTemplate( + xpp, + /* parent= */ null, + ImmutableList.of(), + periodStartUnixTimeMs, + durationMs, + baseUrlAvailabilityTimeOffsetUs, + segmentBaseAvailabilityTimeOffsetUs, + timeShiftBufferDepthMs); } else if (XmlPullParserUtil.isStartTag(xpp, "AssetIdentifier")) { assetIdentifier = parseDescriptor(xpp, "AssetIdentifier"); } else { @@ -281,7 +360,14 @@ protected Period buildPeriod( // AdaptationSet parsing. protected AdaptationSet parseAdaptationSet( - XmlPullParser xpp, String baseUrl, @Nullable SegmentBase segmentBase, long periodDurationMs) + XmlPullParser xpp, + String baseUrl, + @Nullable SegmentBase segmentBase, + long periodDurationMs, + long baseUrlAvailabilityTimeOffsetUs, + long segmentBaseAvailabilityTimeOffsetUs, + long periodStartUnixTimeMs, + long timeShiftBufferDepthMs) throws XmlPullParserException, IOException { int id = parseInt(xpp, "id", AdaptationSet.ID_UNSET); int contentType = parseContentType(xpp); @@ -309,6 +395,8 @@ protected AdaptationSet parseAdaptationSet( xpp.next(); if (XmlPullParserUtil.isStartTag(xpp, "BaseURL")) { if (!seenFirstBaseUrl) { + baseUrlAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, baseUrlAvailabilityTimeOffsetUs); baseUrl = parseBaseUrl(xpp, baseUrl); seenFirstBaseUrl = true; } @@ -351,7 +439,11 @@ protected AdaptationSet parseAdaptationSet( essentialProperties, supplementalProperties, segmentBase, - periodDurationMs); + periodStartUnixTimeMs, + periodDurationMs, + baseUrlAvailabilityTimeOffsetUs, + segmentBaseAvailabilityTimeOffsetUs, + timeShiftBufferDepthMs); contentType = checkContentTypeConsistency( contentType, MimeTypes.getTrackType(representationInfo.format.sampleMimeType)); @@ -359,11 +451,30 @@ protected AdaptationSet parseAdaptationSet( } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentBase")) { segmentBase = parseSegmentBase(xpp, (SingleSegmentBase) segmentBase); } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentList")) { - segmentBase = parseSegmentList(xpp, (SegmentList) segmentBase, periodDurationMs); + segmentBaseAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, segmentBaseAvailabilityTimeOffsetUs); + segmentBase = + parseSegmentList( + xpp, + (SegmentList) segmentBase, + periodStartUnixTimeMs, + periodDurationMs, + baseUrlAvailabilityTimeOffsetUs, + segmentBaseAvailabilityTimeOffsetUs, + timeShiftBufferDepthMs); } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentTemplate")) { + segmentBaseAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, segmentBaseAvailabilityTimeOffsetUs); segmentBase = parseSegmentTemplate( - xpp, (SegmentTemplate) segmentBase, supplementalProperties, periodDurationMs); + xpp, + (SegmentTemplate) segmentBase, + supplementalProperties, + periodStartUnixTimeMs, + periodDurationMs, + baseUrlAvailabilityTimeOffsetUs, + segmentBaseAvailabilityTimeOffsetUs, + timeShiftBufferDepthMs); } else if (XmlPullParserUtil.isStartTag(xpp, "InbandEventStream")) { inbandEventStreams.add(parseDescriptor(xpp, "InbandEventStream")); } else if (XmlPullParserUtil.isStartTag(xpp, "Label")) { @@ -524,7 +635,11 @@ protected RepresentationInfo parseRepresentation( List adaptationSetEssentialProperties, List adaptationSetSupplementalProperties, @Nullable SegmentBase segmentBase, - long periodDurationMs) + long periodStartUnixTimeMs, + long periodDurationMs, + long baseUrlAvailabilityTimeOffsetUs, + long segmentBaseAvailabilityTimeOffsetUs, + long timeShiftBufferDepthMs) throws XmlPullParserException, IOException { String id = xpp.getAttributeValue(null, "id"); int bandwidth = parseInt(xpp, "bandwidth", Format.NO_VALUE); @@ -548,6 +663,8 @@ protected RepresentationInfo parseRepresentation( xpp.next(); if (XmlPullParserUtil.isStartTag(xpp, "BaseURL")) { if (!seenFirstBaseUrl) { + baseUrlAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, baseUrlAvailabilityTimeOffsetUs); baseUrl = parseBaseUrl(xpp, baseUrl); seenFirstBaseUrl = true; } @@ -556,14 +673,30 @@ protected RepresentationInfo parseRepresentation( } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentBase")) { segmentBase = parseSegmentBase(xpp, (SingleSegmentBase) segmentBase); } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentList")) { - segmentBase = parseSegmentList(xpp, (SegmentList) segmentBase, periodDurationMs); + segmentBaseAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, segmentBaseAvailabilityTimeOffsetUs); + segmentBase = + parseSegmentList( + xpp, + (SegmentList) segmentBase, + periodStartUnixTimeMs, + periodDurationMs, + baseUrlAvailabilityTimeOffsetUs, + segmentBaseAvailabilityTimeOffsetUs, + timeShiftBufferDepthMs); } else if (XmlPullParserUtil.isStartTag(xpp, "SegmentTemplate")) { + segmentBaseAvailabilityTimeOffsetUs = + parseAvailabilityTimeOffsetUs(xpp, segmentBaseAvailabilityTimeOffsetUs); segmentBase = parseSegmentTemplate( xpp, (SegmentTemplate) segmentBase, adaptationSetSupplementalProperties, - periodDurationMs); + periodStartUnixTimeMs, + periodDurationMs, + baseUrlAvailabilityTimeOffsetUs, + segmentBaseAvailabilityTimeOffsetUs, + timeShiftBufferDepthMs); } else if (XmlPullParserUtil.isStartTag(xpp, "ContentProtection")) { Pair contentProtection = parseContentProtection(xpp); if (contentProtection.first != null) { @@ -728,7 +861,13 @@ protected SingleSegmentBase buildSingleSegmentBase(RangedUri initialization, lon } protected SegmentList parseSegmentList( - XmlPullParser xpp, @Nullable SegmentList parent, long periodDurationMs) + XmlPullParser xpp, + @Nullable SegmentList parent, + long periodStartUnixTimeMs, + long periodDurationMs, + long baseUrlAvailabilityTimeOffsetUs, + long segmentBaseAvailabilityTimeOffsetUs, + long timeShiftBufferDepthMs) throws XmlPullParserException, IOException { long timescale = parseLong(xpp, "timescale", parent != null ? parent.timescale : 1); @@ -736,6 +875,9 @@ protected SegmentList parseSegmentList( parent != null ? parent.presentationTimeOffset : 0); long duration = parseLong(xpp, "duration", parent != null ? parent.duration : C.TIME_UNSET); long startNumber = parseLong(xpp, "startNumber", parent != null ? parent.startNumber : 1); + long availabilityTimeOffsetUs = + getFinalAvailabilityTimeOffset( + baseUrlAvailabilityTimeOffsetUs, segmentBaseAvailabilityTimeOffsetUs); RangedUri initialization = null; List timeline = null; @@ -763,8 +905,17 @@ protected SegmentList parseSegmentList( segments = segments != null ? segments : parent.mediaSegments; } - return buildSegmentList(initialization, timescale, presentationTimeOffset, - startNumber, duration, timeline, segments); + return buildSegmentList( + initialization, + timescale, + presentationTimeOffset, + startNumber, + duration, + timeline, + availabilityTimeOffsetUs, + segments, + timeShiftBufferDepthMs, + periodStartUnixTimeMs); } protected SegmentList buildSegmentList( @@ -774,16 +925,32 @@ protected SegmentList buildSegmentList( long startNumber, long duration, @Nullable List timeline, - @Nullable List segments) { - return new SegmentList(initialization, timescale, presentationTimeOffset, - startNumber, duration, timeline, segments); + long availabilityTimeOffsetUs, + @Nullable List segments, + long timeShiftBufferDepthMs, + long periodStartUnixTimeMs) { + return new SegmentList( + initialization, + timescale, + presentationTimeOffset, + startNumber, + duration, + timeline, + availabilityTimeOffsetUs, + segments, + C.msToUs(timeShiftBufferDepthMs), + C.msToUs(periodStartUnixTimeMs)); } protected SegmentTemplate parseSegmentTemplate( XmlPullParser xpp, @Nullable SegmentTemplate parent, List adaptationSetSupplementalProperties, - long periodDurationMs) + long periodStartUnixTimeMs, + long periodDurationMs, + long baseUrlAvailabilityTimeOffsetUs, + long segmentBaseAvailabilityTimeOffsetUs, + long timeShiftBufferDepthMs) throws XmlPullParserException, IOException { long timescale = parseLong(xpp, "timescale", parent != null ? parent.timescale : 1); long presentationTimeOffset = parseLong(xpp, "presentationTimeOffset", @@ -792,6 +959,9 @@ protected SegmentTemplate parseSegmentTemplate( long startNumber = parseLong(xpp, "startNumber", parent != null ? parent.startNumber : 1); long endNumber = parseLastSegmentNumberSupplementalProperty(adaptationSetSupplementalProperties); + long availabilityTimeOffsetUs = + getFinalAvailabilityTimeOffset( + baseUrlAvailabilityTimeOffsetUs, segmentBaseAvailabilityTimeOffsetUs); UrlTemplate mediaTemplate = parseUrlTemplate(xpp, "media", parent != null ? parent.mediaTemplate : null); @@ -825,8 +995,11 @@ protected SegmentTemplate parseSegmentTemplate( endNumber, duration, timeline, + availabilityTimeOffsetUs, initializationTemplate, - mediaTemplate); + mediaTemplate, + timeShiftBufferDepthMs, + periodStartUnixTimeMs); } protected SegmentTemplate buildSegmentTemplate( @@ -837,8 +1010,11 @@ protected SegmentTemplate buildSegmentTemplate( long endNumber, long duration, List timeline, + long availabilityTimeOffsetUs, @Nullable UrlTemplate initializationTemplate, - @Nullable UrlTemplate mediaTemplate) { + @Nullable UrlTemplate mediaTemplate, + long timeShiftBufferDepthMs, + long periodStartUnixTimeMs) { return new SegmentTemplate( initialization, timescale, @@ -847,8 +1023,11 @@ protected SegmentTemplate buildSegmentTemplate( endNumber, duration, timeline, + availabilityTimeOffsetUs, initializationTemplate, - mediaTemplate); + mediaTemplate, + C.msToUs(timeShiftBufferDepthMs), + C.msToUs(periodStartUnixTimeMs)); } /** @@ -1161,6 +1340,27 @@ protected String parseBaseUrl(XmlPullParser xpp, String parentBaseUrl) return UriUtil.resolve(parentBaseUrl, parseText(xpp, "BaseURL")); } + /** + * Parses the availabilityTimeOffset value and returns the parsed value or the parent value if it + * doesn't exist. + * + * @param xpp The parser from which to read. + * @param parentAvailabilityTimeOffsetUs The availability time offset of a parent element in + * microseconds. + * @return The parsed availabilityTimeOffset in microseconds. + */ + protected long parseAvailabilityTimeOffsetUs( + XmlPullParser xpp, long parentAvailabilityTimeOffsetUs) { + String value = xpp.getAttributeValue(/* namespace= */ null, "availabilityTimeOffset"); + if (value == null) { + return parentAvailabilityTimeOffsetUs; + } + if ("INF".equals(value)) { + return Long.MAX_VALUE; + } + return (long) (Float.parseFloat(value) * C.MICROS_PER_SECOND); + } + // AudioChannelConfiguration parsing. protected int parseAudioChannelConfiguration(XmlPullParser xpp) @@ -1543,6 +1743,11 @@ protected static long parseLong(XmlPullParser xpp, String name, long defaultValu return value == null ? defaultValue : Long.parseLong(value); } + protected static float parseFloat(XmlPullParser xpp, String name, float defaultValue) { + String value = xpp.getAttributeValue(null, name); + return value == null ? defaultValue : Float.parseFloat(value); + } + protected static String parseString(XmlPullParser xpp, String name, String defaultValue) { String value = xpp.getAttributeValue(null, name); return value == null ? defaultValue : value; @@ -1604,6 +1809,20 @@ protected static long parseLastSegmentNumberSupplementalProperty( return C.INDEX_UNSET; } + private static long getFinalAvailabilityTimeOffset( + long baseUrlAvailabilityTimeOffsetUs, long segmentBaseAvailabilityTimeOffsetUs) { + long availabilityTimeOffsetUs = segmentBaseAvailabilityTimeOffsetUs; + if (availabilityTimeOffsetUs == C.TIME_UNSET) { + // Fall back to BaseURL values if no SegmentBase specifies an offset. + availabilityTimeOffsetUs = baseUrlAvailabilityTimeOffsetUs; + } + if (availabilityTimeOffsetUs == Long.MAX_VALUE) { + // Replace INF value with TIME_UNSET to specify that all segments are available immediately. + availabilityTimeOffsetUs = C.TIME_UNSET; + } + return availabilityTimeOffsetUs; + } + /** A parsed Representation element. */ protected static final class RepresentationInfo { diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/Representation.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/Representation.java index 80ad15cd8f5..c0b1dceec53 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/Representation.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/Representation.java @@ -17,6 +17,7 @@ import android.net.Uri; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.source.dash.DashSegmentIndex; @@ -90,7 +91,12 @@ public static Representation newInstance( SegmentBase segmentBase, @Nullable List inbandEventStreams) { return newInstance( - revisionId, format, baseUrl, segmentBase, inbandEventStreams, /* cacheKey= */ null); + revisionId, + format, + baseUrl, + segmentBase, + inbandEventStreams, + /* cacheKey= */ null); } /** @@ -275,9 +281,11 @@ public String getCacheKey() { public static class MultiSegmentRepresentation extends Representation implements DashSegmentIndex { - private final MultiSegmentBase segmentBase; + @VisibleForTesting /* package */ final MultiSegmentBase segmentBase; /** + * Creates the multi-segment Representation. + * * @param revisionId Identifies the revision of the content. * @param format The format of the representation. * @param baseUrl The base URL of the representation. @@ -338,11 +346,26 @@ public long getFirstSegmentNum() { return segmentBase.getFirstSegmentNum(); } + @Override + public long getFirstAvailableSegmentNum(long periodDurationUs, long nowUnixTimeUs) { + return segmentBase.getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs); + } + @Override public int getSegmentCount(long periodDurationUs) { return segmentBase.getSegmentCount(periodDurationUs); } + @Override + public int getAvailableSegmentCount(long periodDurationUs, long nowUnixTimeUs) { + return segmentBase.getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs); + } + + @Override + public long getNextSegmentAvailableTimeUs(long periodDurationUs, long nowUnixTimeUs) { + return segmentBase.getNextSegmentAvailableTimeUs(periodDurationUs, nowUnixTimeUs); + } + @Override public boolean isExplicit() { return segmentBase.isExplicit(); diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/SegmentBase.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/SegmentBase.java index d6206c1c0d3..495f288805c 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/SegmentBase.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/SegmentBase.java @@ -15,9 +15,12 @@ */ package com.google.android.exoplayer2.source.dash.manifest; +import static com.google.android.exoplayer2.source.dash.DashSegmentIndex.INDEX_UNBOUNDED; +import static java.lang.Math.max; import static java.lang.Math.min; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.source.dash.DashSegmentIndex; import com.google.android.exoplayer2.util.Util; @@ -119,6 +122,17 @@ public abstract static class MultiSegmentBase extends SegmentBase { /* package */ final long startNumber; /* package */ final long duration; @Nullable /* package */ final List segmentTimeline; + private final long timeShiftBufferDepthUs; + private final long periodStartUnixTimeUs; + + /** + * Offset to the current realtime at which segments become available, in microseconds, or {@link + * C#TIME_UNSET} if all segments are available immediately. + * + *

    Segments will be available once their end time ≤ currentRealTime + + * availabilityTimeOffset. + */ + @VisibleForTesting /* package */ final long availabilityTimeOffsetUs; /** * @param initialization A {@link RangedUri} corresponding to initialization data, if such data @@ -133,6 +147,11 @@ public abstract static class MultiSegmentBase extends SegmentBase { * @param segmentTimeline A segment timeline corresponding to the segments. If null, then * segments are assumed to be of fixed duration as specified by the {@code duration} * parameter. + * @param availabilityTimeOffsetUs The offset to the current realtime at which segments become + * available in microseconds, or {@link C#TIME_UNSET} if not applicable. + * @param timeShiftBufferDepthUs The time shift buffer depth in microseconds. + * @param periodStartUnixTimeUs The start of the enclosing period in microseconds since the Unix + * epoch. */ public MultiSegmentBase( @Nullable RangedUri initialization, @@ -140,14 +159,20 @@ public MultiSegmentBase( long presentationTimeOffset, long startNumber, long duration, - @Nullable List segmentTimeline) { + @Nullable List segmentTimeline, + long availabilityTimeOffsetUs, + long timeShiftBufferDepthUs, + long periodStartUnixTimeUs) { super(initialization, timescale, presentationTimeOffset); this.startNumber = startNumber; this.duration = duration; this.segmentTimeline = segmentTimeline; + this.availabilityTimeOffsetUs = availabilityTimeOffsetUs; + this.timeShiftBufferDepthUs = timeShiftBufferDepthUs; + this.periodStartUnixTimeUs = periodStartUnixTimeUs; } - /** @see DashSegmentIndex#getSegmentNum(long, long) */ + /** See {@link DashSegmentIndex#getSegmentNum(long, long)}. */ public long getSegmentNum(long timeUs, long periodDurationUs) { final long firstSegmentNum = getFirstSegmentNum(); final long segmentCount = getSegmentCount(periodDurationUs); @@ -161,7 +186,7 @@ public long getSegmentNum(long timeUs, long periodDurationUs) { // Ensure we stay within bounds. return segmentNum < firstSegmentNum ? firstSegmentNum - : segmentCount == DashSegmentIndex.INDEX_UNBOUNDED + : segmentCount == INDEX_UNBOUNDED ? segmentNum : min(segmentNum, firstSegmentNum + segmentCount - 1); } else { @@ -183,21 +208,21 @@ public long getSegmentNum(long timeUs, long periodDurationUs) { } } - /** @see DashSegmentIndex#getDurationUs(long, long) */ + /** See {@link DashSegmentIndex#getDurationUs(long, long)}. */ public final long getSegmentDurationUs(long sequenceNumber, long periodDurationUs) { if (segmentTimeline != null) { long duration = segmentTimeline.get((int) (sequenceNumber - startNumber)).duration; return (duration * C.MICROS_PER_SECOND) / timescale; } else { int segmentCount = getSegmentCount(periodDurationUs); - return segmentCount != DashSegmentIndex.INDEX_UNBOUNDED - && sequenceNumber == (getFirstSegmentNum() + segmentCount - 1) + return segmentCount != INDEX_UNBOUNDED + && sequenceNumber == (getFirstSegmentNum() + segmentCount - 1) ? (periodDurationUs - getSegmentTimeUs(sequenceNumber)) : ((duration * C.MICROS_PER_SECOND) / timescale); } } - /** @see DashSegmentIndex#getTimeUs(long) */ + /** See {@link DashSegmentIndex#getTimeUs(long)}. */ public final long getSegmentTimeUs(long sequenceNumber) { long unscaledSegmentTime; if (segmentTimeline != null) { @@ -214,27 +239,66 @@ public final long getSegmentTimeUs(long sequenceNumber) { * Returns a {@link RangedUri} defining the location of a segment for the given index in the * given representation. * - * @see DashSegmentIndex#getSegmentUrl(long) + *

    See {@link DashSegmentIndex#getSegmentUrl(long)}. */ public abstract RangedUri getSegmentUrl(Representation representation, long index); - /** @see DashSegmentIndex#getFirstSegmentNum() */ + /** See {@link DashSegmentIndex#getFirstSegmentNum()}. */ public long getFirstSegmentNum() { return startNumber; } - /** - * @see DashSegmentIndex#getSegmentCount(long) - */ - public abstract int getSegmentCount(long periodDurationUs); + /** See {@link DashSegmentIndex#getFirstAvailableSegmentNum(long, long)}. */ + public long getFirstAvailableSegmentNum(long periodDurationUs, long nowUnixTimeUs) { + long segmentCount = getSegmentCount(periodDurationUs); + if (segmentCount != INDEX_UNBOUNDED || timeShiftBufferDepthUs == C.TIME_UNSET) { + return getFirstSegmentNum(); + } + // The index is itself unbounded. We need to use the current time to calculate the range of + // available segments. + long liveEdgeTimeInPeriodUs = nowUnixTimeUs - periodStartUnixTimeUs; + long timeShiftBufferStartInPeriodUs = liveEdgeTimeInPeriodUs - timeShiftBufferDepthUs; + long timeShiftBufferStartSegmentNum = + getSegmentNum(timeShiftBufferStartInPeriodUs, periodDurationUs); + return max(getFirstSegmentNum(), timeShiftBufferStartSegmentNum); + } - /** - * @see DashSegmentIndex#isExplicit() - */ + /** See {@link DashSegmentIndex#getAvailableSegmentCount(long, long)}. */ + public int getAvailableSegmentCount(long periodDurationUs, long nowUnixTimeUs) { + int segmentCount = getSegmentCount(periodDurationUs); + if (segmentCount != INDEX_UNBOUNDED) { + return segmentCount; + } + // The index is itself unbounded. We need to use the current time to calculate the range of + // available segments. + long liveEdgeTimeInPeriodUs = nowUnixTimeUs - periodStartUnixTimeUs; + long availabilityTimeOffsetUs = liveEdgeTimeInPeriodUs + this.availabilityTimeOffsetUs; + // getSegmentNum(availabilityTimeOffsetUs) will not be completed yet. + long firstIncompleteSegmentNum = getSegmentNum(availabilityTimeOffsetUs, periodDurationUs); + long firstAvailableSegmentNum = getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs); + return (int) (firstIncompleteSegmentNum - firstAvailableSegmentNum); + } + + /** See {@link DashSegmentIndex#getNextSegmentAvailableTimeUs(long, long)}. */ + public long getNextSegmentAvailableTimeUs(long periodDurationUs, long nowUnixTimeUs) { + if (segmentTimeline != null) { + return C.TIME_UNSET; + } + long firstIncompleteSegmentNum = + getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs) + + getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs); + return getSegmentTimeUs(firstIncompleteSegmentNum) + + getSegmentDurationUs(firstIncompleteSegmentNum, periodDurationUs) + - availabilityTimeOffsetUs; + } + + /** See {@link DashSegmentIndex#isExplicit()} */ public boolean isExplicit() { return segmentTimeline != null; } + /** See {@link DashSegmentIndex#getSegmentCount(long)}. */ + public abstract int getSegmentCount(long periodDurationUs); } /** A {@link MultiSegmentBase} that uses a SegmentList to define its segments. */ @@ -255,7 +319,12 @@ public static final class SegmentList extends MultiSegmentBase { * @param segmentTimeline A segment timeline corresponding to the segments. If null, then * segments are assumed to be of fixed duration as specified by the {@code duration} * parameter. + * @param availabilityTimeOffsetUs The offset to the current realtime at which segments become + * available in microseconds, or {@link C#TIME_UNSET} if not applicable. * @param mediaSegments A list of {@link RangedUri}s indicating the locations of the segments. + * @param timeShiftBufferDepthUs The time shift buffer depth in microseconds. + * @param periodStartUnixTimeUs The start of the enclosing period in microseconds since the Unix + * epoch. */ public SegmentList( RangedUri initialization, @@ -264,9 +333,20 @@ public SegmentList( long startNumber, long duration, @Nullable List segmentTimeline, - @Nullable List mediaSegments) { - super(initialization, timescale, presentationTimeOffset, startNumber, duration, - segmentTimeline); + long availabilityTimeOffsetUs, + @Nullable List mediaSegments, + long timeShiftBufferDepthUs, + long periodStartUnixTimeUs) { + super( + initialization, + timescale, + presentationTimeOffset, + startNumber, + duration, + segmentTimeline, + availabilityTimeOffsetUs, + timeShiftBufferDepthUs, + periodStartUnixTimeUs); this.mediaSegments = mediaSegments; } @@ -311,10 +391,15 @@ public static final class SegmentTemplate extends MultiSegmentBase { * @param segmentTimeline A segment timeline corresponding to the segments. If null, then * segments are assumed to be of fixed duration as specified by the {@code duration} * parameter. + * @param availabilityTimeOffsetUs The offset to the current realtime at which segments become + * available in microseconds, or {@link C#TIME_UNSET} if not applicable. * @param initializationTemplate A template defining the location of initialization data, if * such data exists. If non-null then the {@code initialization} parameter is ignored. If * null then {@code initialization} will be used. * @param mediaTemplate A template defining the location of each media segment. + * @param timeShiftBufferDepthUs The time shift buffer depth in microseconds. + * @param periodStartUnixTimeUs The start of the enclosing period in microseconds since the Unix + * epoch. */ public SegmentTemplate( RangedUri initialization, @@ -324,15 +409,21 @@ public SegmentTemplate( long endNumber, long duration, @Nullable List segmentTimeline, + long availabilityTimeOffsetUs, @Nullable UrlTemplate initializationTemplate, - @Nullable UrlTemplate mediaTemplate) { + @Nullable UrlTemplate mediaTemplate, + long timeShiftBufferDepthUs, + long periodStartUnixTimeUs) { super( initialization, timescale, presentationTimeOffset, startNumber, duration, - segmentTimeline); + segmentTimeline, + availabilityTimeOffsetUs, + timeShiftBufferDepthUs, + periodStartUnixTimeUs); this.initializationTemplate = initializationTemplate; this.mediaTemplate = mediaTemplate; this.endNumber = endNumber; @@ -373,7 +464,7 @@ public int getSegmentCount(long periodDurationUs) { long durationUs = (duration * C.MICROS_PER_SECOND) / timescale; return (int) Util.ceilDivide(periodDurationUs, durationUs); } else { - return DashSegmentIndex.INDEX_UNBOUNDED; + return INDEX_UNBOUNDED; } } } diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/ServiceDescriptionElement.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/ServiceDescriptionElement.java new file mode 100644 index 00000000000..eec862f4f45 --- /dev/null +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/ServiceDescriptionElement.java @@ -0,0 +1,66 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.dash.manifest; + +import com.google.android.exoplayer2.C; + +/** Represents a service description element. */ +public final class ServiceDescriptionElement { + + /** The target live offset in milliseconds, or {@link C#TIME_UNSET} if undefined. */ + public final long targetOffsetMs; + /** The minimum live offset in milliseconds, or {@link C#TIME_UNSET} if undefined. */ + public final long minOffsetMs; + /** The maximum live offset in milliseconds, or {@link C#TIME_UNSET} if undefined. */ + public final long maxOffsetMs; + /** + * The minimum factor by which playback can be sped up for live speed adjustment, or {@link + * C#RATE_UNSET} if undefined. + */ + public final float minPlaybackSpeed; + /** + * The maximum factor by which playback can be sped up for live speed adjustment, or {@link + * C#RATE_UNSET} if undefined. + */ + public final float maxPlaybackSpeed; + + /** + * Creates a service description element. + * + * @param targetOffsetMs The target live offset in milliseconds, or {@link C#TIME_UNSET} if + * undefined. + * @param minOffsetMs The minimum live offset in milliseconds, or {@link C#TIME_UNSET} if + * undefined. + * @param maxOffsetMs The maximum live offset in milliseconds, or {@link C#TIME_UNSET} if + * undefined. + * @param minPlaybackSpeed The minimum factor by which playback can be sped up for live speed + * adjustment, or {@link C#RATE_UNSET} if undefined. + * @param maxPlaybackSpeed The maximum factor by which playback can be sped up for live speed + * adjustment, or {@link C#RATE_UNSET} if undefined. + */ + public ServiceDescriptionElement( + long targetOffsetMs, + long minOffsetMs, + long maxOffsetMs, + float minPlaybackSpeed, + float maxPlaybackSpeed) { + this.targetOffsetMs = targetOffsetMs; + this.minOffsetMs = minOffsetMs; + this.maxOffsetMs = maxOffsetMs; + this.minPlaybackSpeed = minPlaybackSpeed; + this.maxPlaybackSpeed = maxPlaybackSpeed; + } +} diff --git a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/SingleSegmentIndex.java b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/SingleSegmentIndex.java index a56a11fe50e..523bc2d0719 100644 --- a/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/SingleSegmentIndex.java +++ b/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/SingleSegmentIndex.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.source.dash.manifest; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.source.dash.DashSegmentIndex; /** @@ -56,11 +57,26 @@ public long getFirstSegmentNum() { return 0; } + @Override + public long getFirstAvailableSegmentNum(long periodDurationUs, long nowUnixTimeUs) { + return 0; + } + @Override public int getSegmentCount(long periodDurationUs) { return 1; } + @Override + public int getAvailableSegmentCount(long periodDurationUs, long nowUnixTimeUs) { + return 1; + } + + @Override + public long getNextSegmentAvailableTimeUs(long periodDurationUs, long nowUnixTimeUs) { + return C.TIME_UNSET; + } + @Override public boolean isExplicit() { return true; diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/e2etest/DashPlaybackTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/e2etest/DashPlaybackTest.java index e0ea43b1147..d14c702a252 100644 --- a/library/dash/src/test/java/com/google/android/exoplayer2/e2etest/DashPlaybackTest.java +++ b/library/dash/src/test/java/com/google/android/exoplayer2/e2etest/DashPlaybackTest.java @@ -17,6 +17,7 @@ import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import android.content.Context; import android.graphics.SurfaceTexture; import android.view.Surface; import androidx.test.core.app.ApplicationProvider; @@ -26,9 +27,10 @@ import com.google.android.exoplayer2.SimpleExoPlayer; import com.google.android.exoplayer2.robolectric.PlaybackOutput; import com.google.android.exoplayer2.robolectric.ShadowMediaCodecConfig; +import com.google.android.exoplayer2.robolectric.TestPlayerRunHelper; import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.CapturingRenderersFactory; import com.google.android.exoplayer2.testutil.DumpFileAsserts; -import com.google.android.exoplayer2.testutil.TestExoPlayer; import com.google.android.exoplayer2.trackselection.DefaultTrackSelector; import org.junit.Rule; import org.junit.Test; @@ -48,12 +50,15 @@ public final class DashPlaybackTest { // https://github.com/google/ExoPlayer/issues/7985 @Test public void webvttInMp4() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext); SimpleExoPlayer player = - new SimpleExoPlayer.Builder(ApplicationProvider.getApplicationContext()) + new SimpleExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(new AutoAdvancingFakeClock()) .build(); player.setVideoSurface(new Surface(new SurfaceTexture(/* texName= */ 1))); - PlaybackOutput playbackOutput = PlaybackOutput.register(player, mediaCodecConfig); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); // Ensure the subtitle track is selected. DefaultTrackSelector trackSelector = @@ -62,12 +67,10 @@ public void webvttInMp4() throws Exception { player.setMediaItem(MediaItem.fromUri("asset:///media/dash/webvtt-in-mp4/sample.mpd")); player.prepare(); player.play(); - TestExoPlayer.runUntilPlaybackState(player, Player.STATE_ENDED); + TestPlayerRunHelper.runUntilPlaybackState(player, Player.STATE_ENDED); player.release(); DumpFileAsserts.assertOutput( - ApplicationProvider.getApplicationContext(), - playbackOutput, - "playbackdumps/dash/webvtt-in-mp4.dump"); + applicationContext, playbackOutput, "playbackdumps/dash/webvtt-in-mp4.dump"); } } diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DashMediaPeriodTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DashMediaPeriodTest.java index a21e73b0abf..99fd1694376 100644 --- a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DashMediaPeriodTest.java +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DashMediaPeriodTest.java @@ -201,7 +201,7 @@ private static DashMediaPeriod createDashMediaPeriod(DashManifest manifest, int periodIndex, mock(DashChunkSource.Factory.class), mock(TransferListener.class), - DrmSessionManager.getDummyDrmSessionManager(), + DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher() .withParameters(/* windowIndex= */ 0, mediaPeriodId), mock(LoadErrorHandlingPolicy.class), diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DashMediaSourceTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DashMediaSourceTest.java index aa65237095c..d1269e18a1f 100644 --- a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DashMediaSourceTest.java +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DashMediaSourceTest.java @@ -16,27 +16,57 @@ package com.google.android.exoplayer2.source.dash; import static com.google.common.truth.Truth.assertThat; +import static java.util.concurrent.TimeUnit.MILLISECONDS; import static org.junit.Assert.fail; +import static org.robolectric.annotation.LooperMode.Mode.PAUSED; import android.net.Uri; import androidx.annotation.Nullable; +import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.ParserException; +import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.Timeline.Window; import com.google.android.exoplayer2.offline.StreamKey; +import com.google.android.exoplayer2.source.MediaSource; +import com.google.android.exoplayer2.source.MediaSource.MediaSourceCaller; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.android.exoplayer2.upstream.ByteArrayDataSource; +import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.FileDataSource; import com.google.android.exoplayer2.upstream.ParsingLoadable; import com.google.android.exoplayer2.util.Util; import com.google.common.collect.ImmutableList; import java.io.ByteArrayInputStream; import java.io.IOException; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.atomic.AtomicReference; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.annotation.LooperMode; +import org.robolectric.shadows.ShadowLooper; /** Unit test for {@link DashMediaSource}. */ @RunWith(AndroidJUnit4.class) +@LooperMode(PAUSED) public final class DashMediaSourceTest { + private static final String SAMPLE_MPD_LIVE_WITHOUT_LIVE_CONFIGURATION = + "media/mpd/sample_mpd_live_without_live_configuration"; + private static final String + SAMPLE_MPD_LIVE_WITH_SUGGESTED_PRESENTATION_DELAY_2S_MIN_BUFFER_TIME_500MS = + "media/mpd/sample_mpd_live_with_suggested_presentation_delay_2s_min_buffer_time_500ms"; + private static final String SAMPLE_MPD_LIVE_WITH_COMPLETE_SERVICE_DESCRIPTION = + "media/mpd/sample_mpd_live_with_complete_service_description"; + private static final String SAMPLE_MPD_LIVE_WITH_OFFSET_INSIDE_WINDOW = + "media/mpd/sample_mpd_live_with_offset_inside_window"; + private static final String SAMPLE_MPD_LIVE_WITH_OFFSET_TOO_SHORT = + "media/mpd/sample_mpd_live_with_offset_too_short"; + private static final String SAMPLE_MPD_LIVE_WITH_OFFSET_TOO_LONG = + "media/mpd/sample_mpd_live_with_offset_too_long"; + @Test public void iso8601ParserParse() throws IOException { DashMediaSource.Iso8601Parser parser = new DashMediaSource.Iso8601Parser(); @@ -157,7 +187,7 @@ public void factorySetStreamKeys_emptyMediaItemStreamKeys_setsMediaItemStreamKey // Tests backwards compatibility @SuppressWarnings("deprecation") @Test - public void factorySetStreamKeys_withMediaItemStreamKeys_doesNotsOverrideMediaItemStreamKeys() { + public void factorySetStreamKeys_withMediaItemStreamKeys_doesNotOverrideMediaItemStreamKeys() { StreamKey mediaItemStreamKey = new StreamKey(/* groupIndex= */ 0, /* trackIndex= */ 1); MediaItem mediaItem = new MediaItem.Builder() @@ -187,6 +217,306 @@ public void replaceManifestUri_doesNotChangeMediaItem() { assertThat(mediaSource.getMediaItem()).isEqualTo(mediaItem); } + @Test + public void factorySetFallbackTargetLiveOffsetMs_withMediaLiveTargetOffsetMs_usesMediaOffset() { + MediaItem mediaItem = + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(2L).build(); + DashMediaSource.Factory factory = + new DashMediaSource.Factory(new FileDataSource.Factory()) + .setFallbackTargetLiveOffsetMs(1234L); + + MediaItem dashMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); + + assertThat(dashMediaItem.liveConfiguration.targetOffsetMs).isEqualTo(2L); + } + + @Test + public void factorySetLivePresentationDelayMs_withMediaLiveTargetOffset_usesMediaOffset() { + MediaItem mediaItem = + new MediaItem.Builder().setUri(Uri.EMPTY).setLiveTargetOffsetMs(2L).build(); + DashMediaSource.Factory factory = + new DashMediaSource.Factory(new FileDataSource.Factory()) + .setLivePresentationDelayMs(1234L, /* overridesManifest= */ true); + + MediaItem dashMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); + + assertThat(dashMediaItem.liveConfiguration.targetOffsetMs).isEqualTo(2L); + } + + @Test + public void factorySetLivePresentationDelayMs_overridingManifest_mixedIntoMediaItem() { + MediaItem mediaItem = new MediaItem.Builder().setUri(Uri.EMPTY).build(); + DashMediaSource.Factory factory = + new DashMediaSource.Factory(new FileDataSource.Factory()) + .setLivePresentationDelayMs(2000L, /* overridesManifest= */ true); + + MediaItem dashMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); + + assertThat(dashMediaItem.liveConfiguration.targetOffsetMs).isEqualTo(2000L); + } + + @Test + public void factorySetLivePresentationDelayMs_notOverridingManifest_unsetInMediaItem() { + MediaItem mediaItem = new MediaItem.Builder().setUri(Uri.EMPTY).build(); + DashMediaSource.Factory factory = + new DashMediaSource.Factory(new FileDataSource.Factory()) + .setLivePresentationDelayMs(2000L, /* overridesManifest= */ false); + + MediaItem dashMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); + + assertThat(dashMediaItem.liveConfiguration.targetOffsetMs).isEqualTo(C.TIME_UNSET); + } + + @Test + public void factorySetFallbackTargetLiveOffsetMs_doesNotChangeMediaItem() { + DashMediaSource.Factory factory = + new DashMediaSource.Factory(new FileDataSource.Factory()) + .setFallbackTargetLiveOffsetMs(2000L); + + MediaItem dashMediaItem = + factory.createMediaSource(MediaItem.fromUri(Uri.EMPTY)).getMediaItem(); + + assertThat(dashMediaItem.liveConfiguration.targetOffsetMs).isEqualTo(C.TIME_UNSET); + } + + @Test + public void prepare_withoutLiveConfiguration_withoutMediaItemLiveProperties_usesDefaultFallback() + throws InterruptedException { + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> createSampleMpdDataSource(SAMPLE_MPD_LIVE_WITHOUT_LIVE_CONFIGURATION)) + .createMediaSource(MediaItem.fromUri(Uri.EMPTY)); + + MediaItem.LiveConfiguration liveConfiguration = + prepareAndWaitForTimelineRefresh(mediaSource).liveConfiguration; + + assertThat(liveConfiguration.targetOffsetMs) + .isEqualTo(DashMediaSource.DEFAULT_FALLBACK_TARGET_LIVE_OFFSET_MS); + assertThat(liveConfiguration.minOffsetMs).isEqualTo(0L); + assertThat(liveConfiguration.maxOffsetMs).isEqualTo(58_000L); + assertThat(liveConfiguration.minPlaybackSpeed).isEqualTo(C.RATE_UNSET); + assertThat(liveConfiguration.maxPlaybackSpeed).isEqualTo(C.RATE_UNSET); + } + + @Test + public void prepare_withoutLiveConfiguration_withoutMediaItemLiveProperties_usesFallback() + throws InterruptedException { + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> createSampleMpdDataSource(SAMPLE_MPD_LIVE_WITHOUT_LIVE_CONFIGURATION)) + .setFallbackTargetLiveOffsetMs(1234L) + .createMediaSource(MediaItem.fromUri(Uri.EMPTY)); + + MediaItem.LiveConfiguration liveConfiguration = + prepareAndWaitForTimelineRefresh(mediaSource).liveConfiguration; + + assertThat(liveConfiguration.targetOffsetMs).isEqualTo(1234L); + assertThat(liveConfiguration.minOffsetMs).isEqualTo(0L); + assertThat(liveConfiguration.maxOffsetMs).isEqualTo(58_000L); + assertThat(liveConfiguration.minPlaybackSpeed).isEqualTo(C.RATE_UNSET); + assertThat(liveConfiguration.maxPlaybackSpeed).isEqualTo(C.RATE_UNSET); + } + + @Test + public void prepare_withoutLiveConfiguration_withMediaItemLiveProperties_usesMediaItem() + throws InterruptedException { + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(Uri.EMPTY) + .setLiveTargetOffsetMs(876L) + .setLiveMinPlaybackSpeed(23f) + .setLiveMaxPlaybackSpeed(42f) + .setLiveMinOffsetMs(500L) + .setLiveMaxOffsetMs(20_000L) + .build(); + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> createSampleMpdDataSource(SAMPLE_MPD_LIVE_WITHOUT_LIVE_CONFIGURATION)) + .setFallbackTargetLiveOffsetMs(1234L) + .createMediaSource(mediaItem); + + MediaItem.LiveConfiguration liveConfiguration = + prepareAndWaitForTimelineRefresh(mediaSource).liveConfiguration; + + assertThat(liveConfiguration).isEqualTo(mediaItem.liveConfiguration); + } + + @Test + public void prepare_withSuggestedPresentationDelayAndMinBufferTime_usesManifestValue() + throws InterruptedException { + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> + createSampleMpdDataSource( + SAMPLE_MPD_LIVE_WITH_SUGGESTED_PRESENTATION_DELAY_2S_MIN_BUFFER_TIME_500MS)) + .setFallbackTargetLiveOffsetMs(1234L) + .createMediaSource(MediaItem.fromUri(Uri.EMPTY)); + + MediaItem.LiveConfiguration liveConfiguration = + prepareAndWaitForTimelineRefresh(mediaSource).liveConfiguration; + + assertThat(liveConfiguration.targetOffsetMs).isEqualTo(2_000L); + assertThat(liveConfiguration.minOffsetMs).isEqualTo(500L); + assertThat(liveConfiguration.maxOffsetMs).isEqualTo(58_000L); + assertThat(liveConfiguration.minPlaybackSpeed).isEqualTo(C.RATE_UNSET); + assertThat(liveConfiguration.maxPlaybackSpeed).isEqualTo(C.RATE_UNSET); + } + + @Test + public void + prepare_withSuggestedPresentationDelayAndMinBufferTime_withMediaItemLiveProperties_usesMediaItem() + throws InterruptedException { + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(Uri.EMPTY) + .setLiveTargetOffsetMs(876L) + .setLiveMinPlaybackSpeed(23f) + .setLiveMaxPlaybackSpeed(42f) + .setLiveMinOffsetMs(200L) + .setLiveMaxOffsetMs(999L) + .build(); + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> + createSampleMpdDataSource( + SAMPLE_MPD_LIVE_WITH_SUGGESTED_PRESENTATION_DELAY_2S_MIN_BUFFER_TIME_500MS)) + .setFallbackTargetLiveOffsetMs(1234L) + .createMediaSource(mediaItem); + + MediaItem.LiveConfiguration liveConfiguration = + prepareAndWaitForTimelineRefresh(mediaSource).liveConfiguration; + + assertThat(liveConfiguration.targetOffsetMs).isEqualTo(876L); + assertThat(liveConfiguration.minOffsetMs).isEqualTo(200L); + assertThat(liveConfiguration.maxOffsetMs).isEqualTo(999L); + assertThat(liveConfiguration.minPlaybackSpeed).isEqualTo(23f); + assertThat(liveConfiguration.maxPlaybackSpeed).isEqualTo(42f); + } + + @Test + public void prepare_withCompleteServiceDescription_usesManifestValue() + throws InterruptedException { + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> createSampleMpdDataSource(SAMPLE_MPD_LIVE_WITH_COMPLETE_SERVICE_DESCRIPTION)) + .setFallbackTargetLiveOffsetMs(1234L) + .createMediaSource(MediaItem.fromUri(Uri.EMPTY)); + + MediaItem.LiveConfiguration liveConfiguration = + prepareAndWaitForTimelineRefresh(mediaSource).liveConfiguration; + + assertThat(liveConfiguration.targetOffsetMs).isEqualTo(4_000L); + assertThat(liveConfiguration.minOffsetMs).isEqualTo(2_000L); + assertThat(liveConfiguration.maxOffsetMs).isEqualTo(6_000L); + assertThat(liveConfiguration.minPlaybackSpeed).isEqualTo(0.96f); + assertThat(liveConfiguration.maxPlaybackSpeed).isEqualTo(1.04f); + } + + @Test + public void prepare_withCompleteServiceDescription_withMediaItemLiveProperties_usesMediaItem() + throws InterruptedException { + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(Uri.EMPTY) + .setLiveTargetOffsetMs(876L) + .setLiveMinPlaybackSpeed(23f) + .setLiveMaxPlaybackSpeed(42f) + .setLiveMinOffsetMs(100L) + .setLiveMaxOffsetMs(999L) + .build(); + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> createSampleMpdDataSource(SAMPLE_MPD_LIVE_WITH_COMPLETE_SERVICE_DESCRIPTION)) + .setFallbackTargetLiveOffsetMs(1234L) + .createMediaSource(mediaItem); + + MediaItem.LiveConfiguration liveConfiguration = + prepareAndWaitForTimelineRefresh(mediaSource).liveConfiguration; + + assertThat(liveConfiguration.targetOffsetMs).isEqualTo(876L); + assertThat(liveConfiguration.minOffsetMs).isEqualTo(100L); + assertThat(liveConfiguration.maxOffsetMs).isEqualTo(999L); + assertThat(liveConfiguration.minPlaybackSpeed).isEqualTo(23f); + assertThat(liveConfiguration.maxPlaybackSpeed).isEqualTo(42f); + } + + @Test + public void prepare_targetLiveOffsetInWindow_manifestTargetOffsetAndAlignedWindowStartPosition() + throws InterruptedException { + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> createSampleMpdDataSource(SAMPLE_MPD_LIVE_WITH_OFFSET_INSIDE_WINDOW)) + .createMediaSource(MediaItem.fromUri(Uri.EMPTY)); + + Window window = prepareAndWaitForTimelineRefresh(mediaSource); + + // Expect the target live offset as defined in the manifest. + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(3000); + // Expect the default position at the first segment start before the live edge. + assertThat(window.getDefaultPositionMs()).isEqualTo(2_000); + } + + @Test + public void prepare_targetLiveOffsetTooLong_correctedTargetOffsetAndAlignedWindowStartPosition() + throws InterruptedException { + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> createSampleMpdDataSource(SAMPLE_MPD_LIVE_WITH_OFFSET_TOO_LONG)) + .createMediaSource(MediaItem.fromUri(Uri.EMPTY)); + + Window window = prepareAndWaitForTimelineRefresh(mediaSource); + + // Expect the default position at the first segment start below the minimum live start position. + assertThat(window.getDefaultPositionMs()).isEqualTo(4_000); + // Expect the target live offset reaching from now time to the minimum live start position. + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(9000); + } + + @Test + public void prepare_targetLiveOffsetTooShort_correctedTargetOffsetAndAlignedWindowStartPosition() + throws InterruptedException { + // Load manifest with now time far behind the start of the window. + DashMediaSource mediaSource = + new DashMediaSource.Factory( + () -> createSampleMpdDataSource(SAMPLE_MPD_LIVE_WITH_OFFSET_TOO_SHORT)) + .createMediaSource(MediaItem.fromUri(Uri.EMPTY)); + + Window window = prepareAndWaitForTimelineRefresh(mediaSource); + + // Expect the default position at the start of the last segment. + assertThat(window.getDefaultPositionMs()).isEqualTo(12_000); + // Expect the target live offset reaching from now time to the end of the window. + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(60_000 - 16_000); + } + + private static Window prepareAndWaitForTimelineRefresh(MediaSource mediaSource) + throws InterruptedException { + AtomicReference windowReference = new AtomicReference<>(); + CountDownLatch countDownLatch = new CountDownLatch(/* count= */ 1); + MediaSourceCaller caller = + (MediaSource source, Timeline timeline) -> { + if (windowReference.get() == null) { + windowReference.set(timeline.getWindow(0, new Timeline.Window())); + countDownLatch.countDown(); + } + }; + mediaSource.prepareSource(caller, /* mediaTransferListener= */ null); + while (!countDownLatch.await(/* timeout= */ 10, MILLISECONDS)) { + ShadowLooper.idleMainLooper(); + } + return windowReference.get(); + } + + private static DataSource createSampleMpdDataSource(String fileName) { + byte[] manifestData = new byte[0]; + try { + manifestData = TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), fileName); + } catch (IOException e) { + fail(e.getMessage()); + } + return new ByteArrayDataSource(manifestData); + } + private static void assertParseStringToLong( long expected, ParsingLoadable.Parser parser, String data) throws IOException { long actual = parser.parse(null, new ByteArrayInputStream(Util.getUtf8Bytes(data))); diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DefaultDashChunkSourceTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DefaultDashChunkSourceTest.java new file mode 100644 index 00000000000..93700d5ec3e --- /dev/null +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/DefaultDashChunkSourceTest.java @@ -0,0 +1,130 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.dash; + +import static com.google.common.truth.Truth.assertThat; + +import android.net.Uri; +import android.os.SystemClock; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.source.TrackGroup; +import com.google.android.exoplayer2.source.chunk.ChunkHolder; +import com.google.android.exoplayer2.source.dash.manifest.DashManifest; +import com.google.android.exoplayer2.source.dash.manifest.DashManifestParser; +import com.google.android.exoplayer2.testutil.FakeDataSource; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.android.exoplayer2.trackselection.FixedTrackSelection; +import com.google.android.exoplayer2.upstream.DataSpec; +import com.google.android.exoplayer2.upstream.LoaderErrorThrower; +import com.google.common.collect.ImmutableList; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link DefaultDashChunkSource}. */ +@RunWith(AndroidJUnit4.class) +public class DefaultDashChunkSourceTest { + + private static final String SAMPLE_MPD_LIVE_WITH_OFFSET_INSIDE_WINDOW = + "media/mpd/sample_mpd_live_with_offset_inside_window"; + private static final String SAMPLE_MPD_VOD = "media/mpd/sample_mpd_vod"; + + @Test + public void getNextChunk_forLowLatencyManifest_setsCorrectMayNotLoadAtFullNetworkSpeedFlag() + throws Exception { + long nowMs = 2_000_000_000_000L; + SystemClock.setCurrentTimeMillis(nowMs); + DashManifest manifest = + new DashManifestParser() + .parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), + SAMPLE_MPD_LIVE_WITH_OFFSET_INSIDE_WINDOW)); + DefaultDashChunkSource chunkSource = + new DefaultDashChunkSource( + new LoaderErrorThrower.Dummy(), + manifest, + /* periodIndex= */ 0, + /* adaptationSetIndices= */ new int[] {0}, + new FixedTrackSelection(new TrackGroup(new Format.Builder().build()), /* track= */ 0), + C.TRACK_TYPE_VIDEO, + new FakeDataSource(), + /* elapsedRealtimeOffsetMs= */ 0, + /* maxSegmentsPerLoad= */ 1, + /* enableEventMessageTrack= */ false, + /* closedCaptionFormats */ ImmutableList.of(), + /* playerTrackEmsgHandler= */ null); + + long nowInPeriodUs = C.msToUs(nowMs - manifest.availabilityStartTimeMs); + ChunkHolder output = new ChunkHolder(); + + chunkSource.getNextChunk( + /* playbackPositionUs= */ nowInPeriodUs - 5 * C.MICROS_PER_SECOND, + /* loadPositionUs= */ nowInPeriodUs - 5 * C.MICROS_PER_SECOND, + /* queue= */ ImmutableList.of(), + output); + assertThat(output.chunk.dataSpec.flags & DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED) + .isEqualTo(0); + + chunkSource.getNextChunk( + /* playbackPositionUs= */ nowInPeriodUs, + /* loadPositionUs= */ nowInPeriodUs, + /* queue= */ ImmutableList.of(), + output); + assertThat(output.chunk.dataSpec.flags & DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED) + .isNotEqualTo(0); + } + + @Test + public void getNextChunk_forVodManifest_doesNotSetMayNotLoadAtFullNetworkSpeedFlag() + throws Exception { + long nowMs = 2_000_000_000_000L; + SystemClock.setCurrentTimeMillis(nowMs); + DashManifest manifest = + new DashManifestParser() + .parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), SAMPLE_MPD_VOD)); + DefaultDashChunkSource chunkSource = + new DefaultDashChunkSource( + new LoaderErrorThrower.Dummy(), + manifest, + /* periodIndex= */ 0, + /* adaptationSetIndices= */ new int[] {0}, + new FixedTrackSelection(new TrackGroup(new Format.Builder().build()), /* track= */ 0), + C.TRACK_TYPE_VIDEO, + new FakeDataSource(), + /* elapsedRealtimeOffsetMs= */ 0, + /* maxSegmentsPerLoad= */ 1, + /* enableEventMessageTrack= */ false, + /* closedCaptionFormats */ ImmutableList.of(), + /* playerTrackEmsgHandler= */ null); + + ChunkHolder output = new ChunkHolder(); + chunkSource.getNextChunk( + /* playbackPositionUs= */ 0, + /* loadPositionUs= */ 0, + /* queue= */ ImmutableList.of(), + output); + + assertThat(output.chunk.dataSpec.flags & DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED) + .isEqualTo(0); + } +} diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParserTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParserTest.java index c2ea12bcd75..c82f2012678 100644 --- a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParserTest.java +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParserTest.java @@ -42,7 +42,7 @@ @RunWith(AndroidJUnit4.class) public class DashManifestParserTest { - private static final String SAMPLE_MPD = "media/mpd/sample_mpd"; + private static final String SAMPLE_MPD_LIVE = "media/mpd/sample_mpd_live"; private static final String SAMPLE_MPD_UNKNOWN_MIME_TYPE = "media/mpd/sample_mpd_unknown_mime_type"; private static final String SAMPLE_MPD_SEGMENT_TEMPLATE = "media/mpd/sample_mpd_segment_template"; @@ -51,6 +51,18 @@ public class DashManifestParserTest { private static final String SAMPLE_MPD_ASSET_IDENTIFIER = "media/mpd/sample_mpd_asset_identifier"; private static final String SAMPLE_MPD_TEXT = "media/mpd/sample_mpd_text"; private static final String SAMPLE_MPD_TRICK_PLAY = "media/mpd/sample_mpd_trick_play"; + private static final String SAMPLE_MPD_AVAILABILITY_TIME_OFFSET_BASE_URL = + "media/mpd/sample_mpd_availabilityTimeOffset_baseUrl"; + private static final String SAMPLE_MPD_AVAILABILITY_TIME_OFFSET_SEGMENT_TEMPLATE = + "media/mpd/sample_mpd_availabilityTimeOffset_segmentTemplate"; + private static final String SAMPLE_MPD_AVAILABILITY_TIME_OFFSET_SEGMENT_LIST = + "media/mpd/sample_mpd_availabilityTimeOffset_segmentList"; + private static final String SAMPLE_MPD_SERVICE_DESCRIPTION_LOW_LATENCY = + "media/mpd/sample_mpd_service_description_low_latency"; + private static final String SAMPLE_MPD_SERVICE_DESCRIPTION_LOW_LATENCY_ONLY_PLAYBACK_RATES = + "media/mpd/sample_mpd_service_description_low_latency_only_playback_rates"; + private static final String SAMPLE_MPD_SERVICE_DESCRIPTION_LOW_LATENCY_ONLY_TARGET_LATENCY = + "media/mpd/sample_mpd_service_description_low_latency_only_target_latency"; private static final String NEXT_TAG_NAME = "Next"; private static final String NEXT_TAG = "<" + NEXT_TAG_NAME + "/>"; @@ -61,7 +73,7 @@ public void parseMediaPresentationDescription() throws IOException { DashManifestParser parser = new DashManifestParser(); parser.parse( Uri.parse("https://example.com/test.mpd"), - TestUtil.getInputStream(ApplicationProvider.getApplicationContext(), SAMPLE_MPD)); + TestUtil.getInputStream(ApplicationProvider.getApplicationContext(), SAMPLE_MPD_LIVE)); parser.parse( Uri.parse("https://example.com/test.mpd"), TestUtil.getInputStream( @@ -172,7 +184,7 @@ public void parseMediaPresentationDescription_programInformation() throws IOExce DashManifest manifest = parser.parse( Uri.parse("https://example.com/test.mpd"), - TestUtil.getInputStream(ApplicationProvider.getApplicationContext(), SAMPLE_MPD)); + TestUtil.getInputStream(ApplicationProvider.getApplicationContext(), SAMPLE_MPD_LIVE)); ProgramInformation expectedProgramInformation = new ProgramInformation( "MediaTitle", "MediaSource", "MediaCopyright", "www.example.com", "enUs"); @@ -469,6 +481,163 @@ public void parsePeriodAssetIdentifier() throws IOException { assertThat(assetIdentifier.id).isEqualTo("uniqueId"); } + @Test + public void availabilityTimeOffset_staticManifest_setToTimeUnset() throws IOException { + DashManifestParser parser = new DashManifestParser(); + DashManifest manifest = + parser.parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream(ApplicationProvider.getApplicationContext(), SAMPLE_MPD_TEXT)); + + assertThat(manifest.getPeriodCount()).isEqualTo(1); + List adaptationSets = manifest.getPeriod(0).adaptationSets; + assertThat(adaptationSets).hasSize(3); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets.get(0))).isEqualTo(C.TIME_UNSET); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets.get(1))).isEqualTo(C.TIME_UNSET); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets.get(2))).isEqualTo(C.TIME_UNSET); + } + + @Test + public void availabilityTimeOffset_dynamicManifest_valuesInBaseUrl_setsCorrectValues() + throws IOException { + DashManifestParser parser = new DashManifestParser(); + DashManifest manifest = + parser.parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), + SAMPLE_MPD_AVAILABILITY_TIME_OFFSET_BASE_URL)); + + assertThat(manifest.getPeriodCount()).isEqualTo(2); + List adaptationSets0 = manifest.getPeriod(0).adaptationSets; + List adaptationSets1 = manifest.getPeriod(1).adaptationSets; + assertThat(adaptationSets0).hasSize(4); + assertThat(adaptationSets1).hasSize(1); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(0))).isEqualTo(5_000_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(1))).isEqualTo(4_321_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(2))).isEqualTo(9_876_543); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(3))).isEqualTo(C.TIME_UNSET); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets1.get(0))).isEqualTo(0); + } + + @Test + public void availabilityTimeOffset_dynamicManifest_valuesInSegmentTemplate_setsCorrectValues() + throws IOException { + DashManifestParser parser = new DashManifestParser(); + DashManifest manifest = + parser.parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), + SAMPLE_MPD_AVAILABILITY_TIME_OFFSET_SEGMENT_TEMPLATE)); + + assertThat(manifest.getPeriodCount()).isEqualTo(2); + List adaptationSets0 = manifest.getPeriod(0).adaptationSets; + List adaptationSets1 = manifest.getPeriod(1).adaptationSets; + assertThat(adaptationSets0).hasSize(4); + assertThat(adaptationSets1).hasSize(1); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(0))).isEqualTo(2_000_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(1))).isEqualTo(3_210_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(2))).isEqualTo(1_230_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(3))).isEqualTo(100_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets1.get(0))).isEqualTo(9_999_000); + } + + @Test + public void availabilityTimeOffset_dynamicManifest_valuesInSegmentList_setsCorrectValues() + throws IOException { + DashManifestParser parser = new DashManifestParser(); + DashManifest manifest = + parser.parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), + SAMPLE_MPD_AVAILABILITY_TIME_OFFSET_SEGMENT_LIST)); + + assertThat(manifest.getPeriodCount()).isEqualTo(2); + List adaptationSets0 = manifest.getPeriod(0).adaptationSets; + List adaptationSets1 = manifest.getPeriod(1).adaptationSets; + assertThat(adaptationSets0).hasSize(4); + assertThat(adaptationSets1).hasSize(1); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(0))).isEqualTo(2_000_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(1))).isEqualTo(3_210_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(2))).isEqualTo(1_230_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets0.get(3))).isEqualTo(100_000); + assertThat(getAvailabilityTimeOffsetUs(adaptationSets1.get(0))).isEqualTo(9_999_000); + } + + @Test + public void serviceDescriptionElement_allValuesSet() throws IOException { + DashManifestParser parser = new DashManifestParser(); + + DashManifest manifest = + parser.parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), + SAMPLE_MPD_SERVICE_DESCRIPTION_LOW_LATENCY)); + + assertThat(manifest.serviceDescription).isNotNull(); + assertThat(manifest.serviceDescription.targetOffsetMs).isEqualTo(20_000); + assertThat(manifest.serviceDescription.minOffsetMs).isEqualTo(1_000); + assertThat(manifest.serviceDescription.maxOffsetMs).isEqualTo(30_000); + assertThat(manifest.serviceDescription.minPlaybackSpeed).isEqualTo(0.1f); + assertThat(manifest.serviceDescription.maxPlaybackSpeed).isEqualTo(99f); + } + + @Test + public void serviceDescriptionElement_onlyPlaybackRates_latencyValuesUnset() throws IOException { + DashManifestParser parser = new DashManifestParser(); + + DashManifest manifest = + parser.parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), + SAMPLE_MPD_SERVICE_DESCRIPTION_LOW_LATENCY_ONLY_PLAYBACK_RATES)); + + assertThat(manifest.serviceDescription).isNotNull(); + assertThat(manifest.serviceDescription.targetOffsetMs).isEqualTo(C.TIME_UNSET); + assertThat(manifest.serviceDescription.minOffsetMs).isEqualTo(C.TIME_UNSET); + assertThat(manifest.serviceDescription.maxOffsetMs).isEqualTo(C.TIME_UNSET); + assertThat(manifest.serviceDescription.minPlaybackSpeed).isEqualTo(0.1f); + assertThat(manifest.serviceDescription.maxPlaybackSpeed).isEqualTo(99f); + } + + @Test + public void serviceDescriptionElement_onlyTargetLatency_playbackRatesAndMinMaxLatencyUnset() + throws IOException { + DashManifestParser parser = new DashManifestParser(); + + DashManifest manifest = + parser.parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), + SAMPLE_MPD_SERVICE_DESCRIPTION_LOW_LATENCY_ONLY_TARGET_LATENCY)); + + assertThat(manifest.serviceDescription).isNotNull(); + assertThat(manifest.serviceDescription.targetOffsetMs).isEqualTo(20_000); + assertThat(manifest.serviceDescription.minOffsetMs).isEqualTo(C.TIME_UNSET); + assertThat(manifest.serviceDescription.maxOffsetMs).isEqualTo(C.TIME_UNSET); + assertThat(manifest.serviceDescription.minPlaybackSpeed).isEqualTo(C.RATE_UNSET); + assertThat(manifest.serviceDescription.maxPlaybackSpeed).isEqualTo(C.RATE_UNSET); + } + + @Test + public void serviceDescriptionElement_noServiceDescription_isNullInManifest() throws IOException { + DashManifestParser parser = new DashManifestParser(); + + DashManifest manifest = + parser.parse( + Uri.parse("https://example.com/test.mpd"), + TestUtil.getInputStream( + ApplicationProvider.getApplicationContext(), + SAMPLE_MPD_AVAILABILITY_TIME_OFFSET_SEGMENT_LIST)); + + assertThat(manifest.serviceDescription).isNull(); + } + private static List buildCea608AccessibilityDescriptors(String value) { return Collections.singletonList(new Descriptor("urn:scte:dash:cc:cea-608:2015", value, null)); } @@ -482,4 +651,13 @@ private static void assertNextTag(XmlPullParser xpp) throws Exception { assertThat(xpp.getEventType()).isEqualTo(XmlPullParser.START_TAG); assertThat(xpp.getName()).isEqualTo(NEXT_TAG_NAME); } + + private static long getAvailabilityTimeOffsetUs(AdaptationSet adaptationSet) { + assertThat(adaptationSet.representations).isNotEmpty(); + Representation representation = adaptationSet.representations.get(0); + assertThat(representation).isInstanceOf(Representation.MultiSegmentRepresentation.class); + SegmentBase.MultiSegmentBase segmentBase = + ((Representation.MultiSegmentRepresentation) representation).segmentBase; + return segmentBase.availabilityTimeOffsetUs; + } } diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestTest.java index a1b971068dd..7b3b3cab512 100644 --- a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestTest.java +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestTest.java @@ -18,6 +18,7 @@ import static com.google.common.truth.Truth.assertThat; import android.net.Uri; +import androidx.annotation.Nullable; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.offline.StreamKey; @@ -40,9 +41,17 @@ public class DashManifestTest { @Test public void copy() { Representation[][][] representations = newRepresentations(3, 2, 3); + ServiceDescriptionElement serviceDescriptionElement = + new ServiceDescriptionElement( + /* targetOffsetMs= */ 20, + /* minOffsetMs= */ 10, + /* maxOffsetMs= */ 40, + /* minPlaybackSpeed= */ 0.9f, + /* maxPlaybackSpeed= */ 1.1f); DashManifest sourceManifest = newDashManifest( 10, + serviceDescriptionElement, newPeriod( "1", 1, @@ -78,6 +87,7 @@ public void copy() { DashManifest expectedManifest = newDashManifest( 10, + serviceDescriptionElement, newPeriod( "1", 1, @@ -102,6 +112,7 @@ public void copySameAdaptationIndexButDifferentPeriod() { DashManifest sourceManifest = newDashManifest( 10, + /* serviceDescription= */ null, newPeriod("1", 1, newAdaptationSet(2, representations[0][0])), newPeriod("4", 4, newAdaptationSet(5, representations[1][0]))); @@ -111,6 +122,7 @@ public void copySameAdaptationIndexButDifferentPeriod() { DashManifest expectedManifest = newDashManifest( 10, + /* serviceDescription= */ null, newPeriod("1", 1, newAdaptationSet(2, representations[0][0])), newPeriod("4", 4, newAdaptationSet(5, representations[1][0]))); assertManifestEquals(expectedManifest, copyManifest); @@ -122,6 +134,7 @@ public void copySkipPeriod() { DashManifest sourceManifest = newDashManifest( 10, + /* serviceDescription= */ null, newPeriod( "1", 1, @@ -151,6 +164,7 @@ public void copySkipPeriod() { DashManifest expectedManifest = newDashManifest( 7, + /* serviceDescription= */ null, newPeriod( "1", 1, @@ -177,6 +191,7 @@ private static void assertManifestEquals(DashManifest expected, DashManifest act assertThat(actual.utcTiming).isEqualTo(expected.utcTiming); assertThat(actual.location).isEqualTo(expected.location); assertThat(actual.getPeriodCount()).isEqualTo(expected.getPeriodCount()); + assertThat(actual.serviceDescription).isEqualTo(expected.serviceDescription); for (int i = 0; i < expected.getPeriodCount(); i++) { Period expectedPeriod = expected.getPeriod(i); Period actualPeriod = actual.getPeriod(i); @@ -217,7 +232,8 @@ private static Representation newRepresentation() { return Representation.newInstance(/* revisionId= */ 0, FORMAT, /* baseUrl= */ "", SEGMENT_BASE); } - private static DashManifest newDashManifest(int duration, Period... periods) { + private static DashManifest newDashManifest( + int duration, @Nullable ServiceDescriptionElement serviceDescription, Period... periods) { return new DashManifest( /* availabilityStartTimeMs= */ 0, duration, @@ -229,6 +245,7 @@ private static DashManifest newDashManifest(int duration, Period... periods) { /* publishTimeMs= */ 12345, /* programInformation= */ null, UTC_TIMING, + serviceDescription, Uri.EMPTY, Arrays.asList(periods)); } diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/SegmentBaseTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/SegmentBaseTest.java new file mode 100644 index 00000000000..dd442a91f45 --- /dev/null +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/manifest/SegmentBaseTest.java @@ -0,0 +1,185 @@ +/* + * Copyright (C) 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.dash.manifest; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link SegmentBase}. */ +@RunWith(AndroidJUnit4.class) +public final class SegmentBaseTest { + + @Test + public void getFirstAvailableSegmentNum_unboundedSegmentTemplate() { + long periodStartUnixTimeUs = 123_000_000_000_000L; + SegmentBase.SegmentTemplate segmentTemplate = + new SegmentBase.SegmentTemplate( + /* initialization= */ null, + /* timescale= */ 1000, + /* presentationTimeOffset= */ 0, + /* startNumber= */ 42, + /* endNumber= */ C.INDEX_UNSET, + /* duration= */ 2000, + /* segmentTimeline= */ null, + /* availabilityTimeOffsetUs= */ 500_000, + /* initializationTemplate= */ null, + /* mediaTemplate= */ null, + /* timeShiftBufferDepthUs= */ 6_000_000, + /* periodStartUnixTimeUs= */ periodStartUnixTimeUs); + + assertThat( + segmentTemplate.getFirstAvailableSegmentNum( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs - 10_000_000)) + .isEqualTo(42); + assertThat( + segmentTemplate.getFirstAvailableSegmentNum( + /* periodDurationUs= */ C.TIME_UNSET, /* nowUnixTimeUs= */ periodStartUnixTimeUs)) + .isEqualTo(42); + assertThat( + segmentTemplate.getFirstAvailableSegmentNum( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 7_999_999)) + .isEqualTo(42); + assertThat( + segmentTemplate.getFirstAvailableSegmentNum( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 8_000_000)) + .isEqualTo(43); + assertThat( + segmentTemplate.getFirstAvailableSegmentNum( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 9_999_999)) + .isEqualTo(43); + assertThat( + segmentTemplate.getFirstAvailableSegmentNum( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 10_000_000)) + .isEqualTo(44); + } + + @Test + public void getAvailableSegmentCount_unboundedSegmentTemplate() { + long periodStartUnixTimeUs = 123_000_000_000_000L; + SegmentBase.SegmentTemplate segmentTemplate = + new SegmentBase.SegmentTemplate( + /* initialization= */ null, + /* timescale= */ 1000, + /* presentationTimeOffset= */ 0, + /* startNumber= */ 42, + /* endNumber= */ C.INDEX_UNSET, + /* duration= */ 2000, + /* segmentTimeline= */ null, + /* availabilityTimeOffsetUs= */ 500_000, + /* initializationTemplate= */ null, + /* mediaTemplate= */ null, + /* timeShiftBufferDepthUs= */ 6_000_000, + /* periodStartUnixTimeUs= */ periodStartUnixTimeUs); + + assertThat( + segmentTemplate.getAvailableSegmentCount( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs - 10_000_000)) + .isEqualTo(0); + assertThat( + segmentTemplate.getAvailableSegmentCount( + /* periodDurationUs= */ C.TIME_UNSET, /* nowUnixTimeUs= */ periodStartUnixTimeUs)) + .isEqualTo(0); + assertThat( + segmentTemplate.getAvailableSegmentCount( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 1_499_999)) + .isEqualTo(0); + assertThat( + segmentTemplate.getAvailableSegmentCount( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 1_500_000)) + .isEqualTo(1); + assertThat( + segmentTemplate.getAvailableSegmentCount( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 7_499_999)) + .isEqualTo(3); + assertThat( + segmentTemplate.getAvailableSegmentCount( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 7_500_000)) + .isEqualTo(4); + assertThat( + segmentTemplate.getAvailableSegmentCount( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 7_999_999)) + .isEqualTo(4); + assertThat( + segmentTemplate.getAvailableSegmentCount( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 8_000_000)) + .isEqualTo(3); + } + + @Test + public void getNextSegmentShiftTimeUse_unboundedSegmentTemplate() { + long periodStartUnixTimeUs = 123_000_000_000_000L; + SegmentBase.SegmentTemplate segmentTemplate = + new SegmentBase.SegmentTemplate( + /* initialization= */ null, + /* timescale= */ 1000, + /* presentationTimeOffset= */ 0, + /* startNumber= */ 42, + /* endNumber= */ C.INDEX_UNSET, + /* duration= */ 2000, + /* segmentTimeline= */ null, + /* availabilityTimeOffsetUs= */ 500_000, + /* initializationTemplate= */ null, + /* mediaTemplate= */ null, + /* timeShiftBufferDepthUs= */ 6_000_000, + /* periodStartUnixTimeUs= */ periodStartUnixTimeUs); + + assertThat( + segmentTemplate.getNextSegmentAvailableTimeUs( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs - 10_000_000)) + .isEqualTo(1_500_000); + assertThat( + segmentTemplate.getNextSegmentAvailableTimeUs( + /* periodDurationUs= */ C.TIME_UNSET, /* nowUnixTimeUs= */ periodStartUnixTimeUs)) + .isEqualTo(1_500_000); + assertThat( + segmentTemplate.getNextSegmentAvailableTimeUs( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 1_499_999)) + .isEqualTo(1_500_000); + assertThat( + segmentTemplate.getNextSegmentAvailableTimeUs( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 1_500_000)) + .isEqualTo(3_500_000); + assertThat( + segmentTemplate.getNextSegmentAvailableTimeUs( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 17_499_999)) + .isEqualTo(17_500_000); + assertThat( + segmentTemplate.getNextSegmentAvailableTimeUs( + /* periodDurationUs= */ C.TIME_UNSET, + /* nowUnixTimeUs= */ periodStartUnixTimeUs + 17_500_000)) + .isEqualTo(19_500_000); + } +} diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadHelperTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadHelperTest.java index b2fae93bcaf..bfc11cb47ae 100644 --- a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadHelperTest.java +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadHelperTest.java @@ -42,6 +42,6 @@ public void staticDownloadHelperForDash_doesNotThrow() { DownloadHelper.DEFAULT_TRACK_SELECTOR_PARAMETERS_WITHOUT_CONTEXT, (handler, videoListener, audioListener, text, metadata) -> new Renderer[0], new FakeDataSource.Factory(), - /* drmSessionManager= */ DrmSessionManager.getDummyDrmSessionManager()); + /* drmSessionManager= */ DrmSessionManager.DRM_UNSUPPORTED); } } diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadManagerDashTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadManagerDashTest.java index 2993bb4442a..6bdc84438b9 100644 --- a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadManagerDashTest.java +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadManagerDashTest.java @@ -31,13 +31,13 @@ import com.google.android.exoplayer2.offline.DownloadManager; import com.google.android.exoplayer2.offline.DownloadRequest; import com.google.android.exoplayer2.offline.StreamKey; +import com.google.android.exoplayer2.robolectric.TestDownloadManagerListener; import com.google.android.exoplayer2.scheduler.Requirements; import com.google.android.exoplayer2.testutil.CacheAsserts.RequestSet; import com.google.android.exoplayer2.testutil.DummyMainThread; import com.google.android.exoplayer2.testutil.DummyMainThread.TestRunnable; import com.google.android.exoplayer2.testutil.FakeDataSet; import com.google.android.exoplayer2.testutil.FakeDataSource; -import com.google.android.exoplayer2.testutil.TestDownloadManagerListener; import com.google.android.exoplayer2.testutil.TestUtil; import com.google.android.exoplayer2.upstream.DataSource.Factory; import com.google.android.exoplayer2.upstream.cache.CacheDataSource; diff --git a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadServiceDashTest.java b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadServiceDashTest.java index 6b528cdd824..98a7f6e8873 100644 --- a/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadServiceDashTest.java +++ b/library/dash/src/test/java/com/google/android/exoplayer2/source/dash/offline/DownloadServiceDashTest.java @@ -34,11 +34,11 @@ import com.google.android.exoplayer2.offline.DownloadRequest; import com.google.android.exoplayer2.offline.DownloadService; import com.google.android.exoplayer2.offline.StreamKey; +import com.google.android.exoplayer2.robolectric.TestDownloadManagerListener; import com.google.android.exoplayer2.scheduler.Scheduler; import com.google.android.exoplayer2.testutil.DummyMainThread; import com.google.android.exoplayer2.testutil.FakeDataSet; import com.google.android.exoplayer2.testutil.FakeDataSource; -import com.google.android.exoplayer2.testutil.TestDownloadManagerListener; import com.google.android.exoplayer2.testutil.TestUtil; import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.cache.CacheDataSource; diff --git a/library/extractor/proguard-rules.txt b/library/extractor/proguard-rules.txt index d79f79a4a16..c39f894dfee 100644 --- a/library/extractor/proguard-rules.txt +++ b/library/extractor/proguard-rules.txt @@ -1,10 +1,14 @@ # Proguard rules specific to the extractor module. -# Constructors accessed via reflection in DefaultExtractorsFactory +# Methods accessed via reflection in DefaultExtractorsFactory -dontnote com.google.android.exoplayer2.ext.flac.FlacExtractor -keepclassmembers class com.google.android.exoplayer2.ext.flac.FlacExtractor { (int); } +-dontnote com.google.android.exoplayer2.ext.flac.FlacLibrary +-keepclassmembers class com.google.android.exoplayer2.ext.flac.FlacLibrary { + public static boolean isAvailable(); +} # Don't warn about checkerframework and Kotlin annotations -dontwarn org.checkerframework.** diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/DefaultExtractorsFactory.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/DefaultExtractorsFactory.java index 2068853d9ea..ff887000a3d 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/DefaultExtractorsFactory.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/DefaultExtractorsFactory.java @@ -23,6 +23,7 @@ import com.google.android.exoplayer2.extractor.amr.AmrExtractor; import com.google.android.exoplayer2.extractor.flac.FlacExtractor; import com.google.android.exoplayer2.extractor.flv.FlvExtractor; +import com.google.android.exoplayer2.extractor.jpeg.JpegExtractor; import com.google.android.exoplayer2.extractor.mkv.MatroskaExtractor; import com.google.android.exoplayer2.extractor.mp3.Mp3Extractor; import com.google.android.exoplayer2.extractor.mp4.FragmentedMp4Extractor; @@ -69,12 +70,15 @@ * generally include a FLAC decoder before API 27. This can be worked around by using * the FLAC extension or the FFmpeg extension. * + *

  • JPEG ({@link JpegExtractor}) * */ public final class DefaultExtractorsFactory implements ExtractorsFactory { // Extractors order is optimized according to // https://docs.google.com/document/d/1w2mKaWMxfz2Ei8-LdxqbPs1VLe_oudB-eryXXw9OvQQ. + // The JPEG extractor appears after audio/video extractors because we expect audio/video input to + // be more common. private static final int[] DEFAULT_EXTRACTOR_ORDER = new int[] { FileTypes.FLV, @@ -90,6 +94,7 @@ public final class DefaultExtractorsFactory implements ExtractorsFactory { FileTypes.AC3, FileTypes.AC4, FileTypes.MP3, + FileTypes.JPEG, }; @Nullable @@ -382,6 +387,11 @@ private void addExtractorsForFileType(@FileTypes.Type int fileType, List void setRetryPosition(long position, E e) throws E { + input.setRetryPosition(position, e); + } +} diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/flac/FlacExtractor.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/flac/FlacExtractor.java index 48fc13a7355..2b6e3415003 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/flac/FlacExtractor.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/flac/FlacExtractor.java @@ -303,13 +303,11 @@ private void getFrameStartMarker(ExtractorInput input) throws IOException { if (buffer.bytesLeft() < FlacConstants.MAX_FRAME_HEADER_SIZE) { // The next frame header may not fit in the rest of the buffer, so put the trailing bytes at // the start of the buffer, and reset the position and limit. + int bytesLeft = buffer.bytesLeft(); System.arraycopy( - buffer.getData(), - buffer.getPosition(), - buffer.getData(), - /* destPos= */ 0, - buffer.bytesLeft()); - buffer.reset(buffer.bytesLeft()); + buffer.getData(), buffer.getPosition(), buffer.getData(), /* destPos= */ 0, bytesLeft); + buffer.setPosition(0); + buffer.setLimit(bytesLeft); } return Extractor.RESULT_CONTINUE; diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/flv/VideoTagPayloadReader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/flv/VideoTagPayloadReader.java index c91f6ce0371..6ab4da1acf5 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/flv/VideoTagPayloadReader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/flv/VideoTagPayloadReader.java @@ -93,6 +93,7 @@ protected boolean parsePayload(ParsableByteArray data, long timeUs) throws Parse Format format = new Format.Builder() .setSampleMimeType(MimeTypes.VIDEO_H264) + .setCodecs(avcConfig.codecs) .setWidth(avcConfig.width) .setHeight(avcConfig.height) .setPixelWidthHeightRatio(avcConfig.pixelWidthAspectRatio) diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/JpegExtractor.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/JpegExtractor.java new file mode 100644 index 00000000000..3dbbc85d845 --- /dev/null +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/JpegExtractor.java @@ -0,0 +1,282 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.jpeg; + +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; + +import androidx.annotation.IntDef; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.extractor.Extractor; +import com.google.android.exoplayer2.extractor.ExtractorInput; +import com.google.android.exoplayer2.extractor.ExtractorOutput; +import com.google.android.exoplayer2.extractor.PositionHolder; +import com.google.android.exoplayer2.extractor.SeekMap; +import com.google.android.exoplayer2.extractor.TrackOutput; +import com.google.android.exoplayer2.extractor.mp4.Mp4Extractor; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.mp4.MotionPhotoMetadata; +import com.google.android.exoplayer2.util.ParsableByteArray; +import java.io.IOException; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; + +/** Extracts JPEG image using the Exif format. */ +public final class JpegExtractor implements Extractor { + + /** Parser states. */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + STATE_READING_MARKER, + STATE_READING_SEGMENT_LENGTH, + STATE_READING_SEGMENT, + STATE_SNIFFING_MOTION_PHOTO_VIDEO, + STATE_READING_MOTION_PHOTO_VIDEO, + STATE_ENDED, + }) + private @interface State {} + + private static final int STATE_READING_MARKER = 0; + private static final int STATE_READING_SEGMENT_LENGTH = 1; + private static final int STATE_READING_SEGMENT = 2; + private static final int STATE_SNIFFING_MOTION_PHOTO_VIDEO = 4; + private static final int STATE_READING_MOTION_PHOTO_VIDEO = 5; + private static final int STATE_ENDED = 6; + + private static final int JPEG_EXIF_HEADER_LENGTH = 12; + private static final long EXIF_HEADER = 0x45786966; // Exif + private static final int MARKER_SOI = 0xFFD8; // Start of image marker + private static final int MARKER_SOS = 0xFFDA; // Start of scan (image data) marker + private static final int MARKER_APP1 = 0xFFE1; // Application data 1 marker + private static final String HEADER_XMP_APP1 = "http://ns.adobe.com/xap/1.0/"; + + /** + * The identifier to use for the image track. Chosen to avoid colliding with track IDs used by + * {@link Mp4Extractor} for motion photos. + */ + private static final int IMAGE_TRACK_ID = 1024; + + private final ParsableByteArray scratch; + + private @MonotonicNonNull ExtractorOutput extractorOutput; + + @State private int state; + private int marker; + private int segmentLength; + private long mp4StartPosition; + + @Nullable private MotionPhotoMetadata motionPhotoMetadata; + private @MonotonicNonNull ExtractorInput lastExtractorInput; + private @MonotonicNonNull StartOffsetExtractorInput mp4ExtractorStartOffsetExtractorInput; + private @MonotonicNonNull Mp4Extractor mp4Extractor; + + public JpegExtractor() { + scratch = new ParsableByteArray(JPEG_EXIF_HEADER_LENGTH); + mp4StartPosition = C.POSITION_UNSET; + } + + @Override + public boolean sniff(ExtractorInput input) throws IOException { + // See ITU-T.81 (1992) subsection B.1.1.3 and Exif version 2.2 (2002) subsection 4.5.4. + input.peekFully(scratch.getData(), /* offset= */ 0, JPEG_EXIF_HEADER_LENGTH); + if (scratch.readUnsignedShort() != MARKER_SOI || scratch.readUnsignedShort() != MARKER_APP1) { + return false; + } + scratch.skipBytes(2); // Unused segment length + return scratch.readUnsignedInt() == EXIF_HEADER && scratch.readUnsignedShort() == 0; // Exif\0\0 + } + + @Override + public void init(ExtractorOutput output) { + extractorOutput = output; + } + + @Override + @ReadResult + public int read(ExtractorInput input, PositionHolder seekPosition) throws IOException { + switch (state) { + case STATE_READING_MARKER: + readMarker(input); + return RESULT_CONTINUE; + case STATE_READING_SEGMENT_LENGTH: + readSegmentLength(input); + return RESULT_CONTINUE; + case STATE_READING_SEGMENT: + readSegment(input); + return RESULT_CONTINUE; + case STATE_SNIFFING_MOTION_PHOTO_VIDEO: + if (input.getPosition() != mp4StartPosition) { + seekPosition.position = mp4StartPosition; + return RESULT_SEEK; + } + sniffMotionPhotoVideo(input); + return RESULT_CONTINUE; + case STATE_READING_MOTION_PHOTO_VIDEO: + if (mp4ExtractorStartOffsetExtractorInput == null || input != lastExtractorInput) { + lastExtractorInput = input; + mp4ExtractorStartOffsetExtractorInput = + new StartOffsetExtractorInput(input, mp4StartPosition); + } + @ReadResult + int readResult = + checkNotNull(mp4Extractor).read(mp4ExtractorStartOffsetExtractorInput, seekPosition); + if (readResult == RESULT_SEEK) { + seekPosition.position += mp4StartPosition; + } + return readResult; + case STATE_ENDED: + return RESULT_END_OF_INPUT; + default: + throw new IllegalStateException(); + } + } + + @Override + public void seek(long position, long timeUs) { + if (position == 0) { + state = STATE_READING_MARKER; + } else if (state == STATE_READING_MOTION_PHOTO_VIDEO) { + checkNotNull(mp4Extractor).seek(position, timeUs); + } + } + + @Override + public void release() { + if (mp4Extractor != null) { + mp4Extractor.release(); + } + } + + private void readMarker(ExtractorInput input) throws IOException { + scratch.reset(/* limit= */ 2); + input.readFully(scratch.getData(), /* offset= */ 0, /* length= */ 2); + marker = scratch.readUnsignedShort(); + if (marker == MARKER_SOS) { // Start of scan. + if (mp4StartPosition != C.POSITION_UNSET) { + state = STATE_SNIFFING_MOTION_PHOTO_VIDEO; + } else { + endReadingWithImageTrack(); + } + } else if ((marker < 0xFFD0 || marker > 0xFFD9) && marker != 0xFF01) { + state = STATE_READING_SEGMENT_LENGTH; + } + } + + private void readSegmentLength(ExtractorInput input) throws IOException { + scratch.reset(2); + input.readFully(scratch.getData(), /* offset= */ 0, /* length= */ 2); + segmentLength = scratch.readUnsignedShort() - 2; + state = STATE_READING_SEGMENT; + } + + private void readSegment(ExtractorInput input) throws IOException { + if (marker == MARKER_APP1) { + ParsableByteArray payload = new ParsableByteArray(segmentLength); + input.readFully(payload.getData(), /* offset= */ 0, /* length= */ segmentLength); + if (motionPhotoMetadata == null + && HEADER_XMP_APP1.equals(payload.readNullTerminatedString())) { + @Nullable String xmpString = payload.readNullTerminatedString(); + if (xmpString != null) { + motionPhotoMetadata = getMotionPhotoMetadata(xmpString, input.getLength()); + if (motionPhotoMetadata != null) { + mp4StartPosition = motionPhotoMetadata.videoStartPosition; + } + } + } + } else { + input.skipFully(segmentLength); + } + state = STATE_READING_MARKER; + } + + private void sniffMotionPhotoVideo(ExtractorInput input) throws IOException { + // Check if the file is truncated. + boolean peekedData = + input.peekFully( + scratch.getData(), /* offset= */ 0, /* length= */ 1, /* allowEndOfInput= */ true); + if (!peekedData) { + endReadingWithImageTrack(); + } else { + input.resetPeekPosition(); + if (mp4Extractor == null) { + mp4Extractor = new Mp4Extractor(); + } + mp4ExtractorStartOffsetExtractorInput = + new StartOffsetExtractorInput(input, mp4StartPosition); + if (mp4Extractor.sniff(mp4ExtractorStartOffsetExtractorInput)) { + mp4Extractor.init( + new StartOffsetExtractorOutput(mp4StartPosition, checkNotNull(extractorOutput))); + startReadingMotionPhoto(); + } else { + endReadingWithImageTrack(); + } + } + } + + private void startReadingMotionPhoto() { + outputImageTrack(checkNotNull(motionPhotoMetadata)); + state = STATE_READING_MOTION_PHOTO_VIDEO; + } + + private void endReadingWithImageTrack() { + outputImageTrack(); + checkNotNull(extractorOutput).endTracks(); + extractorOutput.seekMap(new SeekMap.Unseekable(/* durationUs= */ C.TIME_UNSET)); + state = STATE_ENDED; + } + + private void outputImageTrack(Metadata.Entry... metadataEntries) { + TrackOutput imageTrackOutput = + checkNotNull(extractorOutput).track(IMAGE_TRACK_ID, C.TRACK_TYPE_IMAGE); + imageTrackOutput.format( + new Format.Builder().setMetadata(new Metadata(metadataEntries)).build()); + } + + /** + * Attempts to parse the specified XMP data describing the motion photo, returning the resulting + * {@link MotionPhotoMetadata} or {@code null} if it wasn't possible to derive motion photo + * metadata. + * + * @param xmpString A string of XML containing XMP motion photo metadata to attempt to parse. + * @param inputLength The length of the input stream in bytes, or {@link C#LENGTH_UNSET} if + * unknown. + * @return The {@link MotionPhotoMetadata}, or {@code null} if it wasn't possible to derive motion + * photo metadata. + * @throws IOException If an error occurs parsing the XMP string. + */ + @Nullable + private static MotionPhotoMetadata getMotionPhotoMetadata(String xmpString, long inputLength) + throws IOException { + // Metadata defines offsets from the end of the stream, so we need the stream length to + // determine start offsets. + if (inputLength == C.LENGTH_UNSET) { + return null; + } + + // Motion photos have (at least) a primary image media item and a secondary video media item. + @Nullable + MotionPhotoDescription motionPhotoDescription = + XmpMotionPhotoDescriptionParser.parse(xmpString); + if (motionPhotoDescription == null) { + return null; + } + return motionPhotoDescription.getMotionPhotoMetadata(inputLength); + } +} diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/MotionPhotoDescription.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/MotionPhotoDescription.java new file mode 100644 index 00000000000..3117dfa5f49 --- /dev/null +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/MotionPhotoDescription.java @@ -0,0 +1,122 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.jpeg; + +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.metadata.mp4.MotionPhotoMetadata; +import com.google.android.exoplayer2.util.MimeTypes; +import java.util.List; + +/** Describes the layout and metadata of a motion photo file. */ +/* package */ final class MotionPhotoDescription { + + /** Describes a media item in the motion photo. */ + public static final class ContainerItem { + /** The MIME type of the media item. */ + public final String mime; + /** The application-specific meaning of the media item. */ + public final String semantic; + /** + * The positive integer length in bytes of the media item, or 0 for primary media items and + * secondary media items that share their resource with the preceding media item. + */ + public final long length; + /** + * The number of bytes of additional padding between the end of the primary media item and the + * start of the next media item. 0 for secondary media items. + */ + public final long padding; + + public ContainerItem(String mime, String semantic, long length, long padding) { + this.mime = mime; + this.semantic = semantic; + this.length = length; + this.padding = padding; + } + } + + /** + * The presentation timestamp of the primary media item, in microseconds, or {@link C#TIME_UNSET} + * if unknown. + */ + public final long photoPresentationTimestampUs; + /** + * The media items represented by the motion photo file, in order. The primary media item is + * listed first, followed by any secondary media items. + */ + public final List items; + + public MotionPhotoDescription(long photoPresentationTimestampUs, List items) { + this.photoPresentationTimestampUs = photoPresentationTimestampUs; + this.items = items; + } + + /** + * Returns the {@link MotionPhotoMetadata} for the motion photo represented by this instance, or + * {@code null} if there wasn't enough information to derive the metadata. + * + * @param motionPhotoLength The length of the motion photo file, in bytes. + * @return The motion photo metadata, or {@code null}. + */ + @Nullable + public MotionPhotoMetadata getMotionPhotoMetadata(long motionPhotoLength) { + if (items.size() < 2) { + // We need a primary item (photo) and at least one secondary item (video). + return null; + } + // Iterate backwards through the items to find the earlier video in the list. If we find a video + // item with length zero, we need to keep scanning backwards to find the preceding item with + // non-zero length, which is the item that contains the video data. + long photoStartPosition = C.POSITION_UNSET; + long photoLength = C.LENGTH_UNSET; + long mp4StartPosition = C.POSITION_UNSET; + long mp4Length = C.LENGTH_UNSET; + boolean itemContainsMp4 = false; + long itemStartPosition = motionPhotoLength; + long itemEndPosition = motionPhotoLength; + for (int i = items.size() - 1; i >= 0; i--) { + MotionPhotoDescription.ContainerItem item = items.get(i); + itemContainsMp4 |= MimeTypes.VIDEO_MP4.equals(item.mime); + itemEndPosition = itemStartPosition; + if (i == 0) { + // Padding is only applied for the primary item. + itemStartPosition = 0; + itemEndPosition -= item.padding; + } else { + itemStartPosition -= item.length; + } + if (itemContainsMp4 && itemStartPosition != itemEndPosition) { + mp4StartPosition = itemStartPosition; + mp4Length = itemEndPosition - itemStartPosition; + // Reset in case there's another video earlier in the list. + itemContainsMp4 = false; + } + if (i == 0) { + photoStartPosition = itemStartPosition; + photoLength = itemEndPosition; + } + } + if (mp4StartPosition == C.POSITION_UNSET + || mp4Length == C.LENGTH_UNSET + || photoStartPosition == C.POSITION_UNSET + || photoLength == C.LENGTH_UNSET) { + return null; + } + return new MotionPhotoMetadata( + photoStartPosition, photoLength, photoPresentationTimestampUs, mp4StartPosition, mp4Length); + } +} diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/StartOffsetExtractorInput.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/StartOffsetExtractorInput.java new file mode 100644 index 00000000000..132660349b5 --- /dev/null +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/StartOffsetExtractorInput.java @@ -0,0 +1,69 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.jpeg; + +import static com.google.android.exoplayer2.util.Assertions.checkArgument; + +import com.google.android.exoplayer2.extractor.ExtractorInput; +import com.google.android.exoplayer2.extractor.ForwardingExtractorInput; + +/** + * An extractor input that wraps another extractor input and exposes data starting at a given start + * byte offset. + * + *

    This is useful for reading data from a container that's concatenated after some prefix data + * but where the container's extractor doesn't handle a non-zero start offset (for example, because + * it seeks to absolute positions read from the container data). + */ +/* package */ final class StartOffsetExtractorInput extends ForwardingExtractorInput { + + private final long startOffset; + + /** + * Creates a new wrapper reading from the given start byte offset. + * + * @param input The extractor input to wrap. The reading position must be at or after the start + * offset, otherwise data could be read from before the start offset. + * @param startOffset The offset from which this extractor input provides data, in bytes. + * @throws IllegalArgumentException Thrown if the start offset is before the current reading + * position. + */ + public StartOffsetExtractorInput(ExtractorInput input, long startOffset) { + super(input); + checkArgument(input.getPosition() >= startOffset); + this.startOffset = startOffset; + } + + @Override + public long getPosition() { + return super.getPosition() - startOffset; + } + + @Override + public long getPeekPosition() { + return super.getPeekPosition() - startOffset; + } + + @Override + public long getLength() { + return super.getLength() - startOffset; + } + + @Override + public void setRetryPosition(long position, E e) throws E { + super.setRetryPosition(position + startOffset, e); + } +} diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/StartOffsetExtractorOutput.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/StartOffsetExtractorOutput.java new file mode 100644 index 00000000000..d0c4730fcbe --- /dev/null +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/StartOffsetExtractorOutput.java @@ -0,0 +1,75 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.jpeg; + +import com.google.android.exoplayer2.extractor.ExtractorOutput; +import com.google.android.exoplayer2.extractor.SeekMap; +import com.google.android.exoplayer2.extractor.SeekPoint; +import com.google.android.exoplayer2.extractor.TrackOutput; + +/** + * An extractor output that wraps another extractor output and applies a give start byte offset to + * seek positions. + * + *

    This is useful for extracting from a container that's concatenated after some prefix data but + * where the container's extractor doesn't handle a non-zero start offset (for example, because it + * seeks to absolute positions read from the container data). + */ +public final class StartOffsetExtractorOutput implements ExtractorOutput { + + private final long startOffset; + private final ExtractorOutput extractorOutput; + + /** Creates a new wrapper reading from the given start byte offset. */ + public StartOffsetExtractorOutput(long startOffset, ExtractorOutput extractorOutput) { + this.startOffset = startOffset; + this.extractorOutput = extractorOutput; + } + + @Override + public TrackOutput track(int id, int type) { + return extractorOutput.track(id, type); + } + + @Override + public void endTracks() { + extractorOutput.endTracks(); + } + + @Override + public void seekMap(SeekMap seekMap) { + extractorOutput.seekMap( + new SeekMap() { + @Override + public boolean isSeekable() { + return seekMap.isSeekable(); + } + + @Override + public long getDurationUs() { + return seekMap.getDurationUs(); + } + + @Override + public SeekPoints getSeekPoints(long timeUs) { + SeekPoints seekPoints = seekMap.getSeekPoints(timeUs); + return new SeekPoints( + new SeekPoint(seekPoints.first.timeUs, seekPoints.first.position + startOffset), + new SeekPoint(seekPoints.second.timeUs, seekPoints.second.position + startOffset)); + } + }); + } +} diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/XmpMotionPhotoDescriptionParser.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/XmpMotionPhotoDescriptionParser.java new file mode 100644 index 00000000000..3273edc2d7c --- /dev/null +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/XmpMotionPhotoDescriptionParser.java @@ -0,0 +1,183 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.jpeg; + +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.ParserException; +import com.google.android.exoplayer2.util.Log; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.android.exoplayer2.util.XmlPullParserUtil; +import com.google.common.collect.ImmutableList; +import java.io.IOException; +import java.io.StringReader; +import java.util.List; +import org.xmlpull.v1.XmlPullParser; +import org.xmlpull.v1.XmlPullParserException; +import org.xmlpull.v1.XmlPullParserFactory; + +/** + * Parser for motion photo metadata, handling XMP following the Motion Photo V1 and Micro Video V1b + * specifications. + */ +/* package */ final class XmpMotionPhotoDescriptionParser { + + /** + * Attempts to parse the specified XMP data describing the motion photo, returning the resulting + * {@link MotionPhotoDescription} or {@code null} if it wasn't possible to derive a motion photo + * description. + * + * @param xmpString A string of XML containing XMP motion photo metadata to attempt to parse. + * @return The {@link MotionPhotoDescription}, or {@code null} if it wasn't possible to derive a + * motion photo description. + * @throws IOException If an error occurs reading data from the stream. + */ + @Nullable + public static MotionPhotoDescription parse(String xmpString) throws IOException { + try { + return parseInternal(xmpString); + } catch (XmlPullParserException | ParserException | NumberFormatException e) { + Log.w(TAG, "Ignoring unexpected XMP metadata"); + return null; + } + } + + private static final String TAG = "MotionPhotoXmpParser"; + + private static final String[] MOTION_PHOTO_ATTRIBUTE_NAMES = + new String[] { + "Camera:MotionPhoto", // Motion Photo V1 + "GCamera:MotionPhoto", // Motion Photo V1 (legacy element naming) + "Camera:MicroVideo", // Micro Video V1b + "GCamera:MicroVideo", // Micro Video V1b (legacy element naming) + }; + private static final String[] DESCRIPTION_MOTION_PHOTO_PRESENTATION_TIMESTAMP_ATTRIBUTE_NAMES = + new String[] { + "Camera:MotionPhotoPresentationTimestampUs", // Motion Photo V1 + "GCamera:MotionPhotoPresentationTimestampUs", // Motion Photo V1 (legacy element naming) + "Camera:MicroVideoPresentationTimestampUs", // Micro Video V1b + "GCamera:MicroVideoPresentationTimestampUs", // Micro Video V1b (legacy element naming) + }; + private static final String[] DESCRIPTION_MICRO_VIDEO_OFFSET_ATTRIBUTE_NAMES = + new String[] { + "Camera:MicroVideoOffset", // Micro Video V1b + "GCamera:MicroVideoOffset", // Micro Video V1b (legacy element naming) + }; + + @Nullable + private static MotionPhotoDescription parseInternal(String xmpString) + throws XmlPullParserException, IOException { + XmlPullParserFactory xmlPullParserFactory = XmlPullParserFactory.newInstance(); + XmlPullParser xpp = xmlPullParserFactory.newPullParser(); + xpp.setInput(new StringReader(xmpString)); + xpp.next(); + if (!XmlPullParserUtil.isStartTag(xpp, "x:xmpmeta")) { + throw new ParserException("Couldn't find xmp metadata"); + } + long motionPhotoPresentationTimestampUs = C.TIME_UNSET; + List containerItems = ImmutableList.of(); + do { + xpp.next(); + if (XmlPullParserUtil.isStartTag(xpp, "rdf:Description")) { + if (!parseMotionPhotoFlagFromDescription(xpp)) { + // The motion photo flag is not set, so the file should not be treated as a motion photo. + return null; + } + motionPhotoPresentationTimestampUs = + parseMotionPhotoPresentationTimestampUsFromDescription(xpp); + containerItems = parseMicroVideoOffsetFromDescription(xpp); + } else if (XmlPullParserUtil.isStartTag(xpp, "Container:Directory")) { + containerItems = parseMotionPhotoV1Directory(xpp); + } + } while (!XmlPullParserUtil.isEndTag(xpp, "x:xmpmeta")); + if (containerItems.isEmpty()) { + // No motion photo information was parsed. + return null; + } + return new MotionPhotoDescription(motionPhotoPresentationTimestampUs, containerItems); + } + + private static boolean parseMotionPhotoFlagFromDescription(XmlPullParser xpp) { + for (String attributeName : MOTION_PHOTO_ATTRIBUTE_NAMES) { + @Nullable String attributeValue = XmlPullParserUtil.getAttributeValue(xpp, attributeName); + if (attributeValue != null) { + int motionPhotoFlag = Integer.parseInt(attributeValue); + return motionPhotoFlag == 1; + } + } + return false; + } + + private static long parseMotionPhotoPresentationTimestampUsFromDescription(XmlPullParser xpp) { + for (String attributeName : DESCRIPTION_MOTION_PHOTO_PRESENTATION_TIMESTAMP_ATTRIBUTE_NAMES) { + @Nullable String attributeValue = XmlPullParserUtil.getAttributeValue(xpp, attributeName); + if (attributeValue != null) { + long presentationTimestampUs = Long.parseLong(attributeValue); + return presentationTimestampUs == -1 ? C.TIME_UNSET : presentationTimestampUs; + } + } + return C.TIME_UNSET; + } + + private static ImmutableList + parseMicroVideoOffsetFromDescription(XmlPullParser xpp) { + // We store a new Motion Photo item list based on the MicroVideo offset, so that the same + // representation is used for both specifications. + for (String attributeName : DESCRIPTION_MICRO_VIDEO_OFFSET_ATTRIBUTE_NAMES) { + @Nullable String attributeValue = XmlPullParserUtil.getAttributeValue(xpp, attributeName); + if (attributeValue != null) { + long microVideoOffset = Long.parseLong(attributeValue); + return ImmutableList.of( + new MotionPhotoDescription.ContainerItem( + MimeTypes.IMAGE_JPEG, "Primary", /* length= */ 0, /* padding= */ 0), + new MotionPhotoDescription.ContainerItem( + MimeTypes.VIDEO_MP4, + "MotionPhoto", + /* length= */ microVideoOffset, + /* padding= */ 0)); + } + } + return ImmutableList.of(); + } + + private static ImmutableList parseMotionPhotoV1Directory( + XmlPullParser xpp) throws XmlPullParserException, IOException { + ImmutableList.Builder containerItems = + ImmutableList.builder(); + do { + xpp.next(); + if (XmlPullParserUtil.isStartTag(xpp, "Container:Item")) { + @Nullable String mime = XmlPullParserUtil.getAttributeValue(xpp, "Item:Mime"); + @Nullable String semantic = XmlPullParserUtil.getAttributeValue(xpp, "Item:Semantic"); + @Nullable String length = XmlPullParserUtil.getAttributeValue(xpp, "Item:Length"); + @Nullable String padding = XmlPullParserUtil.getAttributeValue(xpp, "Item:Padding"); + if (mime == null || semantic == null) { + // Required values are missing. + return ImmutableList.of(); + } + containerItems.add( + new MotionPhotoDescription.ContainerItem( + mime, + semantic, + length != null ? Long.parseLong(length) : 0, + padding != null ? Long.parseLong(padding) : 0)); + } + } while (!XmlPullParserUtil.isEndTag(xpp, "Container:Directory")); + return containerItems.build(); + } + + private XmpMotionPhotoDescriptionParser() {} +} diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/package-info.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/package-info.java new file mode 100644 index 00000000000..7e0522b275f --- /dev/null +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/jpeg/package-info.java @@ -0,0 +1,19 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +@NonNullApi +package com.google.android.exoplayer2.extractor.jpeg; + +import com.google.android.exoplayer2.util.NonNullApi; diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mkv/MatroskaExtractor.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mkv/MatroskaExtractor.java index f4022fe780d..c3f3e5e9019 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mkv/MatroskaExtractor.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mkv/MatroskaExtractor.java @@ -15,6 +15,10 @@ */ package com.google.android.exoplayer2.extractor.mkv; +import static com.google.android.exoplayer2.util.Assertions.checkArgument; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static com.google.android.exoplayer2.util.Assertions.checkStateNotNull; import static java.lang.Math.max; import static java.lang.Math.min; @@ -39,7 +43,6 @@ import com.google.android.exoplayer2.extractor.PositionHolder; import com.google.android.exoplayer2.extractor.SeekMap; import com.google.android.exoplayer2.extractor.TrackOutput; -import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.LongArray; import com.google.android.exoplayer2.util.MimeTypes; @@ -50,6 +53,7 @@ import com.google.android.exoplayer2.video.ColorInfo; import com.google.android.exoplayer2.video.DolbyVisionConfig; import com.google.android.exoplayer2.video.HevcConfig; +import com.google.common.collect.ImmutableList; import java.io.IOException; import java.lang.annotation.Documented; import java.lang.annotation.Retention; @@ -65,7 +69,9 @@ import java.util.Map; import java.util.UUID; import org.checkerframework.checker.nullness.compatqual.NullableType; +import org.checkerframework.checker.nullness.qual.EnsuresNonNull; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.checkerframework.checker.nullness.qual.RequiresNonNull; /** Extracts data from the Matroska and WebM container formats. */ public class MatroskaExtractor implements Extractor { @@ -370,7 +376,7 @@ public class MatroskaExtractor implements Extractor { private final ParsableByteArray encryptionInitializationVector; private final ParsableByteArray encryptionSubsampleData; private final ParsableByteArray blockAdditionalData; - private ByteBuffer encryptionSubsampleDataBuffer; + private @MonotonicNonNull ByteBuffer encryptionSubsampleDataBuffer; private long segmentContentSize; private long segmentContentPosition = C.POSITION_UNSET; @@ -494,7 +500,9 @@ public final int read(ExtractorInput input, PositionHolder seekPosition) throws } if (!continueReading) { for (int i = 0; i < tracks.size(); i++) { - tracks.valueAt(i).outputPendingSampleMetadata(); + Track track = tracks.valueAt(i); + track.assertOutputInitialized(); + track.outputPendingSampleMetadata(); } return Extractor.RESULT_END_OF_INPUT; } @@ -629,6 +637,7 @@ protected boolean isLevel1Element(int id) { @CallSuper protected void startMasterElement(int id, long contentPosition, long contentSize) throws ParserException { + assertInitialized(); switch (id) { case ID_SEGMENT: if (segmentContentPosition != C.POSITION_UNSET @@ -670,13 +679,13 @@ protected void startMasterElement(int id, long contentPosition, long contentSize // TODO: check and fail if more than one content encoding is present. break; case ID_CONTENT_ENCRYPTION: - currentTrack.hasContentEncryption = true; + getCurrentTrack(id).hasContentEncryption = true; break; case ID_TRACK_ENTRY: currentTrack = new Track(); break; case ID_MASTERING_METADATA: - currentTrack.hasColorInfo = true; + getCurrentTrack(id).hasColorInfo = true; break; default: break; @@ -690,6 +699,7 @@ protected void startMasterElement(int id, long contentPosition, long contentSize */ @CallSuper protected void endMasterElement(int id) throws ParserException { + assertInitialized(); switch (id) { case ID_SEGMENT_INFO: if (timecodeScale == C.TIME_UNSET) { @@ -710,11 +720,13 @@ protected void endMasterElement(int id) throws ParserException { break; case ID_CUES: if (!sentSeekMap) { - extractorOutput.seekMap(buildSeekMap()); + extractorOutput.seekMap(buildSeekMap(cueTimesUs, cueClusterPositions)); sentSeekMap = true; } else { // We have already built the cues. Ignore. } + this.cueTimesUs = null; + this.cueClusterPositions = null; break; case ID_BLOCK_GROUP: if (blockState != BLOCK_STATE_DATA) { @@ -727,6 +739,7 @@ protected void endMasterElement(int id) throws ParserException { sampleOffset += blockSampleSizes[i]; } Track track = tracks.get(blockTrackNumber); + track.assertOutputInitialized(); for (int i = 0; i < blockSampleCount; i++) { long sampleTimeUs = blockTimeUs + (i * track.defaultSampleDurationNs) / 1000; int sampleFlags = blockFlags; @@ -742,6 +755,7 @@ protected void endMasterElement(int id) throws ParserException { blockState = BLOCK_STATE_START; break; case ID_CONTENT_ENCODING: + assertInTrackEntry(id); if (currentTrack.hasContentEncryption) { if (currentTrack.cryptoData == null) { throw new ParserException("Encrypted Track found but ContentEncKeyID was not found"); @@ -751,16 +765,22 @@ protected void endMasterElement(int id) throws ParserException { } break; case ID_CONTENT_ENCODINGS: + assertInTrackEntry(id); if (currentTrack.hasContentEncryption && currentTrack.sampleStrippedBytes != null) { throw new ParserException("Combining encryption and compression is not supported"); } break; case ID_TRACK_ENTRY: - if (isCodecSupported(currentTrack.codecId)) { - currentTrack.initializeOutput(extractorOutput, currentTrack.number); - tracks.put(currentTrack.number, currentTrack); + Track currentTrack = checkStateNotNull(this.currentTrack); + if (currentTrack.codecId == null) { + throw new ParserException("CodecId is missing in TrackEntry element"); + } else { + if (isCodecSupported(currentTrack.codecId)) { + currentTrack.initializeOutput(extractorOutput, currentTrack.number); + tracks.put(currentTrack.number, currentTrack); + } } - currentTrack = null; + this.currentTrack = null; break; case ID_TRACKS: if (tracks.size() == 0) { @@ -802,52 +822,52 @@ protected void integerElement(int id, long value) throws ParserException { timecodeScale = value; break; case ID_PIXEL_WIDTH: - currentTrack.width = (int) value; + getCurrentTrack(id).width = (int) value; break; case ID_PIXEL_HEIGHT: - currentTrack.height = (int) value; + getCurrentTrack(id).height = (int) value; break; case ID_DISPLAY_WIDTH: - currentTrack.displayWidth = (int) value; + getCurrentTrack(id).displayWidth = (int) value; break; case ID_DISPLAY_HEIGHT: - currentTrack.displayHeight = (int) value; + getCurrentTrack(id).displayHeight = (int) value; break; case ID_DISPLAY_UNIT: - currentTrack.displayUnit = (int) value; + getCurrentTrack(id).displayUnit = (int) value; break; case ID_TRACK_NUMBER: - currentTrack.number = (int) value; + getCurrentTrack(id).number = (int) value; break; case ID_FLAG_DEFAULT: - currentTrack.flagDefault = value == 1; + getCurrentTrack(id).flagDefault = value == 1; break; case ID_FLAG_FORCED: - currentTrack.flagForced = value == 1; + getCurrentTrack(id).flagForced = value == 1; break; case ID_TRACK_TYPE: - currentTrack.type = (int) value; + getCurrentTrack(id).type = (int) value; break; case ID_DEFAULT_DURATION: - currentTrack.defaultSampleDurationNs = (int) value; + getCurrentTrack(id).defaultSampleDurationNs = (int) value; break; case ID_MAX_BLOCK_ADDITION_ID: - currentTrack.maxBlockAdditionId = (int) value; + getCurrentTrack(id).maxBlockAdditionId = (int) value; break; case ID_BLOCK_ADD_ID_TYPE: - currentTrack.blockAddIdType = (int) value; + getCurrentTrack(id).blockAddIdType = (int) value; break; case ID_CODEC_DELAY: - currentTrack.codecDelayNs = value; + getCurrentTrack(id).codecDelayNs = value; break; case ID_SEEK_PRE_ROLL: - currentTrack.seekPreRollNs = value; + getCurrentTrack(id).seekPreRollNs = value; break; case ID_CHANNELS: - currentTrack.channelCount = (int) value; + getCurrentTrack(id).channelCount = (int) value; break; case ID_AUDIO_BIT_DEPTH: - currentTrack.audioBitDepth = (int) value; + getCurrentTrack(id).audioBitDepth = (int) value; break; case ID_REFERENCE_BLOCK: blockHasReferenceBlock = true; @@ -883,10 +903,12 @@ protected void integerElement(int id, long value) throws ParserException { } break; case ID_CUE_TIME: + assertInCues(id); cueTimesUs.add(scaleTimecodeToUs(value)); break; case ID_CUE_CLUSTER_POSITION: if (!seenClusterPositionForCurrentCuePoint) { + assertInCues(id); // If there's more than one video/audio track, then there could be more than one // CueTrackPositions within a single CuePoint. In such a case, ignore all but the first // one (since the cluster position will be quite close for all the tracks). @@ -902,6 +924,7 @@ protected void integerElement(int id, long value) throws ParserException { break; case ID_STEREO_MODE: int layout = (int) value; + assertInTrackEntry(id); switch (layout) { case 0: currentTrack.stereoMode = C.STEREO_MODE_MONO; @@ -920,6 +943,7 @@ protected void integerElement(int id, long value) throws ParserException { } break; case ID_COLOUR_PRIMARIES: + assertInTrackEntry(id); currentTrack.hasColorInfo = true; switch ((int) value) { case 1: @@ -939,6 +963,7 @@ protected void integerElement(int id, long value) throws ParserException { } break; case ID_COLOUR_TRANSFER: + assertInTrackEntry(id); switch ((int) value) { case 1: // BT.709. case 6: // SMPTE 170M. @@ -956,6 +981,7 @@ protected void integerElement(int id, long value) throws ParserException { } break; case ID_COLOUR_RANGE: + assertInTrackEntry(id); switch((int) value) { case 1: // Broadcast range. currentTrack.colorRange = C.COLOR_RANGE_LIMITED; @@ -968,12 +994,13 @@ protected void integerElement(int id, long value) throws ParserException { } break; case ID_MAX_CLL: - currentTrack.maxContentLuminance = (int) value; + getCurrentTrack(id).maxContentLuminance = (int) value; break; case ID_MAX_FALL: - currentTrack.maxFrameAverageLuminance = (int) value; + getCurrentTrack(id).maxFrameAverageLuminance = (int) value; break; case ID_PROJECTION_TYPE: + assertInTrackEntry(id); switch ((int) value) { case 0: currentTrack.projectionType = C.PROJECTION_RECTANGULAR; @@ -1011,46 +1038,46 @@ protected void floatElement(int id, double value) throws ParserException { durationTimecode = (long) value; break; case ID_SAMPLING_FREQUENCY: - currentTrack.sampleRate = (int) value; + getCurrentTrack(id).sampleRate = (int) value; break; case ID_PRIMARY_R_CHROMATICITY_X: - currentTrack.primaryRChromaticityX = (float) value; + getCurrentTrack(id).primaryRChromaticityX = (float) value; break; case ID_PRIMARY_R_CHROMATICITY_Y: - currentTrack.primaryRChromaticityY = (float) value; + getCurrentTrack(id).primaryRChromaticityY = (float) value; break; case ID_PRIMARY_G_CHROMATICITY_X: - currentTrack.primaryGChromaticityX = (float) value; + getCurrentTrack(id).primaryGChromaticityX = (float) value; break; case ID_PRIMARY_G_CHROMATICITY_Y: - currentTrack.primaryGChromaticityY = (float) value; + getCurrentTrack(id).primaryGChromaticityY = (float) value; break; case ID_PRIMARY_B_CHROMATICITY_X: - currentTrack.primaryBChromaticityX = (float) value; + getCurrentTrack(id).primaryBChromaticityX = (float) value; break; case ID_PRIMARY_B_CHROMATICITY_Y: - currentTrack.primaryBChromaticityY = (float) value; + getCurrentTrack(id).primaryBChromaticityY = (float) value; break; case ID_WHITE_POINT_CHROMATICITY_X: - currentTrack.whitePointChromaticityX = (float) value; + getCurrentTrack(id).whitePointChromaticityX = (float) value; break; case ID_WHITE_POINT_CHROMATICITY_Y: - currentTrack.whitePointChromaticityY = (float) value; + getCurrentTrack(id).whitePointChromaticityY = (float) value; break; case ID_LUMNINANCE_MAX: - currentTrack.maxMasteringLuminance = (float) value; + getCurrentTrack(id).maxMasteringLuminance = (float) value; break; case ID_LUMNINANCE_MIN: - currentTrack.minMasteringLuminance = (float) value; + getCurrentTrack(id).minMasteringLuminance = (float) value; break; case ID_PROJECTION_POSE_YAW: - currentTrack.projectionPoseYaw = (float) value; + getCurrentTrack(id).projectionPoseYaw = (float) value; break; case ID_PROJECTION_POSE_PITCH: - currentTrack.projectionPosePitch = (float) value; + getCurrentTrack(id).projectionPosePitch = (float) value; break; case ID_PROJECTION_POSE_ROLL: - currentTrack.projectionPoseRoll = (float) value; + getCurrentTrack(id).projectionPoseRoll = (float) value; break; default: break; @@ -1072,13 +1099,13 @@ protected void stringElement(int id, String value) throws ParserException { } break; case ID_NAME: - currentTrack.name = value; + getCurrentTrack(id).name = value; break; case ID_CODEC_ID: - currentTrack.codecId = value; + getCurrentTrack(id).codecId = value; break; case ID_LANGUAGE: - currentTrack.language = value; + getCurrentTrack(id).language = value; break; default: break; @@ -1100,17 +1127,20 @@ protected void binaryElement(int id, int contentSize, ExtractorInput input) thro seekEntryId = (int) seekEntryIdBytes.readUnsignedInt(); break; case ID_BLOCK_ADD_ID_EXTRA_DATA: - handleBlockAddIDExtraData(currentTrack, input, contentSize); + handleBlockAddIDExtraData(getCurrentTrack(id), input, contentSize); break; case ID_CODEC_PRIVATE: + assertInTrackEntry(id); currentTrack.codecPrivate = new byte[contentSize]; input.readFully(currentTrack.codecPrivate, 0, contentSize); break; case ID_PROJECTION_PRIVATE: + assertInTrackEntry(id); currentTrack.projectionData = new byte[contentSize]; input.readFully(currentTrack.projectionData, 0, contentSize); break; case ID_CONTENT_COMPRESSION_SETTINGS: + assertInTrackEntry(id); // This extractor only supports header stripping, so the payload is the stripped bytes. currentTrack.sampleStrippedBytes = new byte[contentSize]; input.readFully(currentTrack.sampleStrippedBytes, 0, contentSize); @@ -1118,8 +1148,9 @@ protected void binaryElement(int id, int contentSize, ExtractorInput input) thro case ID_CONTENT_ENCRYPTION_KEY_ID: byte[] encryptionKey = new byte[contentSize]; input.readFully(encryptionKey, 0, contentSize); - currentTrack.cryptoData = new TrackOutput.CryptoData(C.CRYPTO_MODE_AES_CTR, encryptionKey, - 0, 0); // We assume patternless AES-CTR. + getCurrentTrack(id).cryptoData = + new TrackOutput.CryptoData( + C.CRYPTO_MODE_AES_CTR, encryptionKey, 0, 0); // We assume patternless AES-CTR. break; case ID_SIMPLE_BLOCK: case ID_BLOCK: @@ -1145,6 +1176,8 @@ protected void binaryElement(int id, int contentSize, ExtractorInput input) thro return; } + track.assertOutputInitialized(); + if (blockState == BLOCK_STATE_HEADER) { // Read the relative timecode (2 bytes) and flags (1 byte). readScratch(input, 3); @@ -1295,6 +1328,26 @@ protected void handleBlockAdditionalData( } } + @EnsuresNonNull("currentTrack") + private void assertInTrackEntry(int id) throws ParserException { + if (currentTrack == null) { + throw new ParserException("Element " + id + " must be in a TrackEntry"); + } + } + + @EnsuresNonNull({"cueTimesUs", "cueClusterPositions"}) + private void assertInCues(int id) throws ParserException { + if (cueTimesUs == null || cueClusterPositions == null) { + throw new ParserException("Element " + id + " must be in a Cues"); + } + } + + private Track getCurrentTrack(int currentElementId) throws ParserException { + assertInTrackEntry(currentElementId); + return currentTrack; + } + + @RequiresNonNull("#1.output") private void commitSampleToOutput( Track track, long timeUs, @C.BufferFlags int flags, int size, int offset) { if (track.trueHdSampleRechunker != null) { @@ -1350,9 +1403,7 @@ private void readScratch(ExtractorInput input, int requiredLength) throws IOExce return; } if (scratch.capacity() < requiredLength) { - scratch.reset( - Arrays.copyOf(scratch.getData(), max(scratch.getData().length * 2, requiredLength)), - scratch.limit()); + scratch.ensureCapacity(max(scratch.capacity() * 2, requiredLength)); } input.readFully(scratch.getData(), scratch.limit(), requiredLength - scratch.limit()); scratch.setLimit(requiredLength); @@ -1367,6 +1418,7 @@ private void readScratch(ExtractorInput input, int requiredLength) throws IOExce * @return The final size of the written sample. * @throws IOException If an error occurs reading from the input. */ + @RequiresNonNull("#2.output") private int writeSampleData(ExtractorInput input, Track track, int size) throws IOException { if (CODEC_ID_SUBRIP.equals(track.codecId)) { writeSubtitleSampleData(input, SUBRIP_PREFIX, size); @@ -1524,7 +1576,7 @@ private int writeSampleData(ExtractorInput input, Track track, int size) throws } } else { if (track.trueHdSampleRechunker != null) { - Assertions.checkState(sampleStrippedBytes.limit() == 0); + checkState(sampleStrippedBytes.limit() == 0); track.trueHdSampleRechunker.startSample(input); } while (sampleBytesRead < size) { @@ -1584,7 +1636,8 @@ private void writeSubtitleSampleData(ExtractorInput input, byte[] samplePrefix, System.arraycopy(samplePrefix, 0, subtitleSample.getData(), 0, samplePrefix.length); } input.readFully(subtitleSample.getData(), samplePrefix.length, size); - subtitleSample.reset(sizeWithPrefix); + subtitleSample.setPosition(0); + subtitleSample.setLimit(sizeWithPrefix); // Defer writing the data to the track output. We need to modify the sample data by setting // the correct end timecode, which we might not have yet. } @@ -1629,7 +1682,7 @@ private static void setSubtitleEndTime(String codecId, long durationUs, byte[] s */ private static byte[] formatSubtitleTimecode( long timeUs, String timecodeFormat, long lastTimecodeValueScalingFactor) { - Assertions.checkArgument(timeUs != C.TIME_UNSET); + checkArgument(timeUs != C.TIME_UNSET); byte[] timeCodeData; int hours = (int) (timeUs / (3600 * C.MICROS_PER_SECOND)); timeUs -= (hours * 3600 * C.MICROS_PER_SECOND); @@ -1680,13 +1733,12 @@ private int writeToOutput(ExtractorInput input, TrackOutput output, int length) * @return The built {@link SeekMap}. The returned {@link SeekMap} may be unseekable if cues * information was missing or incomplete. */ - private SeekMap buildSeekMap() { + private SeekMap buildSeekMap( + @Nullable LongArray cueTimesUs, @Nullable LongArray cueClusterPositions) { if (segmentContentPosition == C.POSITION_UNSET || durationUs == C.TIME_UNSET || cueTimesUs == null || cueTimesUs.size() == 0 || cueClusterPositions == null || cueClusterPositions.size() != cueTimesUs.size()) { // Cues information is missing or incomplete. - cueTimesUs = null; - cueClusterPositions = null; return new SeekMap.Unseekable(durationUs); } int cuePointsSize = cueTimesUs.size(); @@ -1715,8 +1767,6 @@ private SeekMap buildSeekMap() { timesUs = Arrays.copyOf(timesUs, timesUs.length - 1); } - cueTimesUs = null; - cueClusterPositions = null; return new ChunkIndex(sizes, offsets, durationsUs, timesUs); } @@ -1797,7 +1847,7 @@ private static boolean isCodecSupported(String codecId) { * Returns an array that can store (at least) {@code length} elements, which will be either a new * array or {@code array} if it's not null and large enough. */ - private static int[] ensureArrayCapacity(int[] array, int length) { + private static int[] ensureArrayCapacity(@Nullable int[] array, int length) { if (array == null) { return new int[length]; } else if (array.length >= length) { @@ -1808,6 +1858,11 @@ private static int[] ensureArrayCapacity(int[] array, int length) { } } + @EnsuresNonNull("extractorOutput") + private void assertInitialized() { + checkStateNotNull(extractorOutput); + } + /** Passes events through to the outer {@link MatroskaExtractor}. */ private final class InnerEbmlProcessor implements EbmlProcessor { @@ -1889,6 +1944,7 @@ public void startSample(ExtractorInput input) throws IOException { foundSyncframe = true; } + @RequiresNonNull("#1.output") public void sampleMetadata( Track track, long timeUs, @C.BufferFlags int flags, int size, int offset) { if (!foundSyncframe) { @@ -1907,6 +1963,7 @@ public void sampleMetadata( } } + @RequiresNonNull("#1.output") public void outputPendingSampleMetadata(Track track) { if (chunkSampleCount > 0) { track.output.sampleMetadata( @@ -1931,18 +1988,18 @@ private static final class Track { private static final int DEFAULT_MAX_FALL = 200; // nits. // Common elements. - public String name; - public String codecId; + public @MonotonicNonNull String name; + public @MonotonicNonNull String codecId; public int number; public int type; public int defaultSampleDurationNs; public int maxBlockAdditionId; private int blockAddIdType; public boolean hasContentEncryption; - public byte[] sampleStrippedBytes; - public TrackOutput.CryptoData cryptoData; - public byte[] codecPrivate; - public DrmInitData drmInitData; + public byte @MonotonicNonNull [] sampleStrippedBytes; + public TrackOutput.@MonotonicNonNull CryptoData cryptoData; + public byte @MonotonicNonNull [] codecPrivate; + public @MonotonicNonNull DrmInitData drmInitData; // Video elements. public int width = Format.NO_VALUE; @@ -1954,7 +2011,7 @@ private static final class Track { public float projectionPoseYaw = 0f; public float projectionPosePitch = 0f; public float projectionPoseRoll = 0f; - public byte[] projectionData = null; + public byte @MonotonicNonNull [] projectionData = null; @C.StereoMode public int stereoMode = Format.NO_VALUE; public boolean hasColorInfo = false; @@ -1976,7 +2033,7 @@ private static final class Track { public float whitePointChromaticityY = Format.NO_VALUE; public float maxMasteringLuminance = Format.NO_VALUE; public float minMasteringLuminance = Format.NO_VALUE; - @Nullable public byte[] dolbyVisionConfigBytes; + public byte @MonotonicNonNull [] dolbyVisionConfigBytes; // Audio elements. Initially set to their default values. public int channelCount = 1; @@ -1984,7 +2041,7 @@ private static final class Track { public int sampleRate = 8000; public long codecDelayNs = 0; public long seekPreRollNs = 0; - @Nullable public TrueHdSampleRechunker trueHdSampleRechunker; + public @MonotonicNonNull TrueHdSampleRechunker trueHdSampleRechunker; // Text elements. public boolean flagForced; @@ -1992,10 +2049,12 @@ private static final class Track { private String language = "eng"; // Set when the output is initialized. nalUnitLengthFieldLength is only set for H264/H265. - public TrackOutput output; + public @MonotonicNonNull TrackOutput output; public int nalUnitLengthFieldLength; /** Initializes the track with an output. */ + @RequiresNonNull("codecId") + @EnsuresNonNull("this.output") public void initializeOutput(ExtractorOutput output, int trackId) throws ParserException { String mimeType; int maxInputSize = Format.NO_VALUE; @@ -2024,19 +2083,21 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE break; case CODEC_ID_H264: mimeType = MimeTypes.VIDEO_H264; - AvcConfig avcConfig = AvcConfig.parse(new ParsableByteArray(codecPrivate)); + AvcConfig avcConfig = AvcConfig.parse(new ParsableByteArray(getCodecPrivate(codecId))); initializationData = avcConfig.initializationData; nalUnitLengthFieldLength = avcConfig.nalUnitLengthFieldLength; + codecs = avcConfig.codecs; break; case CODEC_ID_H265: mimeType = MimeTypes.VIDEO_H265; - HevcConfig hevcConfig = HevcConfig.parse(new ParsableByteArray(codecPrivate)); + HevcConfig hevcConfig = HevcConfig.parse(new ParsableByteArray(getCodecPrivate(codecId))); initializationData = hevcConfig.initializationData; nalUnitLengthFieldLength = hevcConfig.nalUnitLengthFieldLength; + codecs = hevcConfig.codecs; break; case CODEC_ID_FOURCC: Pair> pair = - parseFourCcPrivate(new ParsableByteArray(codecPrivate)); + parseFourCcPrivate(new ParsableByteArray(getCodecPrivate(codecId))); mimeType = pair.first; initializationData = pair.second; break; @@ -2048,13 +2109,13 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE case CODEC_ID_VORBIS: mimeType = MimeTypes.AUDIO_VORBIS; maxInputSize = VORBIS_MAX_INPUT_SIZE; - initializationData = parseVorbisCodecPrivate(codecPrivate); + initializationData = parseVorbisCodecPrivate(getCodecPrivate(codecId)); break; case CODEC_ID_OPUS: mimeType = MimeTypes.AUDIO_OPUS; maxInputSize = OPUS_MAX_INPUT_SIZE; initializationData = new ArrayList<>(3); - initializationData.add(codecPrivate); + initializationData.add(getCodecPrivate(codecId)); initializationData.add( ByteBuffer.allocate(8).order(ByteOrder.LITTLE_ENDIAN).putLong(codecDelayNs).array()); initializationData.add( @@ -2062,7 +2123,7 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE break; case CODEC_ID_AAC: mimeType = MimeTypes.AUDIO_AAC; - initializationData = Collections.singletonList(codecPrivate); + initializationData = Collections.singletonList(getCodecPrivate(codecId)); AacUtil.Config aacConfig = AacUtil.parseAudioSpecificConfig(codecPrivate); // Update sampleRate and channelCount from the AudioSpecificConfig initialization data, // which is more reliable. See [Internal: b/10903778]. @@ -2097,11 +2158,11 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE break; case CODEC_ID_FLAC: mimeType = MimeTypes.AUDIO_FLAC; - initializationData = Collections.singletonList(codecPrivate); + initializationData = Collections.singletonList(getCodecPrivate(codecId)); break; case CODEC_ID_ACM: mimeType = MimeTypes.AUDIO_RAW; - if (parseMsAcmCodecPrivate(new ParsableByteArray(codecPrivate))) { + if (parseMsAcmCodecPrivate(new ParsableByteArray(getCodecPrivate(codecId)))) { pcmEncoding = Util.getPcmEncoding(audioBitDepth); if (pcmEncoding == C.ENCODING_INVALID) { pcmEncoding = Format.NO_VALUE; @@ -2165,10 +2226,11 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE break; case CODEC_ID_ASS: mimeType = MimeTypes.TEXT_SSA; + initializationData = ImmutableList.of(SSA_DIALOGUE_FORMAT, getCodecPrivate(codecId)); break; case CODEC_ID_VOBSUB: mimeType = MimeTypes.APPLICATION_VOBSUB; - initializationData = Collections.singletonList(codecPrivate); + initializationData = ImmutableList.of(getCodecPrivate(codecId)); break; case CODEC_ID_PGS: mimeType = MimeTypes.APPLICATION_PGS; @@ -2176,8 +2238,9 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE case CODEC_ID_DVBSUB: mimeType = MimeTypes.APPLICATION_DVBSUBS; // Init data: composition_page (2), ancillary_page (2) - initializationData = Collections.singletonList(new byte[] {codecPrivate[0], - codecPrivate[1], codecPrivate[2], codecPrivate[3]}); + byte[] initializationDataBytes = new byte[4]; + System.arraycopy(getCodecPrivate(codecId), 0, initializationDataBytes, 0, 4); + initializationData = ImmutableList.of(initializationDataBytes); break; default: throw new ParserException("Unrecognized codec identifier."); @@ -2224,7 +2287,7 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE } int rotationDegrees = Format.NO_VALUE; - if (TRACK_NAME_TO_ROTATION_DEGREES.containsKey(name)) { + if (name != null && TRACK_NAME_TO_ROTATION_DEGREES.containsKey(name)) { rotationDegrees = TRACK_NAME_TO_ROTATION_DEGREES.get(name); } if (projectionType == C.PROJECTION_RECTANGULAR @@ -2250,14 +2313,9 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE .setProjectionData(projectionData) .setStereoMode(stereoMode) .setColorInfo(colorInfo); - } else if (MimeTypes.APPLICATION_SUBRIP.equals(mimeType)) { - type = C.TRACK_TYPE_TEXT; - } else if (MimeTypes.TEXT_SSA.equals(mimeType)) { - type = C.TRACK_TYPE_TEXT; - initializationData = new ArrayList<>(2); - initializationData.add(SSA_DIALOGUE_FORMAT); - initializationData.add(codecPrivate); - } else if (MimeTypes.APPLICATION_VOBSUB.equals(mimeType) + } else if (MimeTypes.APPLICATION_SUBRIP.equals(mimeType) + || MimeTypes.TEXT_SSA.equals(mimeType) + || MimeTypes.APPLICATION_VOBSUB.equals(mimeType) || MimeTypes.APPLICATION_PGS.equals(mimeType) || MimeTypes.APPLICATION_DVBSUBS.equals(mimeType)) { type = C.TRACK_TYPE_TEXT; @@ -2265,7 +2323,7 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE throw new ParserException("Unexpected MIME type."); } - if (!TRACK_NAME_TO_ROTATION_DEGREES.containsKey(name)) { + if (name != null && !TRACK_NAME_TO_ROTATION_DEGREES.containsKey(name)) { formatBuilder.setLabel(name); } @@ -2286,6 +2344,7 @@ public void initializeOutput(ExtractorOutput output, int trackId) throws ParserE } /** Forces any pending sample metadata to be flushed to the output. */ + @RequiresNonNull("output") public void outputPendingSampleMetadata() { if (trueHdSampleRechunker != null) { trueHdSampleRechunker.outputPendingSampleMetadata(this); @@ -2386,18 +2445,18 @@ private static List parseVorbisCodecPrivate(byte[] codecPrivate) } int offset = 1; int vorbisInfoLength = 0; - while (codecPrivate[offset] == (byte) 0xFF) { + while ((codecPrivate[offset] & 0xFF) == 0xFF) { vorbisInfoLength += 0xFF; offset++; } - vorbisInfoLength += codecPrivate[offset++]; + vorbisInfoLength += codecPrivate[offset++] & 0xFF; int vorbisSkipLength = 0; - while (codecPrivate[offset] == (byte) 0xFF) { + while ((codecPrivate[offset] & 0xFF) == 0xFF) { vorbisSkipLength += 0xFF; offset++; } - vorbisSkipLength += codecPrivate[offset++]; + vorbisSkipLength += codecPrivate[offset++] & 0xFF; if (codecPrivate[offset] != 0x01) { throw new ParserException("Error parsing vorbis codec private"); @@ -2445,5 +2504,26 @@ private static boolean parseMsAcmCodecPrivate(ParsableByteArray buffer) throws P throw new ParserException("Error parsing MS/ACM codec private"); } } + + /** + * Checks that the track has an output. + * + *

    It is unfortunately not possible to mark {@link MatroskaExtractor#tracks} as only + * containing tracks with output with the nullness checker. This method is used to check that + * fact at runtime. + */ + @EnsuresNonNull("output") + private void assertOutputInitialized() { + checkNotNull(output); + } + + @EnsuresNonNull("codecPrivate") + private byte[] getCodecPrivate(String codecId) throws ParserException { + if (codecPrivate == null) { + throw new ParserException("Missing CodecPrivate for codec " + codecId); + } + return codecPrivate; + } } + } diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Atom.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Atom.java index 6e880b905ff..95cd1e2c179 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Atom.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Atom.java @@ -181,6 +181,9 @@ @SuppressWarnings("ConstantCaseForConstants") public static final int TYPE_moov = 0x6d6f6f76; + @SuppressWarnings("ConstantCaseForConstants") + public static final int TYPE_mpvd = 0x6d707664; + @SuppressWarnings("ConstantCaseForConstants") public static final int TYPE_mvhd = 0x6d766864; @@ -331,6 +334,12 @@ @SuppressWarnings("ConstantCaseForConstants") public static final int TYPE_meta = 0x6d657461; + @SuppressWarnings("ConstantCaseForConstants") + public static final int TYPE_smta = 0x736d7461; + + @SuppressWarnings("ConstantCaseForConstants") + public static final int TYPE_saut = 0x73617574; + @SuppressWarnings("ConstantCaseForConstants") public static final int TYPE_keys = 0x6b657973; diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/AtomParsers.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/AtomParsers.java index 551ebc3ea3d..162294f1a06 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/AtomParsers.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/AtomParsers.java @@ -31,6 +31,7 @@ import com.google.android.exoplayer2.drm.DrmInitData; import com.google.android.exoplayer2.extractor.GaplessInfoHolder; import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.mp4.SmtaMetadataEntry; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.CodecSpecificDataUtil; import com.google.android.exoplayer2.util.Log; @@ -145,28 +146,30 @@ public static List parseTraks( * * @param udtaAtom The udta (user data) atom to decode. * @param isQuickTime True for QuickTime media. False otherwise. - * @return Parsed metadata, or null. + * @return A {@link Pair} containing the metadata from the meta child atom as first value (if + * any), and the metadata from the smta child atom as second value (if any). */ - @Nullable - public static Metadata parseUdta(Atom.LeafAtom udtaAtom, boolean isQuickTime) { - if (isQuickTime) { - // Meta boxes are regular boxes rather than full boxes in QuickTime. For now, don't try and - // decode one. - return null; - } + public static Pair<@NullableType Metadata, @NullableType Metadata> parseUdta( + Atom.LeafAtom udtaAtom, boolean isQuickTime) { ParsableByteArray udtaData = udtaAtom.data; udtaData.setPosition(Atom.HEADER_SIZE); + @Nullable Metadata metaMetadata = null; + @Nullable Metadata smtaMetadata = null; while (udtaData.bytesLeft() >= Atom.HEADER_SIZE) { int atomPosition = udtaData.getPosition(); int atomSize = udtaData.readInt(); int atomType = udtaData.readInt(); - if (atomType == Atom.TYPE_meta) { + // Meta boxes are regular boxes rather than full boxes in QuickTime. Ignore them for now. + if (atomType == Atom.TYPE_meta && !isQuickTime) { udtaData.setPosition(atomPosition); - return parseUdtaMeta(udtaData, atomPosition + atomSize); + metaMetadata = parseUdtaMeta(udtaData, atomPosition + atomSize); + } else if (atomType == Atom.TYPE_smta) { + udtaData.setPosition(atomPosition); + smtaMetadata = parseSmta(udtaData, atomPosition + atomSize); } udtaData.setPosition(atomPosition + atomSize); } - return null; + return Pair.create(metaMetadata, smtaMetadata); } /** @@ -312,7 +315,7 @@ private static TrackSampleTable parseStbl( SampleSizeBox sampleSizeBox; @Nullable Atom.LeafAtom stszAtom = stblAtom.getLeafAtomOfType(Atom.TYPE_stsz); if (stszAtom != null) { - sampleSizeBox = new StszSampleSizeBox(stszAtom); + sampleSizeBox = new StszSampleSizeBox(stszAtom, track.format); } else { @Nullable Atom.LeafAtom stz2Atom = stblAtom.getLeafAtomOfType(Atom.TYPE_stz2); if (stz2Atom == null) { @@ -701,6 +704,37 @@ private static Metadata parseIlst(ParsableByteArray ilst, int limit) { return entries.isEmpty() ? null : new Metadata(entries); } + /** + * Parses metadata from a Samsung smta atom. + * + *

    See [Internal: b/150138465#comment76]. + */ + @Nullable + private static Metadata parseSmta(ParsableByteArray smta, int limit) { + smta.skipBytes(Atom.FULL_HEADER_SIZE); + while (smta.getPosition() < limit) { + int atomPosition = smta.getPosition(); + int atomSize = smta.readInt(); + int atomType = smta.readInt(); + if (atomType == Atom.TYPE_saut) { + if (atomSize < 14) { + return null; + } + smta.skipBytes(5); // author (4), reserved = 0 (1). + int recordingMode = smta.readUnsignedByte(); + if (recordingMode != 12 && recordingMode != 13) { + return null; + } + float captureFrameRate = recordingMode == 12 ? 240 : 120; + smta.skipBytes(1); // reserved = 1 (1). + int svcTemporalLayerCount = smta.readUnsignedByte(); + return new Metadata(new SmtaMetadataEntry(captureFrameRate, svcTemporalLayerCount)); + } + smta.setPosition(atomPosition + atomSize); + } + return null; + } + /** * Parses a mvhd atom (defined in ISO/IEC 14496-12), returning the timescale for the movie. * @@ -1024,6 +1058,7 @@ private static void parseVideoSampleEntry( if (!pixelWidthHeightRatioFromPasp) { pixelWidthHeightRatio = avcConfig.pixelWidthAspectRatio; } + codecs = avcConfig.codecs; } else if (childAtomType == Atom.TYPE_hvcC) { Assertions.checkState(mimeType == null); mimeType = MimeTypes.VIDEO_H265; @@ -1031,6 +1066,7 @@ private static void parseVideoSampleEntry( HevcConfig hevcConfig = HevcConfig.parse(parent); initializationData = hevcConfig.initializationData; out.nalUnitLengthFieldLength = hevcConfig.nalUnitLengthFieldLength; + codecs = hevcConfig.codecs; } else if (childAtomType == Atom.TYPE_dvcC || childAtomType == Atom.TYPE_dvvC) { @Nullable DolbyVisionConfig dolbyVisionConfig = DolbyVisionConfig.parse(parent); if (dolbyVisionConfig != null) { @@ -1684,10 +1720,25 @@ private interface SampleSizeBox { private final int sampleCount; private final ParsableByteArray data; - public StszSampleSizeBox(Atom.LeafAtom stszAtom) { + public StszSampleSizeBox(Atom.LeafAtom stszAtom, Format trackFormat) { data = stszAtom.data; data.setPosition(Atom.FULL_HEADER_SIZE); int fixedSampleSize = data.readUnsignedIntToInt(); + if (MimeTypes.AUDIO_RAW.equals(trackFormat.sampleMimeType)) { + int pcmFrameSize = Util.getPcmFrameSize(trackFormat.pcmEncoding, trackFormat.channelCount); + if (fixedSampleSize == 0 || fixedSampleSize % pcmFrameSize != 0) { + // The sample size from the stsz box is inconsistent with the PCM encoding and channel + // count derived from the stsd box. Choose stsd box as source of truth + // [Internal ref: b/171627904]. + Log.w( + TAG, + "Audio sample size mismatch. stsd sample size: " + + pcmFrameSize + + ", stsz sample size: " + + fixedSampleSize); + fixedSampleSize = pcmFrameSize; + } + } this.fixedSampleSize = fixedSampleSize == 0 ? C.LENGTH_UNSET : fixedSampleSize; sampleCount = data.readUnsignedIntToInt(); } diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4Extractor.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4Extractor.java index 859ce49b26f..e14381c5640 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4Extractor.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4Extractor.java @@ -74,8 +74,7 @@ public class FragmentedMp4Extractor implements Extractor { /** * Flags controlling the behavior of the extractor. Possible flag values are {@link * #FLAG_WORKAROUND_EVERY_VIDEO_FRAME_IS_SYNC_FRAME}, {@link #FLAG_WORKAROUND_IGNORE_TFDT_BOX}, - * {@link #FLAG_ENABLE_EMSG_TRACK}, {@link #FLAG_SIDELOADED} and {@link - * #FLAG_WORKAROUND_IGNORE_EDIT_LISTS}. + * {@link #FLAG_ENABLE_EMSG_TRACK} and {@link #FLAG_WORKAROUND_IGNORE_EDIT_LISTS}. */ @Documented @Retention(RetentionPolicy.SOURCE) @@ -85,7 +84,6 @@ public class FragmentedMp4Extractor implements Extractor { FLAG_WORKAROUND_EVERY_VIDEO_FRAME_IS_SYNC_FRAME, FLAG_WORKAROUND_IGNORE_TFDT_BOX, FLAG_ENABLE_EMSG_TRACK, - FLAG_SIDELOADED, FLAG_WORKAROUND_IGNORE_EDIT_LISTS }) public @interface Flags {} @@ -104,11 +102,7 @@ public class FragmentedMp4Extractor implements Extractor { * messages in the stream will be delivered as samples to this track. */ public static final int FLAG_ENABLE_EMSG_TRACK = 1 << 2; // 4 - /** - * Flag to indicate that the {@link Track} was sideloaded, instead of being declared by the MP4 - * container. - */ - private static final int FLAG_SIDELOADED = 1 << 3; // 8 + /** Flag to ignore any edit lists in the stream. */ public static final int FLAG_WORKAROUND_IGNORE_EDIT_LISTS = 1 << 4; // 16 @@ -254,7 +248,7 @@ public FragmentedMp4Extractor( @Nullable Track sideloadedTrack, List closedCaptionFormats, @Nullable TrackOutput additionalEmsgTrackOutput) { - this.flags = flags | (sideloadedTrack != null ? FLAG_SIDELOADED : 0); + this.flags = flags; this.timestampAdjuster = timestampAdjuster; this.sideloadedTrack = sideloadedTrack; this.closedCaptionFormats = Collections.unmodifiableList(closedCaptionFormats); @@ -926,7 +920,7 @@ private static TrackBundle parseTfhd( SparseArray trackBundles, int trackId) { if (trackBundles.size() == 1) { // Ignore track id if there is only one track. This is either because we have a side-loaded - // track (flag FLAG_SIDELOADED) or to cope with non-matching track indices (see + // track or to cope with non-matching track indices (see // https://github.com/google/ExoPlayer/issues/4083). return trackBundles.valueAt(/* index= */ 0); } @@ -1051,7 +1045,7 @@ private static int parseTrun( private static int checkNonNegative(int value) throws ParserException { if (value < 0) { - throw new ParserException("Unexpected negtive value: " + value); + throw new ParserException("Unexpected negative value: " + value); } return value; } @@ -1659,7 +1653,7 @@ public int getCurrentSampleSize() { : fragment.sampleSizeTable[currentSampleIndex]; } - /** Returns the {@link C.BufferFlags} corresponding to the the current sample. */ + /** Returns the {@link C.BufferFlags} corresponding to the current sample. */ @C.BufferFlags public int getCurrentSampleFlags() { int flags = diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/MetadataUtil.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/MetadataUtil.java index d29b54a5e58..53ed2811527 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/MetadataUtil.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/MetadataUtil.java @@ -28,8 +28,10 @@ import com.google.android.exoplayer2.metadata.id3.Id3Frame; import com.google.android.exoplayer2.metadata.id3.InternalFrame; import com.google.android.exoplayer2.metadata.id3.TextInformationFrame; +import com.google.android.exoplayer2.metadata.mp4.MdtaMetadataEntry; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.ParsableByteArray; +import org.checkerframework.checker.nullness.compatqual.NullableType; /** Utilities for handling metadata in MP4. */ /* package */ final class MetadataUtil { @@ -282,39 +284,57 @@ private static final int TYPE_TOP_BYTE_COPYRIGHT = 0xA9; private static final int TYPE_TOP_BYTE_REPLACEMENT = 0xFD; // Truncated value of \uFFFD. - private static final String MDTA_KEY_ANDROID_CAPTURE_FPS = "com.android.capture.fps"; - private MetadataUtil() {} /** Updates a {@link Format.Builder} to include metadata from the provided sources. */ public static void setFormatMetadata( int trackType, - @Nullable Metadata udtaMetadata, + @Nullable Metadata udtaMetaMetadata, @Nullable Metadata mdtaMetadata, - GaplessInfoHolder gaplessInfoHolder, - Format.Builder formatBuilder) { + Format.Builder formatBuilder, + @NullableType Metadata... additionalMetadata) { + Metadata formatMetadata = new Metadata(); + if (trackType == C.TRACK_TYPE_AUDIO) { - if (gaplessInfoHolder.hasGaplessInfo()) { - formatBuilder - .setEncoderDelay(gaplessInfoHolder.encoderDelay) - .setEncoderPadding(gaplessInfoHolder.encoderPadding); - } - // We assume all udta metadata is associated with the audio track. - if (udtaMetadata != null) { - formatBuilder.setMetadata(udtaMetadata); + // We assume all meta metadata in the udta box is associated with the audio track. + if (udtaMetaMetadata != null) { + formatMetadata = udtaMetaMetadata; } - } else if (trackType == C.TRACK_TYPE_VIDEO && mdtaMetadata != null) { + } else if (trackType == C.TRACK_TYPE_VIDEO) { // Populate only metadata keys that are known to be specific to video. - for (int i = 0; i < mdtaMetadata.length(); i++) { - Metadata.Entry entry = mdtaMetadata.get(i); - if (entry instanceof MdtaMetadataEntry) { - MdtaMetadataEntry mdtaMetadataEntry = (MdtaMetadataEntry) entry; - if (MDTA_KEY_ANDROID_CAPTURE_FPS.equals(mdtaMetadataEntry.key)) { - formatBuilder.setMetadata(new Metadata(mdtaMetadataEntry)); + if (mdtaMetadata != null) { + for (int i = 0; i < mdtaMetadata.length(); i++) { + Metadata.Entry entry = mdtaMetadata.get(i); + if (entry instanceof MdtaMetadataEntry) { + MdtaMetadataEntry mdtaMetadataEntry = (MdtaMetadataEntry) entry; + if (MdtaMetadataEntry.KEY_ANDROID_CAPTURE_FPS.equals(mdtaMetadataEntry.key)) { + formatMetadata = new Metadata(mdtaMetadataEntry); + break; + } } } } } + + for (Metadata metadata : additionalMetadata) { + formatMetadata = formatMetadata.copyWithAppendedEntriesFrom(metadata); + } + + if (formatMetadata.length() > 0) { + formatBuilder.setMetadata(formatMetadata); + } + } + + /** + * Updates a {@link Format.Builder} to include audio gapless information from the provided source. + */ + public static void setFormatGaplessInfo( + int trackType, GaplessInfoHolder gaplessInfoHolder, Format.Builder formatBuilder) { + if (trackType == C.TRACK_TYPE_AUDIO && gaplessInfoHolder.hasGaplessInfo()) { + formatBuilder + .setEncoderDelay(gaplessInfoHolder.encoderDelay) + .setEncoderPadding(gaplessInfoHolder.encoderPadding); + } } /** diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Mp4Extractor.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Mp4Extractor.java index f9e70915bca..823fd77fe0c 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Mp4Extractor.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Mp4Extractor.java @@ -16,11 +16,14 @@ package com.google.android.exoplayer2.extractor.mp4; import static com.google.android.exoplayer2.extractor.mp4.AtomParsers.parseTraks; +import static com.google.android.exoplayer2.extractor.mp4.Sniffer.BRAND_HEIC; +import static com.google.android.exoplayer2.extractor.mp4.Sniffer.BRAND_QUICKTIME; import static com.google.android.exoplayer2.util.Assertions.checkNotNull; import static com.google.android.exoplayer2.util.Util.castNonNull; import static java.lang.Math.max; import static java.lang.Math.min; +import android.util.Pair; import androidx.annotation.IntDef; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; @@ -38,6 +41,8 @@ import com.google.android.exoplayer2.extractor.TrackOutput; import com.google.android.exoplayer2.extractor.mp4.Atom.ContainerAtom; import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.mp4.MotionPhotoMetadata; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.NalUnitUtil; @@ -49,6 +54,7 @@ import java.util.ArrayDeque; import java.util.ArrayList; import java.util.List; +import org.checkerframework.checker.nullness.compatqual.NullableType; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; import org.checkerframework.checker.nullness.qual.RequiresNonNull; @@ -61,32 +67,61 @@ public final class Mp4Extractor implements Extractor, SeekMap { public static final ExtractorsFactory FACTORY = () -> new Extractor[] {new Mp4Extractor()}; /** - * Flags controlling the behavior of the extractor. Possible flag value is {@link - * #FLAG_WORKAROUND_IGNORE_EDIT_LISTS}. + * Flags controlling the behavior of the extractor. Possible flag values are {@link + * #FLAG_WORKAROUND_IGNORE_EDIT_LISTS}, {@link #FLAG_READ_MOTION_PHOTO_METADATA} and {@link + * #FLAG_READ_SEF_DATA}. */ @Documented @Retention(RetentionPolicy.SOURCE) @IntDef( flag = true, - value = {FLAG_WORKAROUND_IGNORE_EDIT_LISTS}) + value = { + FLAG_WORKAROUND_IGNORE_EDIT_LISTS, + FLAG_READ_MOTION_PHOTO_METADATA, + FLAG_READ_SEF_DATA + }) public @interface Flags {} + /** Flag to ignore any edit lists in the stream. */ + public static final int FLAG_WORKAROUND_IGNORE_EDIT_LISTS = 1; /** - * Flag to ignore any edit lists in the stream. + * Flag to extract {@link MotionPhotoMetadata} from HEIC motion photos following the Google Photos + * Motion Photo File Format V1.1. + * + *

    As playback is not supported for motion photos, this flag should only be used for metadata + * retrieval use cases. */ - public static final int FLAG_WORKAROUND_IGNORE_EDIT_LISTS = 1; + public static final int FLAG_READ_MOTION_PHOTO_METADATA = 1 << 1; + /** + * Flag to extract {@link SlowMotionData} metadata from Samsung Extension Format (SEF) slow motion + * videos. + */ + public static final int FLAG_READ_SEF_DATA = 1 << 2; /** Parser states. */ @Documented @Retention(RetentionPolicy.SOURCE) - @IntDef({STATE_READING_ATOM_HEADER, STATE_READING_ATOM_PAYLOAD, STATE_READING_SAMPLE}) + @IntDef({ + STATE_READING_ATOM_HEADER, + STATE_READING_ATOM_PAYLOAD, + STATE_READING_SAMPLE, + STATE_READING_SEF, + }) private @interface State {} private static final int STATE_READING_ATOM_HEADER = 0; private static final int STATE_READING_ATOM_PAYLOAD = 1; private static final int STATE_READING_SAMPLE = 2; + private static final int STATE_READING_SEF = 3; + + /** Supported file types. */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({FILE_TYPE_MP4, FILE_TYPE_QUICKTIME, FILE_TYPE_HEIC}) + private @interface FileType {} - /** Brand stored in the ftyp atom for QuickTime media. */ - private static final int BRAND_QUICKTIME = 0x71742020; + private static final int FILE_TYPE_MP4 = 0; + private static final int FILE_TYPE_QUICKTIME = 1; + private static final int FILE_TYPE_HEIC = 2; /** * When seeking within the source, if the offset is greater than or equal to this value (or the @@ -109,6 +144,8 @@ public final class Mp4Extractor implements Extractor, SeekMap { private final ParsableByteArray atomHeader; private final ArrayDeque containerAtoms; + private final SefReader sefReader; + private final List slowMotionMetadataEntries; @State private int parserState; private int atomType; @@ -124,16 +161,18 @@ public final class Mp4Extractor implements Extractor, SeekMap { // Extractor outputs. private @MonotonicNonNull ExtractorOutput extractorOutput; private Mp4Track @MonotonicNonNull [] tracks; + private long @MonotonicNonNull [][] accumulatedSampleSizes; private int firstVideoTrackIndex; private long durationUs; - private boolean isQuickTime; + @FileType private int fileType; + @Nullable private MotionPhotoMetadata motionPhotoMetadata; /** * Creates a new extractor for unfragmented MP4 streams. */ public Mp4Extractor() { - this(0); + this(/* flags= */ 0); } /** @@ -144,6 +183,10 @@ public Mp4Extractor() { */ public Mp4Extractor(@Flags int flags) { this.flags = flags; + parserState = + ((flags & FLAG_READ_SEF_DATA) != 0) ? STATE_READING_SEF : STATE_READING_ATOM_HEADER; + sefReader = new SefReader(); + slowMotionMetadataEntries = new ArrayList<>(); atomHeader = new ParsableByteArray(Atom.LONG_HEADER_SIZE); containerAtoms = new ArrayDeque<>(); nalStartCode = new ParsableByteArray(NalUnitUtil.NAL_START_CODE); @@ -154,7 +197,8 @@ public Mp4Extractor(@Flags int flags) { @Override public boolean sniff(ExtractorInput input) throws IOException { - return Sniffer.sniffUnfragmented(input); + return Sniffer.sniffUnfragmented( + input, /* acceptHeic= */ (flags & FLAG_READ_MOTION_PHOTO_METADATA) != 0); } @Override @@ -171,7 +215,14 @@ public void seek(long position, long timeUs) { sampleBytesWritten = 0; sampleCurrentNalBytesRemaining = 0; if (position == 0) { - enterReadingAtomHeaderState(); + // Reading the SEF data occurs before normal MP4 parsing. Therefore we can not transition to + // reading the atom header until that has completed. + if (parserState != STATE_READING_SEF) { + enterReadingAtomHeaderState(); + } else { + sefReader.reset(); + slowMotionMetadataEntries.clear(); + } } else if (tracks != null) { updateSampleIndices(timeUs); } @@ -198,6 +249,8 @@ public int read(ExtractorInput input, PositionHolder seekPosition) throws IOExce break; case STATE_READING_SAMPLE: return readSample(input, seekPosition); + case STATE_READING_SEF: + return readSefData(input, seekPosition); default: throw new IllegalStateException(); } @@ -280,6 +333,7 @@ private boolean readAtomHeader(ExtractorInput input) throws IOException { if (atomHeaderBytesRead == 0) { // Read the standard length atom header. if (!input.readFully(atomHeader.getData(), 0, Atom.HEADER_SIZE, true)) { + processEndOfStreamReadingAtomHeader(); return false; } atomHeaderBytesRead = Atom.HEADER_SIZE; @@ -335,6 +389,7 @@ private boolean readAtomHeader(ExtractorInput input) throws IOException { this.atomData = atomData; parserState = STATE_READING_ATOM_PAYLOAD; } else { + processUnparsedAtom(input.getPosition() - atomHeaderBytesRead); atomData = null; parserState = STATE_READING_ATOM_PAYLOAD; } @@ -356,7 +411,7 @@ private boolean readAtomPayload(ExtractorInput input, PositionHolder positionHol if (atomData != null) { input.readFully(atomData.getData(), atomHeaderBytesRead, (int) atomPayloadSize); if (atomType == Atom.TYPE_ftyp) { - isQuickTime = processFtypAtom(atomData); + fileType = processFtypAtom(atomData); } else if (!containerAtoms.isEmpty()) { containerAtoms.peek().add(new Atom.LeafAtom(atomType, atomData)); } @@ -373,6 +428,15 @@ private boolean readAtomPayload(ExtractorInput input, PositionHolder positionHol return seekRequired && parserState != STATE_READING_SAMPLE; } + @ReadResult + private int readSefData(ExtractorInput input, PositionHolder seekPosition) throws IOException { + @ReadResult int result = sefReader.read(input, seekPosition, slowMotionMetadataEntries); + if (result == RESULT_SEEK && seekPosition.position == 0) { + enterReadingAtomHeaderState(); + } + return result; + } + private void processAtomEnded(long atomEndPosition) throws ParserException { while (!containerAtoms.isEmpty() && containerAtoms.peek().endPosition == atomEndPosition) { Atom.ContainerAtom containerAtom = containerAtoms.pop(); @@ -399,13 +463,18 @@ private void processMoovAtom(ContainerAtom moov) throws ParserException { List tracks = new ArrayList<>(); // Process metadata. - @Nullable Metadata udtaMetadata = null; + @Nullable Metadata udtaMetaMetadata = null; + @Nullable Metadata smtaMetadata = null; + boolean isQuickTime = fileType == FILE_TYPE_QUICKTIME; GaplessInfoHolder gaplessInfoHolder = new GaplessInfoHolder(); @Nullable Atom.LeafAtom udta = moov.getLeafAtomOfType(Atom.TYPE_udta); if (udta != null) { - udtaMetadata = AtomParsers.parseUdta(udta, isQuickTime); - if (udtaMetadata != null) { - gaplessInfoHolder.setFromMetadata(udtaMetadata); + Pair<@NullableType Metadata, @NullableType Metadata> udtaMetadata = + AtomParsers.parseUdta(udta, isQuickTime); + udtaMetaMetadata = udtaMetadata.first; + smtaMetadata = udtaMetadata.second; + if (udtaMetaMetadata != null) { + gaplessInfoHolder.setFromMetadata(udtaMetaMetadata); } } @Nullable Metadata mdtaMetadata = null; @@ -450,8 +519,15 @@ private void processMoovAtom(ContainerAtom moov) throws ParserException { float frameRate = trackSampleTable.sampleCount / (trackDurationUs / 1000000f); formatBuilder.setFrameRate(frameRate); } + + MetadataUtil.setFormatGaplessInfo(track.type, gaplessInfoHolder, formatBuilder); MetadataUtil.setFormatMetadata( - track.type, udtaMetadata, mdtaMetadata, gaplessInfoHolder, formatBuilder); + track.type, + udtaMetaMetadata, + mdtaMetadata, + formatBuilder, + smtaMetadata, + slowMotionMetadataEntries.isEmpty() ? null : new Metadata(slowMotionMetadataEntries)); mp4Track.trackOutput.format(formatBuilder.build()); if (track.type == C.TRACK_TYPE_VIDEO && firstVideoTrackIndex == C.INDEX_UNSET) { @@ -637,6 +713,20 @@ private void updateSampleIndices(long timeUs) { } } + /** Processes the end of stream in case there is not atom left to read. */ + private void processEndOfStreamReadingAtomHeader() { + if (fileType == FILE_TYPE_HEIC && (flags & FLAG_READ_MOTION_PHOTO_METADATA) != 0) { + // Add image track and prepare media. + ExtractorOutput extractorOutput = checkNotNull(this.extractorOutput); + TrackOutput trackOutput = extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_IMAGE); + @Nullable + Metadata metadata = motionPhotoMetadata == null ? null : new Metadata(motionPhotoMetadata); + trackOutput.format(new Format.Builder().setMetadata(metadata).build()); + extractorOutput.endTracks(); + extractorOutput.seekMap(new SeekMap.Unseekable(/* durationUs= */ C.TIME_UNSET)); + } + } + /** * Possibly skips the version and flags fields (1+3 byte) of a full meta atom of the {@code * input}. @@ -662,6 +752,21 @@ private void maybeSkipRemainingMetaAtomHeaderBytes(ExtractorInput input) throws } } + /** Processes an atom whose payload does not need to be parsed. */ + private void processUnparsedAtom(long atomStartPosition) { + if (atomType == Atom.TYPE_mpvd) { + // The input is an HEIC motion photo following the Google Photos Motion Photo File Format + // V1.1. + motionPhotoMetadata = + new MotionPhotoMetadata( + /* photoStartPosition= */ 0, + /* photoSize= */ atomStartPosition, + /* photoPresentationTimestampUs= */ C.TIME_UNSET, + /* videoStartPosition= */ atomStartPosition + atomHeaderBytesRead, + /* videoSize= */ atomSize - atomHeaderBytesRead); + } + } + /** * For each sample of each track, calculates accumulated size of all samples which need to be read * before this sample can be used. @@ -741,24 +846,39 @@ private static int getSynchronizationSampleIndex(TrackSampleTable sampleTable, l } /** - * Process an ftyp atom to determine whether the media is QuickTime. + * Process an ftyp atom to determine the corresponding {@link FileType}. * * @param atomData The ftyp atom data. - * @return Whether the media is QuickTime. + * @return The {@link FileType}. */ - private static boolean processFtypAtom(ParsableByteArray atomData) { + @FileType + private static int processFtypAtom(ParsableByteArray atomData) { atomData.setPosition(Atom.HEADER_SIZE); int majorBrand = atomData.readInt(); - if (majorBrand == BRAND_QUICKTIME) { - return true; + @FileType int fileType = brandToFileType(majorBrand); + if (fileType != FILE_TYPE_MP4) { + return fileType; } atomData.skipBytes(4); // minor_version while (atomData.bytesLeft() > 0) { - if (atomData.readInt() == BRAND_QUICKTIME) { - return true; + fileType = brandToFileType(atomData.readInt()); + if (fileType != FILE_TYPE_MP4) { + return fileType; } } - return false; + return FILE_TYPE_MP4; + } + + @FileType + private static int brandToFileType(int brand) { + switch (brand) { + case BRAND_QUICKTIME: + return FILE_TYPE_QUICKTIME; + case BRAND_HEIC: + return FILE_TYPE_HEIC; + default: + return FILE_TYPE_MP4; + } } /** Returns whether the extractor should decode a leaf atom with type {@code atom}. */ diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/PsshAtomUtil.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/PsshAtomUtil.java index b4f537f0ce3..fb94fb9ed2b 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/PsshAtomUtil.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/PsshAtomUtil.java @@ -138,7 +138,7 @@ public static byte[] parseSchemeSpecificData(byte[] atom, UUID uuid) { if (parsedAtom == null) { return null; } - if (uuid != null && !uuid.equals(parsedAtom.uuid)) { + if (!uuid.equals(parsedAtom.uuid)) { Log.w(TAG, "UUID mismatch. Expected: " + uuid + ", got: " + parsedAtom.uuid + "."); return null; } diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/SefReader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/SefReader.java new file mode 100644 index 00000000000..ccf5180f412 --- /dev/null +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/SefReader.java @@ -0,0 +1,278 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.mp4; + +import static com.google.android.exoplayer2.extractor.Extractor.RESULT_SEEK; + +import androidx.annotation.IntDef; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.ParserException; +import com.google.android.exoplayer2.extractor.Extractor; +import com.google.android.exoplayer2.extractor.ExtractorInput; +import com.google.android.exoplayer2.extractor.PositionHolder; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData; +import com.google.android.exoplayer2.util.ParsableByteArray; +import com.google.common.base.Splitter; +import java.io.IOException; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.util.ArrayList; +import java.util.List; + +/** + * Reads Samsung Extension Format (SEF) metadata. + * + *

    To be used in conjunction with {@link Mp4Extractor}. + */ +/* package */ final class SefReader { + + /** Reader states. */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + STATE_SHOULD_CHECK_FOR_SEF, + STATE_CHECKING_FOR_SEF, + STATE_READING_SDRS, + STATE_READING_SEF_DATA + }) + private @interface State {} + + private static final int STATE_SHOULD_CHECK_FOR_SEF = 0; + private static final int STATE_CHECKING_FOR_SEF = 1; + private static final int STATE_READING_SDRS = 2; + private static final int STATE_READING_SEF_DATA = 3; + + /** Supported data types. */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + TYPE_SLOW_MOTION_DATA, + TYPE_SUPER_SLOW_MOTION_DATA, + TYPE_SUPER_SLOW_MOTION_BGM, + TYPE_SUPER_SLOW_MOTION_EDIT_DATA, + TYPE_SUPER_SLOW_DEFLICKERING_ON + }) + private @interface DataType {} + + private static final int TYPE_SLOW_MOTION_DATA = 0x0890; // 2192 + private static final int TYPE_SUPER_SLOW_MOTION_DATA = 0x0b00; // 2816 + private static final int TYPE_SUPER_SLOW_MOTION_BGM = 0x0b01; // 2817 + private static final int TYPE_SUPER_SLOW_MOTION_EDIT_DATA = 0x0b03; // 2819 + private static final int TYPE_SUPER_SLOW_DEFLICKERING_ON = 0x0b04; // 2820 + + private static final String TAG = "SefReader"; + + /** + * Hex representation of `SEFT` (in ASCII). + * + *

    This is the last 4 bytes of a file that has Samsung Extension Format (SEF) data. + */ + private static final int SAMSUNG_TAIL_SIGNATURE = 0x53454654; + /** Start signature (4 bytes), SEF version (4 bytes), SDR count (4 bytes). */ + private static final int TAIL_HEADER_LENGTH = 12; + /** Tail offset (4 bytes), tail signature (4 bytes). */ + private static final int TAIL_FOOTER_LENGTH = 8; + + private static final int LENGTH_OF_ONE_SDR = 12; + private static final Splitter COLON_SPLITTER = Splitter.on(':'); + private static final Splitter ASTERISK_SPLITTER = Splitter.on('*'); + + private final List dataReferences; + @State private int readerState; + private int tailLength; + + public SefReader() { + dataReferences = new ArrayList<>(); + readerState = STATE_SHOULD_CHECK_FOR_SEF; + } + + public void reset() { + dataReferences.clear(); + readerState = STATE_SHOULD_CHECK_FOR_SEF; + } + + @Extractor.ReadResult + public int read( + ExtractorInput input, + PositionHolder seekPosition, + List slowMotionMetadataEntries) + throws IOException { + switch (readerState) { + case STATE_SHOULD_CHECK_FOR_SEF: + long inputLength = input.getLength(); + seekPosition.position = + inputLength == C.LENGTH_UNSET || inputLength < TAIL_FOOTER_LENGTH + ? 0 + : inputLength - TAIL_FOOTER_LENGTH; + readerState = STATE_CHECKING_FOR_SEF; + break; + case STATE_CHECKING_FOR_SEF: + checkForSefData(input, seekPosition); + break; + case STATE_READING_SDRS: + readSdrs(input, seekPosition); + break; + case STATE_READING_SEF_DATA: + readSefData(input, slowMotionMetadataEntries); + seekPosition.position = 0; + break; + default: + throw new IllegalStateException(); + } + return RESULT_SEEK; + } + + private void checkForSefData(ExtractorInput input, PositionHolder seekPosition) + throws IOException { + ParsableByteArray scratch = new ParsableByteArray(/* limit= */ TAIL_FOOTER_LENGTH); + input.readFully(scratch.getData(), /* offset= */ 0, /* length= */ TAIL_FOOTER_LENGTH); + tailLength = scratch.readLittleEndianInt() + TAIL_FOOTER_LENGTH; + if (scratch.readInt() != SAMSUNG_TAIL_SIGNATURE) { + seekPosition.position = 0; + return; + } + + // input.getPosition is at the very end of the tail, so jump forward by tailLength, but + // account for the tail header, which needs to be ignored. + seekPosition.position = input.getPosition() - (tailLength - TAIL_HEADER_LENGTH); + readerState = STATE_READING_SDRS; + } + + private void readSdrs(ExtractorInput input, PositionHolder seekPosition) throws IOException { + long streamLength = input.getLength(); + int sdrsLength = tailLength - TAIL_HEADER_LENGTH - TAIL_FOOTER_LENGTH; + ParsableByteArray scratch = new ParsableByteArray(/* limit= */ sdrsLength); + input.readFully(scratch.getData(), /* offset= */ 0, /* length= */ sdrsLength); + + for (int i = 0; i < sdrsLength / LENGTH_OF_ONE_SDR; i++) { + scratch.skipBytes(2); // SDR data sub info flag and reserved bits (2). + @DataType int dataType = scratch.readLittleEndianShort(); + switch (dataType) { + case TYPE_SLOW_MOTION_DATA: + case TYPE_SUPER_SLOW_MOTION_DATA: + case TYPE_SUPER_SLOW_MOTION_BGM: + case TYPE_SUPER_SLOW_MOTION_EDIT_DATA: + case TYPE_SUPER_SLOW_DEFLICKERING_ON: + // The read int is the distance from the tail info to the start of the metadata. + // Calculated as an offset from the start by working backwards. + long startOffset = streamLength - tailLength - scratch.readLittleEndianInt(); + int size = scratch.readLittleEndianInt(); + dataReferences.add(new DataReference(dataType, startOffset, size)); + break; + default: + scratch.skipBytes(8); // startPosition (4), size (4). + } + } + + if (dataReferences.isEmpty()) { + seekPosition.position = 0; + return; + } + + readerState = STATE_READING_SEF_DATA; + seekPosition.position = dataReferences.get(0).startOffset; + } + + private void readSefData(ExtractorInput input, List slowMotionMetadataEntries) + throws IOException { + long dataStartOffset = input.getPosition(); + int totalDataLength = (int) (input.getLength() - input.getPosition() - tailLength); + ParsableByteArray data = new ParsableByteArray(/* limit= */ totalDataLength); + input.readFully(data.getData(), 0, totalDataLength); + + for (int i = 0; i < dataReferences.size(); i++) { + DataReference dataReference = dataReferences.get(i); + int intendedPosition = (int) (dataReference.startOffset - dataStartOffset); + data.setPosition(intendedPosition); + + // The data type is derived from the name because the SEF format has inconsistent data type + // values. + data.skipBytes(4); // data type (2), data sub info (2). + int nameLength = data.readLittleEndianInt(); + String name = data.readString(nameLength); + @DataType int dataType = nameToDataType(name); + + int remainingDataLength = dataReference.size - (8 + nameLength); + switch (dataType) { + case TYPE_SLOW_MOTION_DATA: + slowMotionMetadataEntries.add(readSlowMotionData(data, remainingDataLength)); + break; + case TYPE_SUPER_SLOW_MOTION_DATA: + case TYPE_SUPER_SLOW_MOTION_BGM: + case TYPE_SUPER_SLOW_MOTION_EDIT_DATA: + case TYPE_SUPER_SLOW_DEFLICKERING_ON: + break; + default: + throw new IllegalStateException(); + } + } + } + + private static SlowMotionData readSlowMotionData(ParsableByteArray data, int dataLength) + throws ParserException { + List segments = new ArrayList<>(); + String dataString = data.readString(dataLength); + List segmentStrings = ASTERISK_SPLITTER.splitToList(dataString); + for (int i = 0; i < segmentStrings.size(); i++) { + List values = COLON_SPLITTER.splitToList(segmentStrings.get(i)); + if (values.size() != 3) { + throw new ParserException(); + } + try { + long startTimeMs = Long.parseLong(values.get(0)); + long endTimeMs = Long.parseLong(values.get(1)); + int speedMode = Integer.parseInt(values.get(2)); + int speedDivisor = 1 << (speedMode - 1); + segments.add(new SlowMotionData.Segment(startTimeMs, endTimeMs, speedDivisor)); + } catch (NumberFormatException e) { + throw new ParserException(e); + } + } + return new SlowMotionData(segments); + } + + @DataType + private static int nameToDataType(String name) throws ParserException { + switch (name) { + case "SlowMotion_Data": + return TYPE_SLOW_MOTION_DATA; + case "Super_SlowMotion_Data": + return TYPE_SUPER_SLOW_MOTION_DATA; + case "Super_SlowMotion_BGM": + return TYPE_SUPER_SLOW_MOTION_BGM; + case "Super_SlowMotion_Edit_Data": + return TYPE_SUPER_SLOW_MOTION_EDIT_DATA; + case "Super_SlowMotion_Deflickering_On": + return TYPE_SUPER_SLOW_DEFLICKERING_ON; + default: + throw new ParserException("Invalid SEF name"); + } + } + + private static final class DataReference { + @DataType public final int dataType; + public final long startOffset; + public final int size; + + public DataReference(@DataType int dataType, long startOffset, int size) { + this.dataType = dataType; + this.startOffset = startOffset; + this.size = size; + } + } +} diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Sniffer.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Sniffer.java index 72ce336b739..5a8b467cc2f 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Sniffer.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/mp4/Sniffer.java @@ -26,6 +26,11 @@ */ /* package */ final class Sniffer { + /** Brand stored in the ftyp atom for QuickTime media. */ + public static final int BRAND_QUICKTIME = 0x71742020; + /** Brand stored in the ftyp atom for HEIC media. */ + public static final int BRAND_HEIC = 0x68656963; + /** The maximum number of bytes to peek when sniffing. */ private static final int SEARCH_LENGTH = 4 * 1024; @@ -55,7 +60,7 @@ 0x66347620, // f4v[space] 0x6b646469, // kddi 0x4d345650, // M4VP - 0x71742020, // qt[space][space], Apple QuickTime + BRAND_QUICKTIME, // qt[space][space] 0x4d534e56, // MSNV, Sony PSP 0x64627931, // dby1, Dolby Vision 0x69736d6c, // isml @@ -71,7 +76,7 @@ * @throws IOException If an error occurs reading from the input. */ public static boolean sniffFragmented(ExtractorInput input) throws IOException { - return sniffInternal(input, true); + return sniffInternal(input, /* fragmented= */ true, /* acceptHeic= */ false); } /** @@ -83,10 +88,24 @@ public static boolean sniffFragmented(ExtractorInput input) throws IOException { * @throws IOException If an error occurs reading from the input. */ public static boolean sniffUnfragmented(ExtractorInput input) throws IOException { - return sniffInternal(input, false); + return sniffInternal(input, /* fragmented= */ false, /* acceptHeic= */ false); } - private static boolean sniffInternal(ExtractorInput input, boolean fragmented) + /** + * Returns whether data peeked from the current position in {@code input} is consistent with the + * input being an unfragmented MP4 file. + * + * @param input The extractor input from which to peek data. The peek position will be modified. + * @param acceptHeic Whether {@code true} should be returned for HEIC photos. + * @return Whether the input appears to be in the unfragmented MP4 format. + * @throws IOException If an error occurs reading from the input. + */ + public static boolean sniffUnfragmented(ExtractorInput input, boolean acceptHeic) + throws IOException { + return sniffInternal(input, /* fragmented= */ false, acceptHeic); + } + + private static boolean sniffInternal(ExtractorInput input, boolean fragmented, boolean acceptHeic) throws IOException { long inputLength = input.getLength(); int bytesToSearch = (int) (inputLength == C.LENGTH_UNSET || inputLength > SEARCH_LENGTH @@ -166,7 +185,7 @@ private static boolean sniffInternal(ExtractorInput input, boolean fragmented) if (i == 1) { // This index refers to the minorVersion, not a brand, so skip it. buffer.skipBytes(4); - } else if (isCompatibleBrand(buffer.readInt())) { + } else if (isCompatibleBrand(buffer.readInt(), acceptHeic)) { foundGoodFileType = true; break; } @@ -186,9 +205,11 @@ private static boolean sniffInternal(ExtractorInput input, boolean fragmented) /** * Returns whether {@code brand} is an ftyp atom brand that is compatible with the MP4 extractors. */ - private static boolean isCompatibleBrand(int brand) { - // Accept all brands starting '3gp'. + private static boolean isCompatibleBrand(int brand, boolean acceptHeic) { if (brand >>> 8 == 0x00336770) { + // Brand starts with '3gp'. + return true; + } else if (brand == BRAND_HEIC && acceptHeic) { return true; } for (int compatibleBrand : COMPATIBLE_BRANDS) { diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/FlacReader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/FlacReader.java index e64e6b1dc20..a86b4350c01 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/FlacReader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/FlacReader.java @@ -15,6 +15,9 @@ */ package com.google.android.exoplayer2.extractor.ogg; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; + import androidx.annotation.Nullable; import com.google.android.exoplayer2.extractor.ExtractorInput; import com.google.android.exoplayer2.extractor.FlacFrameReader; @@ -23,11 +26,11 @@ import com.google.android.exoplayer2.extractor.FlacStreamMetadata; import com.google.android.exoplayer2.extractor.FlacStreamMetadata.SeekTable; import com.google.android.exoplayer2.extractor.SeekMap; -import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.FlacConstants; import com.google.android.exoplayer2.util.ParsableByteArray; import com.google.android.exoplayer2.util.Util; import java.util.Arrays; +import org.checkerframework.checker.nullness.qual.EnsuresNonNullIf; /** * {@link StreamReader} to extract Flac data out of Ogg byte stream. @@ -68,6 +71,7 @@ protected long preparePayload(ParsableByteArray packet) { } @Override + @EnsuresNonNullIf(expression = "#3.format", result = false) protected boolean readHeaders(ParsableByteArray packet, long position, SetupData setupData) { byte[] data = packet.getData(); @Nullable FlacStreamMetadata streamMetadata = this.streamMetadata; @@ -76,18 +80,26 @@ protected boolean readHeaders(ParsableByteArray packet, long position, SetupData this.streamMetadata = streamMetadata; byte[] metadata = Arrays.copyOfRange(data, 9, packet.limit()); setupData.format = streamMetadata.getFormat(metadata, /* id3Metadata= */ null); - } else if ((data[0] & 0x7F) == FlacConstants.METADATA_TYPE_SEEK_TABLE) { + return true; + } + + if ((data[0] & 0x7F) == FlacConstants.METADATA_TYPE_SEEK_TABLE) { SeekTable seekTable = FlacMetadataReader.readSeekTableMetadataBlock(packet); streamMetadata = streamMetadata.copyWithSeekTable(seekTable); this.streamMetadata = streamMetadata; flacOggSeeker = new FlacOggSeeker(streamMetadata, seekTable); - } else if (isAudioPacket(data)) { + return true; + } + + if (isAudioPacket(data)) { if (flacOggSeeker != null) { flacOggSeeker.setFirstFrameOffset(position); setupData.oggSeeker = flacOggSeeker; } + checkNotNull(setupData.format); return false; } + return true; } @@ -142,7 +154,7 @@ public void startSeek(long targetGranule) { @Override public SeekMap createSeekMap() { - Assertions.checkState(firstFrameOffset != -1); + checkState(firstFrameOffset != -1); return new FlacSeekTableSeekMap(streamMetadata, firstFrameOffset); } diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggExtractor.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggExtractor.java index 0dfcc4ef91f..6d7f16116c6 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggExtractor.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggExtractor.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.extractor.ogg; +import static com.google.android.exoplayer2.util.Assertions.checkStateNotNull; import static java.lang.Math.min; import com.google.android.exoplayer2.C; @@ -25,7 +26,6 @@ import com.google.android.exoplayer2.extractor.ExtractorsFactory; import com.google.android.exoplayer2.extractor.PositionHolder; import com.google.android.exoplayer2.extractor.TrackOutput; -import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.ParsableByteArray; import java.io.IOException; import org.checkerframework.checker.nullness.qual.EnsuresNonNullIf; @@ -73,7 +73,7 @@ public void release() { @Override public int read(ExtractorInput input, PositionHolder seekPosition) throws IOException { - Assertions.checkStateNotNull(output); // Asserts that init has been called. + checkStateNotNull(output); // Check that init has been called. if (streamReader == null) { if (!sniffInternal(input)) { throw new ParserException("Failed to determine bitstream type"); diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggPacket.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggPacket.java index c7718e7fa9f..14a5fea607d 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggPacket.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggPacket.java @@ -87,11 +87,7 @@ public boolean populate(ExtractorInput input) throws IOException { int size = calculatePacketSize(currentSegmentIndex); int segmentIndex = currentSegmentIndex + segmentCount; if (size > 0) { - if (packetArray.capacity() < packetArray.limit() + size) { - packetArray.reset( - Arrays.copyOf(packetArray.getData(), packetArray.limit() + size), - /* limit= */ packetArray.limit()); - } + packetArray.ensureCapacity(packetArray.limit() + size); input.readFully(packetArray.getData(), packetArray.limit(), size); packetArray.setLimit(packetArray.limit() + size); populated = pageHeader.laces[segmentIndex - 1] != 255; diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggPageHeader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggPageHeader.java index 3fa5f880205..a59eca6c8ce 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggPageHeader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OggPageHeader.java @@ -104,9 +104,10 @@ public boolean skipToNextPage(ExtractorInput input) throws IOException { */ public boolean skipToNextPage(ExtractorInput input, long limit) throws IOException { Assertions.checkArgument(input.getPosition() == input.getPeekPosition()); + scratch.reset(/* limit= */ CAPTURE_PATTERN_SIZE); while ((limit == C.POSITION_UNSET || input.getPosition() + CAPTURE_PATTERN_SIZE < limit) && peekSafely(input, scratch.getData(), 0, CAPTURE_PATTERN_SIZE, /* quiet= */ true)) { - scratch.reset(/* limit= */ CAPTURE_PATTERN_SIZE); + scratch.setPosition(0); if (scratch.readUnsignedInt() == CAPTURE_PATTERN) { input.resetPeekPosition(); return true; diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OpusReader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OpusReader.java index 8144af7b660..08b6fb8caea 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OpusReader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/OpusReader.java @@ -15,12 +15,15 @@ */ package com.google.android.exoplayer2.extractor.ogg; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; + import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.audio.OpusUtil; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.ParsableByteArray; import java.util.Arrays; import java.util.List; +import org.checkerframework.checker.nullness.qual.EnsuresNonNullIf; /** * {@link StreamReader} to extract Opus data out of Ogg byte stream. @@ -55,6 +58,7 @@ protected long preparePayload(ParsableByteArray packet) { } @Override + @EnsuresNonNullIf(expression = "#3.format", result = false) protected boolean readHeaders(ParsableByteArray packet, long position, SetupData setupData) { if (!headerRead) { byte[] headerBytes = Arrays.copyOf(packet.getData(), packet.limit()); @@ -68,12 +72,13 @@ protected boolean readHeaders(ParsableByteArray packet, long position, SetupData .setInitializationData(initializationData) .build(); headerRead = true; + return true; } else { + checkNotNull(setupData.format); // Has been set when the header was read. boolean headerPacket = packet.readInt() == OPUS_CODE; packet.setPosition(0); return headerPacket; } - return true; } /** diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/StreamReader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/StreamReader.java index f28602d9b31..54221d3e5a8 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/StreamReader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/StreamReader.java @@ -15,7 +15,9 @@ */ package com.google.android.exoplayer2.extractor.ogg; -import androidx.annotation.Nullable; +import static com.google.android.exoplayer2.util.Assertions.checkStateNotNull; +import static com.google.android.exoplayer2.util.Util.castNonNull; + import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.extractor.Extractor; @@ -24,10 +26,12 @@ import com.google.android.exoplayer2.extractor.PositionHolder; import com.google.android.exoplayer2.extractor.SeekMap; import com.google.android.exoplayer2.extractor.TrackOutput; -import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.ParsableByteArray; import java.io.IOException; +import org.checkerframework.checker.nullness.qual.EnsuresNonNull; +import org.checkerframework.checker.nullness.qual.EnsuresNonNullIf; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.checkerframework.checker.nullness.qual.RequiresNonNull; /** StreamReader abstract class. */ @SuppressWarnings("UngroupedOverloads") @@ -39,8 +43,8 @@ private static final int STATE_END_OF_INPUT = 3; static class SetupData { - Format format; - OggSeeker oggSeeker; + @MonotonicNonNull Format format; + @MonotonicNonNull OggSeeker oggSeeker; } private final OggPacket oggPacket; @@ -53,13 +57,14 @@ static class SetupData { private long currentGranule; private int state; private int sampleRate; - @Nullable private SetupData setupData; + private SetupData setupData; private long lengthOfReadPacket; private boolean seekMapSet; private boolean formatSet; public StreamReader() { oggPacket = new OggPacket(); + setupData = new SetupData(); } void init(ExtractorOutput output, TrackOutput trackOutput) { @@ -95,7 +100,7 @@ final void seek(long position, long timeUs) { } else { if (state != STATE_READ_HEADERS) { targetGranule = convertTimeToGranule(timeUs); - oggSeeker.startSeek(targetGranule); + castNonNull(oggSeeker).startSeek(targetGranule); state = STATE_READ_PAYLOAD; } } @@ -103,14 +108,16 @@ final void seek(long position, long timeUs) { /** @see Extractor#read(ExtractorInput, PositionHolder) */ final int read(ExtractorInput input, PositionHolder seekPosition) throws IOException { + assertInitialized(); switch (state) { case STATE_READ_HEADERS: - return readHeaders(input); + return readHeadersAndUpdateState(input); case STATE_SKIP_HEADERS: input.skipFully((int) payloadStartPosition); state = STATE_READ_PAYLOAD; return Extractor.RESULT_CONTINUE; case STATE_READ_PAYLOAD: + castNonNull(oggSeeker); return readPayload(input, seekPosition); default: // Never happens. @@ -118,20 +125,42 @@ final int read(ExtractorInput input, PositionHolder seekPosition) throws IOExcep } } - private int readHeaders(ExtractorInput input) throws IOException { - boolean readingHeaders = true; - while (readingHeaders) { + @EnsuresNonNull({"trackOutput", "extractorOutput"}) + private void assertInitialized() { + checkStateNotNull(trackOutput); + castNonNull(extractorOutput); + } + + /** + * Read all header packets. + * + * @param input The {@link ExtractorInput} to read data from. + * @return {@code true} if all headers were read. {@code false} if end of the input is + * encountered. + * @throws IOException If reading from the input fails. + */ + @EnsuresNonNullIf(expression = "setupData.format", result = true) + private boolean readHeaders(ExtractorInput input) throws IOException { + while (true) { if (!oggPacket.populate(input)) { state = STATE_END_OF_INPUT; - return Extractor.RESULT_END_OF_INPUT; + return false; } lengthOfReadPacket = input.getPosition() - payloadStartPosition; - readingHeaders = readHeaders(oggPacket.getPayload(), payloadStartPosition, setupData); - if (readingHeaders) { + if (readHeaders(oggPacket.getPayload(), payloadStartPosition, setupData)) { payloadStartPosition = input.getPosition(); + } else { + return true; // Current packet is not a header, therefore all headers have been read. } } + } + + @RequiresNonNull({"trackOutput"}) + private int readHeadersAndUpdateState(ExtractorInput input) throws IOException { + if (!readHeaders(input)) { + return Extractor.RESULT_END_OF_INPUT; + } sampleRate = setupData.format.sampleRate; if (!formatSet) { @@ -156,13 +185,13 @@ private int readHeaders(ExtractorInput input) throws IOException { isLastPage); } - setupData = null; state = STATE_READ_PAYLOAD; // First payload packet. Trim the payload array of the ogg packet after headers have been read. oggPacket.trimPayload(); return Extractor.RESULT_CONTINUE; } + @RequiresNonNull({"trackOutput", "oggSeeker", "extractorOutput"}) private int readPayload(ExtractorInput input, PositionHolder seekPosition) throws IOException { long position = oggSeeker.read(input); if (position >= 0) { @@ -173,7 +202,7 @@ private int readPayload(ExtractorInput input, PositionHolder seekPosition) throw } if (!seekMapSet) { - SeekMap seekMap = Assertions.checkStateNotNull(oggSeeker.createSeekMap()); + SeekMap seekMap = checkStateNotNull(oggSeeker.createSeekMap()); extractorOutput.seekMap(seekMap); seekMapSet = true; } @@ -234,6 +263,7 @@ protected long convertTimeToGranule(long timeUs) { * @param setupData Setup data to be filled. * @return Whether the packet contains header data. */ + @EnsuresNonNullIf(expression = "#3.format", result = false) protected abstract boolean readHeaders( ParsableByteArray packet, long position, SetupData setupData) throws IOException; diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/VorbisReader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/VorbisReader.java index 6a8068eedbe..7e7b6eb84db 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/VorbisReader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ogg/VorbisReader.java @@ -15,6 +15,9 @@ */ package com.google.android.exoplayer2.extractor.ogg; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkStateNotNull; + import androidx.annotation.Nullable; import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.Format; @@ -26,6 +29,7 @@ import java.io.IOException; import java.util.ArrayList; import java.util.Arrays; +import org.checkerframework.checker.nullness.qual.EnsuresNonNullIf; /** * {@link StreamReader} to extract Vorbis data out of Ogg byte stream. @@ -74,7 +78,7 @@ protected long preparePayload(ParsableByteArray packet) { } // ... we need to decode the block size - int packetBlockSize = decodeBlockSize(packet.getData()[0], vorbisSetup); + int packetBlockSize = decodeBlockSize(packet.getData()[0], checkStateNotNull(vorbisSetup)); // a packet contains samples produced from overlapping the previous and current frame data // (https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-350001.3.2) int samplesInPacket = seenFirstAudioPacket ? (packetBlockSize + previousPacketBlockSize) / 4 @@ -89,9 +93,11 @@ protected long preparePayload(ParsableByteArray packet) { } @Override + @EnsuresNonNullIf(expression = "#3.format", result = false) protected boolean readHeaders(ParsableByteArray packet, long position, SetupData setupData) throws IOException { if (vorbisSetup != null) { + checkNotNull(setupData.format); return false; } @@ -99,6 +105,7 @@ protected boolean readHeaders(ParsableByteArray packet, long position, SetupData if (vorbisSetup == null) { return true; } + VorbisSetup vorbisSetup = this.vorbisSetup; VorbisUtil.VorbisIdHeader idHeader = vorbisSetup.idHeader; @@ -131,6 +138,8 @@ protected boolean readHeaders(ParsableByteArray packet, long position, SetupData commentHeader = VorbisUtil.readVorbisCommentHeader(scratch); return null; } + VorbisUtil.VorbisIdHeader vorbisIdHeader = this.vorbisIdHeader; + VorbisUtil.CommentHeader commentHeader = this.commentHeader; // the third packet contains the setup header byte[] setupHeaderData = new byte[scratch.limit()]; diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/H265Reader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/H265Reader.java index ea23e1ef7a5..7b8cabeb006 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/H265Reader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/H265Reader.java @@ -24,6 +24,7 @@ import com.google.android.exoplayer2.extractor.TrackOutput; import com.google.android.exoplayer2.extractor.ts.TsPayloadReader.TrackIdGenerator; import com.google.android.exoplayer2.util.Assertions; +import com.google.android.exoplayer2.util.CodecSpecificDataUtil; import com.google.android.exoplayer2.util.Log; import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.NalUnitUtil; @@ -334,11 +335,37 @@ private static Format parseMediaFormat( Log.w(TAG, "Unexpected aspect_ratio_idc value: " + aspectRatioIdc); } } + if (bitArray.readBit()) { // overscan_info_present_flag + bitArray.skipBit(); // overscan_appropriate_flag + } + if (bitArray.readBit()) { // video_signal_type_present_flag + bitArray.skipBits(4); // video_format, video_full_range_flag + if (bitArray.readBit()) { // colour_description_present_flag + // colour_primaries, transfer_characteristics, matrix_coeffs + bitArray.skipBits(24); + } + } + if (bitArray.readBit()) { // chroma_loc_info_present_flag + bitArray.readUnsignedExpGolombCodedInt(); // chroma_sample_loc_type_top_field + bitArray.readUnsignedExpGolombCodedInt(); // chroma_sample_loc_type_bottom_field + } + bitArray.skipBit(); // neutral_chroma_indication_flag + if (bitArray.readBit()) { // field_seq_flag + // field_seq_flag equal to 1 indicates that the coded video sequence conveys pictures that + // represent fields, which means that frame height is double the picture height. + picHeightInLumaSamples *= 2; + } } + // Parse the SPS to derive an RFC 6381 codecs string. + bitArray.reset(sps.nalData, 0, sps.nalLength); + bitArray.skipBits(24); // Skip start code. + String codecs = CodecSpecificDataUtil.buildHevcCodecStringFromSps(bitArray); + return new Format.Builder() .setId(formatId) .setSampleMimeType(MimeTypes.VIDEO_H265) + .setCodecs(codecs) .setWidth(picWidthInLumaSamples) .setHeight(picHeightInLumaSamples) .setPixelWidthHeightRatio(pixelWidthHeightRatio) diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/SectionReader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/SectionReader.java index 8d935ad5f3e..3b2bd4ce894 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/SectionReader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/ts/SectionReader.java @@ -91,10 +91,13 @@ public void consume(ParsableByteArray data, @Flags int flags) { } } int headerBytesToRead = min(data.bytesLeft(), SECTION_HEADER_LENGTH - bytesRead); + // sectionData is guaranteed to have enough space because it's initialized with a 32-element + // backing array and headerBytesToRead is at most 3. data.readBytes(sectionData.getData(), bytesRead, headerBytesToRead); bytesRead += headerBytesToRead; if (bytesRead == SECTION_HEADER_LENGTH) { - sectionData.reset(SECTION_HEADER_LENGTH); + sectionData.setPosition(0); + sectionData.setLimit(SECTION_HEADER_LENGTH); sectionData.skipBytes(1); // Skip table id (8). int secondHeaderByte = sectionData.readUnsignedByte(); int thirdHeaderByte = sectionData.readUnsignedByte(); @@ -103,14 +106,15 @@ public void consume(ParsableByteArray data, @Flags int flags) { (((secondHeaderByte & 0x0F) << 8) | thirdHeaderByte) + SECTION_HEADER_LENGTH; if (sectionData.capacity() < totalSectionLength) { // Ensure there is enough space to keep the whole section. - byte[] bytes = sectionData.getData(); - sectionData.reset(min(MAX_SECTION_LENGTH, max(totalSectionLength, bytes.length * 2))); - System.arraycopy(bytes, 0, sectionData.getData(), 0, SECTION_HEADER_LENGTH); + int limit = + min(MAX_SECTION_LENGTH, max(totalSectionLength, sectionData.capacity() * 2)); + sectionData.ensureCapacity(limit); } } } else { // Reading the body. int bodyBytesToRead = min(data.bytesLeft(), totalSectionLength - bytesRead); + // sectionData has been sized large enough for totalSectionLength when reading the header. data.readBytes(sectionData.getData(), bytesRead, bodyBytesToRead); bytesRead += bodyBytesToRead; if (bytesRead == totalSectionLength) { @@ -121,11 +125,12 @@ public void consume(ParsableByteArray data, @Flags int flags) { waitingForPayloadStart = true; return; } - sectionData.reset(totalSectionLength - 4); // Exclude the CRC_32 field. + sectionData.setLimit(totalSectionLength - 4); // Exclude the CRC_32 field. } else { // This is a private section with private defined syntax. - sectionData.reset(totalSectionLength); + sectionData.setLimit(totalSectionLength); } + sectionData.setPosition(0); reader.consume(sectionData); bytesRead = 0; } diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/wav/WavExtractor.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/wav/WavExtractor.java index acb06063b57..c1693abec97 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/wav/WavExtractor.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/wav/WavExtractor.java @@ -472,7 +472,8 @@ private void decode(byte[] input, int blockCount, ParsableByteArray output) { } } int decodedDataSize = numOutputFramesToBytes(framesPerBlock * blockCount); - output.reset(decodedDataSize); + output.setPosition(0); + output.setLimit(decodedDataSize); } private void decodeBlockForChannel( diff --git a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/wav/WavHeaderReader.java b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/wav/WavHeaderReader.java index 4387993f509..af8ede69aa1 100644 --- a/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/wav/WavHeaderReader.java +++ b/library/extractor/src/main/java/com/google/android/exoplayer2/extractor/wav/WavHeaderReader.java @@ -115,7 +115,7 @@ public static Pair skipToData(ExtractorInput input) throws IOExcepti input.resetPeekPosition(); ParsableByteArray scratch = new ParsableByteArray(ChunkHeader.SIZE_IN_BYTES); - // Skip all chunks until we hit the data header. + // Skip all chunks until we find the data header. ChunkHeader chunkHeader = ChunkHeader.peek(input, scratch); while (chunkHeader.id != WavUtil.DATA_FOURCC) { if (chunkHeader.id != WavUtil.RIFF_FOURCC && chunkHeader.id != WavUtil.FMT_FOURCC) { diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/DefaultExtractorsFactoryTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/DefaultExtractorsFactoryTest.java index ba10f56a513..1c6ce7b70c8 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/DefaultExtractorsFactoryTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/DefaultExtractorsFactoryTest.java @@ -22,6 +22,7 @@ import com.google.android.exoplayer2.extractor.amr.AmrExtractor; import com.google.android.exoplayer2.extractor.flac.FlacExtractor; import com.google.android.exoplayer2.extractor.flv.FlvExtractor; +import com.google.android.exoplayer2.extractor.jpeg.JpegExtractor; import com.google.android.exoplayer2.extractor.mkv.MatroskaExtractor; import com.google.android.exoplayer2.extractor.mp3.Mp3Extractor; import com.google.android.exoplayer2.extractor.mp4.FragmentedMp4Extractor; @@ -68,7 +69,8 @@ public void createExtractors_withoutMediaInfo_optimizesSniffingOrder() { AdtsExtractor.class, Ac3Extractor.class, Ac4Extractor.class, - Mp3Extractor.class) + Mp3Extractor.class, + JpegExtractor.class) .inOrder(); } @@ -109,7 +111,8 @@ public void createExtractors_withMediaInfo_optimizesSniffingOrder() { MatroskaExtractor.class, AdtsExtractor.class, Ac3Extractor.class, - Ac4Extractor.class) + Ac4Extractor.class, + JpegExtractor.class) .inOrder(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/amr/AmrExtractorNonParameterizedTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/amr/AmrExtractorNonParameterizedTest.java index 65b8122c4ab..4c31858a41c 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/amr/AmrExtractorNonParameterizedTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/amr/AmrExtractorNonParameterizedTest.java @@ -176,14 +176,14 @@ public void read_amrWb_returnParserException_forInvalidFrameHeader() throws IOEx } private byte[] newWideBandAmrFrameWithType(int frameType) { - byte frameHeader = (byte) ((frameType << 3) & (0b01111100)); + byte frameHeader = (byte) ((frameType << 3) & 0b01111100); int frameContentInBytes = frameSizeBytesByTypeWb(frameType) - 1; return joinData(new byte[] {frameHeader}, randomBytesArrayWithLength(frameContentInBytes)); } private byte[] newNarrowBandAmrFrameWithType(int frameType) { - byte frameHeader = (byte) ((frameType << 3) & (0b01111100)); + byte frameHeader = (byte) ((frameType << 3) & 0b01111100); int frameContentInBytes = frameSizeBytesByTypeNb(frameType) - 1; return joinData(new byte[] {frameHeader}, randomBytesArrayWithLength(frameContentInBytes)); diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/amr/AmrExtractorParameterizedTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/amr/AmrExtractorParameterizedTest.java index 53913e07cc0..1c28a143fa7 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/amr/AmrExtractorParameterizedTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/amr/AmrExtractorParameterizedTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.extractor.amr; import com.google.android.exoplayer2.testutil.ExtractorAsserts; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -33,7 +33,7 @@ public final class AmrExtractorParameterizedTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/flac/FlacExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/flac/FlacExtractorTest.java index 500cdd4e86f..1d776b93551 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/flac/FlacExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/flac/FlacExtractorTest.java @@ -17,7 +17,7 @@ import com.google.android.exoplayer2.testutil.ExtractorAsserts; import com.google.android.exoplayer2.testutil.ExtractorAsserts.AssertionConfig; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -29,7 +29,7 @@ public class FlacExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/flv/FlvExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/flv/FlvExtractorTest.java index 248e4b378db..5a7e0a5a3e9 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/flv/FlvExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/flv/FlvExtractorTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.extractor.flv; import com.google.android.exoplayer2.testutil.ExtractorAsserts; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -28,7 +28,7 @@ public final class FlvExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/jpeg/JpegExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/jpeg/JpegExtractorTest.java new file mode 100644 index 00000000000..9166f335a7e --- /dev/null +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/jpeg/JpegExtractorTest.java @@ -0,0 +1,61 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.jpeg; + +import com.google.android.exoplayer2.testutil.ExtractorAsserts; +import com.google.common.collect.ImmutableList; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; + +/** Unit tests for {@link JpegExtractor}. */ +@RunWith(ParameterizedRobolectricTestRunner.class) +public final class JpegExtractorTest { + + @ParameterizedRobolectricTestRunner.Parameters(name = "{0}") + public static ImmutableList params() { + return ExtractorAsserts.configs(); + } + + @ParameterizedRobolectricTestRunner.Parameter + public ExtractorAsserts.SimulationConfig simulationConfig; + + @Test + public void sampleNonMotionPhotoShortened() throws Exception { + ExtractorAsserts.assertBehavior( + JpegExtractor::new, "media/jpeg/non-motion-photo-shortened.jpg", simulationConfig); + } + + @Test + public void samplePixelMotionPhotoShortened() throws Exception { + ExtractorAsserts.assertBehavior( + JpegExtractor::new, "media/jpeg/pixel-motion-photo-shortened.jpg", simulationConfig); + } + + @Test + public void samplePixelMotionPhotoVideoRemovedShortened() throws Exception { + ExtractorAsserts.assertBehavior( + JpegExtractor::new, + "media/jpeg/pixel-motion-photo-video-removed-shortened.jpg", + simulationConfig); + } + + @Test + public void sampleSsMotionPhotoShortened() throws Exception { + ExtractorAsserts.assertBehavior( + JpegExtractor::new, "media/jpeg/ss-motion-photo-shortened.jpg", simulationConfig); + } +} diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/jpeg/MotionPhotoDescriptionTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/jpeg/MotionPhotoDescriptionTest.java new file mode 100644 index 00000000000..6c068d2587d --- /dev/null +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/jpeg/MotionPhotoDescriptionTest.java @@ -0,0 +1,157 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.jpeg; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.annotation.Nullable; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.metadata.mp4.MotionPhotoMetadata; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.common.collect.ImmutableList; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link MotionPhotoDescription}. */ +@RunWith(AndroidJUnit4.class) +public final class MotionPhotoDescriptionTest { + + private static final long TEST_PRESENTATION_TIMESTAMP_US = 5L; + private static final long TEST_MOTION_PHOTO_LENGTH_BYTES = 20; + private static final long TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES = 7; + private static final long TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES = 1; + + @Test + public void getMotionPhotoMetadata_withPrimaryAndSecondaryMediaItems() { + MotionPhotoDescription motionPhotoDescription = + new MotionPhotoDescription( + TEST_PRESENTATION_TIMESTAMP_US, + ImmutableList.of( + new MotionPhotoDescription.ContainerItem( + MimeTypes.IMAGE_JPEG, + "Primary", + /* length= */ 0, + TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES), + new MotionPhotoDescription.ContainerItem( + MimeTypes.VIDEO_MP4, + "MotionPhoto", + TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES, + /* padding= */ 0))); + + @Nullable + MotionPhotoMetadata metadata = + motionPhotoDescription.getMotionPhotoMetadata(TEST_MOTION_PHOTO_LENGTH_BYTES); + + assertThat(metadata.photoStartPosition).isEqualTo(0); + assertThat(metadata.photoSize) + .isEqualTo( + TEST_MOTION_PHOTO_LENGTH_BYTES + - TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES + - TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES); + assertThat(metadata.photoPresentationTimestampUs).isEqualTo(TEST_PRESENTATION_TIMESTAMP_US); + assertThat(metadata.videoStartPosition) + .isEqualTo(TEST_MOTION_PHOTO_LENGTH_BYTES - TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES); + assertThat(metadata.videoSize).isEqualTo(TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES); + } + + @Test + public void + getMotionPhotoMetadata_withPrimaryAndMultipleSecondaryMediaItems_returnsSecondMediaItemAsVideo() { + MotionPhotoDescription motionPhotoDescription = + new MotionPhotoDescription( + TEST_PRESENTATION_TIMESTAMP_US, + ImmutableList.of( + new MotionPhotoDescription.ContainerItem( + MimeTypes.IMAGE_JPEG, + "Primary", + /* length= */ 0, + TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES), + new MotionPhotoDescription.ContainerItem( + MimeTypes.VIDEO_MP4, + "MotionPhoto", + TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES, + /* padding= */ 0), + new MotionPhotoDescription.ContainerItem( + MimeTypes.VIDEO_MP4, + "MotionPhoto", + TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES, + /* padding= */ 0))); + + @Nullable + MotionPhotoMetadata metadata = + motionPhotoDescription.getMotionPhotoMetadata(TEST_MOTION_PHOTO_LENGTH_BYTES); + + assertThat(metadata.photoStartPosition).isEqualTo(0); + assertThat(metadata.photoSize) + .isEqualTo( + TEST_MOTION_PHOTO_LENGTH_BYTES + - TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES * 2 + - TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES); + assertThat(metadata.photoPresentationTimestampUs).isEqualTo(TEST_PRESENTATION_TIMESTAMP_US); + assertThat(metadata.videoStartPosition) + .isEqualTo(TEST_MOTION_PHOTO_LENGTH_BYTES - TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES * 2); + assertThat(metadata.videoSize).isEqualTo(TEST_MOTION_PHOTO_VIDEO_LENGTH_BYTES); + } + + @Test + public void + getMotionPhotoMetadata_withPrimaryAndSecondaryItemSharingData_returnsPrimaryItemAsPhotoAndVideo() { + // Theoretical example of an HEIF file that has both an image and a video represented in the + // same file, which looks like an MP4. + MotionPhotoDescription motionPhotoDescription = + new MotionPhotoDescription( + TEST_PRESENTATION_TIMESTAMP_US, + ImmutableList.of( + new MotionPhotoDescription.ContainerItem( + MimeTypes.VIDEO_MP4, + "Primary", + /* length= */ 0, + TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES), + new MotionPhotoDescription.ContainerItem( + MimeTypes.VIDEO_MP4, "MotionPhoto", /* length= */ 0, /* padding= */ 0))); + + @Nullable + MotionPhotoMetadata metadata = + motionPhotoDescription.getMotionPhotoMetadata(TEST_MOTION_PHOTO_LENGTH_BYTES); + + assertThat(metadata.photoStartPosition).isEqualTo(0); + assertThat(metadata.photoSize) + .isEqualTo(TEST_MOTION_PHOTO_LENGTH_BYTES - TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES); + assertThat(metadata.photoPresentationTimestampUs).isEqualTo(TEST_PRESENTATION_TIMESTAMP_US); + assertThat(metadata.videoStartPosition).isEqualTo(0); + assertThat(metadata.videoSize) + .isEqualTo(TEST_MOTION_PHOTO_LENGTH_BYTES - TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES); + } + + @Test + public void getMotionPhotoMetadata_withOnlyPrimaryItem_returnsNull() { + MotionPhotoDescription motionPhotoDescription = + new MotionPhotoDescription( + TEST_PRESENTATION_TIMESTAMP_US, + ImmutableList.of( + new MotionPhotoDescription.ContainerItem( + MimeTypes.VIDEO_MP4, + "Primary", + /* length= */ 0, + TEST_MOTION_PHOTO_PHOTO_PADDING_BYTES))); + + @Nullable + MotionPhotoMetadata metadata = + motionPhotoDescription.getMotionPhotoMetadata(TEST_MOTION_PHOTO_LENGTH_BYTES); + + assertThat(metadata).isNull(); + } +} diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mkv/MatroskaExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mkv/MatroskaExtractorTest.java index 45c0bd1c9f7..64faff9a0ec 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mkv/MatroskaExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mkv/MatroskaExtractorTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.extractor.mkv; import com.google.android.exoplayer2.testutil.ExtractorAsserts; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -28,7 +28,7 @@ public final class MatroskaExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } @@ -67,6 +67,12 @@ public void mkvSample_withNullTerminatedSsaSubtitles() throws Exception { simulationConfig); } + @Test + public void mkvSample_withVorbisAudio() throws Exception { + ExtractorAsserts.assertBehavior( + MatroskaExtractor::new, "media/mkv/sample_with_vorbis_audio.mkv", simulationConfig); + } + @Test public void mkvSample_withHtcRotationInfoInTrackName() throws Exception { ExtractorAsserts.assertBehavior( diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp3/Mp3ExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp3/Mp3ExtractorTest.java index f59e3e77a86..f209574de45 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp3/Mp3ExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp3/Mp3ExtractorTest.java @@ -17,7 +17,7 @@ import com.google.android.exoplayer2.testutil.ExtractorAsserts; import com.google.android.exoplayer2.testutil.ExtractorAsserts.AssertionConfig; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -29,7 +29,7 @@ public final class Mp3ExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4ExtractorNoSniffingTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4ExtractorNoSniffingTest.java new file mode 100644 index 00000000000..969c9b8d5a7 --- /dev/null +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4ExtractorNoSniffingTest.java @@ -0,0 +1,66 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.mp4; + +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.testutil.ExtractorAsserts; +import com.google.android.exoplayer2.util.MimeTypes; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.ParameterizedRobolectricTestRunner; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameter; +import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; + +/** + * Tests for {@link FragmentedMp4Extractor} that test behaviours where sniffing must not be tested. + */ +@RunWith(ParameterizedRobolectricTestRunner.class) +public class FragmentedMp4ExtractorNoSniffingTest { + + @Parameters(name = "{0}") + public static List params() { + return ExtractorAsserts.configsNoSniffing(); + } + + @Parameter public ExtractorAsserts.SimulationConfig simulationConfig; + + @Test + public void sampleWithSideLoadedTrack() throws Exception { + // Sideloaded tracks are generally used in Smooth Streaming, where the MP4 files do not contain + // any ftyp box and are not sniffed. + Track sideloadedTrack = + new Track( + /* id= */ 1, + /* type= */ C.TRACK_TYPE_VIDEO, + /* timescale= */ 30_000, + /* movieTimescale= */ 1000, + /* durationUs= */ C.TIME_UNSET, + new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(), + /* sampleTransformation= */ Track.TRANSFORMATION_NONE, + /* sampleDescriptionEncryptionBoxes= */ null, + /* nalUnitLengthFieldLength= */ 4, + /* editListDurations= */ null, + /* editListMediaTimes= */ null); + ExtractorAsserts.assertBehavior( + () -> + new FragmentedMp4Extractor( + /* flags= */ 0, /* timestampAdjuster= */ null, sideloadedTrack), + "media/mp4/sample_fragmented_sideloaded_track.mp4", + simulationConfig); + } +} diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4ExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4ExtractorTest.java index e8ab027e9b1..a9d2397ca71 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4ExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/FragmentedMp4ExtractorTest.java @@ -28,12 +28,12 @@ import org.robolectric.ParameterizedRobolectricTestRunner.Parameter; import org.robolectric.ParameterizedRobolectricTestRunner.Parameters; -/** Unit test for {@link FragmentedMp4Extractor}. */ +/** Tests for {@link FragmentedMp4Extractor} that test behaviours where sniffing must be tested. */ @RunWith(ParameterizedRobolectricTestRunner.class) public final class FragmentedMp4ExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/Mp4ExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/Mp4ExtractorTest.java index c2e23673076..e76bcc395b8 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/Mp4ExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/Mp4ExtractorTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.extractor.mp4; import com.google.android.exoplayer2.testutil.ExtractorAsserts; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -28,7 +28,7 @@ public final class Mp4ExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/SlowMotionDataTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/SlowMotionDataTest.java new file mode 100644 index 00000000000..5b1c33303dd --- /dev/null +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/mp4/SlowMotionDataTest.java @@ -0,0 +1,72 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.extractor.mp4; + +import static com.google.common.truth.Truth.assertThat; + +import android.os.Parcel; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData; +import java.util.ArrayList; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link SlowMotionData} */ +@RunWith(AndroidJUnit4.class) +public class SlowMotionDataTest { + + @Test + public void parcelable() { + List segments = new ArrayList<>(); + segments.add( + new SlowMotionData.Segment( + /* startTimeMs= */ 1000, /* endTimeMs= */ 2000, /* speedDivisor= */ 4)); + segments.add( + new SlowMotionData.Segment( + /* startTimeMs= */ 2600, /* endTimeMs= */ 4000, /* speedDivisor= */ 8)); + segments.add( + new SlowMotionData.Segment( + /* startTimeMs= */ 8765, /* endTimeMs= */ 12485, /* speedDivisor= */ 16)); + + SlowMotionData slowMotionDataToParcel = new SlowMotionData(segments); + Parcel parcel = Parcel.obtain(); + slowMotionDataToParcel.writeToParcel(parcel, /* flags= */ 0); + parcel.setDataPosition(0); + + SlowMotionData slowMotionDataFromParcel = SlowMotionData.CREATOR.createFromParcel(parcel); + assertThat(slowMotionDataFromParcel).isEqualTo(slowMotionDataToParcel); + + parcel.recycle(); + } + + @Test + public void segment_parcelable() { + SlowMotionData.Segment segmentToParcel = + new SlowMotionData.Segment( + /* startTimeMs= */ 1000, /* endTimeMs= */ 2000, /* speedDivisor= */ 4); + + Parcel parcel = Parcel.obtain(); + segmentToParcel.writeToParcel(parcel, /* flags= */ 0); + parcel.setDataPosition(0); + + SlowMotionData.Segment segmentFromParcel = + SlowMotionData.Segment.CREATOR.createFromParcel(parcel); + assertThat(segmentFromParcel).isEqualTo(segmentToParcel); + + parcel.recycle(); + } +} diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ogg/OggExtractorParameterizedTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ogg/OggExtractorParameterizedTest.java index 0731cfd95e5..8cbe254f07f 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ogg/OggExtractorParameterizedTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ogg/OggExtractorParameterizedTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.extractor.ogg; import com.google.android.exoplayer2.testutil.ExtractorAsserts; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -32,7 +32,7 @@ public final class OggExtractorParameterizedTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/rawcc/RawCcExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/rawcc/RawCcExtractorTest.java index 173a4049619..3856f2b573b 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/rawcc/RawCcExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/rawcc/RawCcExtractorTest.java @@ -18,7 +18,7 @@ import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.testutil.ExtractorAsserts; import com.google.android.exoplayer2.util.MimeTypes; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -28,7 +28,7 @@ public final class RawCcExtractorTest { @ParameterizedRobolectricTestRunner.Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/Ac3ExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/Ac3ExtractorTest.java index 4c8ddfb1535..8b0ffef80d3 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/Ac3ExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/Ac3ExtractorTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.extractor.ts; import com.google.android.exoplayer2.testutil.ExtractorAsserts; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -28,7 +28,7 @@ public final class Ac3ExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/Ac4ExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/Ac4ExtractorTest.java index 23b066088aa..39ab1bb5347 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/Ac4ExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/Ac4ExtractorTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.extractor.ts; import com.google.android.exoplayer2.testutil.ExtractorAsserts; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -28,7 +28,7 @@ public final class Ac4ExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/AdtsExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/AdtsExtractorTest.java index dca8ba99383..420d8d589b2 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/AdtsExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/AdtsExtractorTest.java @@ -17,7 +17,7 @@ import com.google.android.exoplayer2.testutil.ExtractorAsserts; import com.google.android.exoplayer2.testutil.ExtractorAsserts.AssertionConfig; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -29,7 +29,7 @@ public final class AdtsExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/PsExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/PsExtractorTest.java index a7bd75a56c8..688cc318f18 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/PsExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/PsExtractorTest.java @@ -16,7 +16,7 @@ package com.google.android.exoplayer2.extractor.ts; import com.google.android.exoplayer2.testutil.ExtractorAsserts; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -28,7 +28,7 @@ public final class PsExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/TsExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/TsExtractorTest.java index c2fe39285fa..87215d45ee8 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/TsExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/ts/TsExtractorTest.java @@ -36,7 +36,7 @@ import com.google.android.exoplayer2.testutil.TestUtil; import com.google.android.exoplayer2.util.ParsableByteArray; import com.google.android.exoplayer2.util.TimestampAdjuster; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -48,7 +48,7 @@ public final class TsExtractorTest { @Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/wav/WavExtractorTest.java b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/wav/WavExtractorTest.java index b411e7517af..4217a1528a3 100644 --- a/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/wav/WavExtractorTest.java +++ b/library/extractor/src/test/java/com/google/android/exoplayer2/extractor/wav/WavExtractorTest.java @@ -17,7 +17,7 @@ import com.google.android.exoplayer2.testutil.ExtractorAsserts; import com.google.android.exoplayer2.testutil.ExtractorAsserts.AssertionConfig; -import java.util.List; +import com.google.common.collect.ImmutableList; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.ParameterizedRobolectricTestRunner; @@ -27,7 +27,7 @@ public final class WavExtractorTest { @ParameterizedRobolectricTestRunner.Parameters(name = "{0}") - public static List params() { + public static ImmutableList params() { return ExtractorAsserts.configs(); } diff --git a/library/hls/build.gradle b/library/hls/build.gradle index 2cc91a5105b..d31f1ce6a23 100644 --- a/library/hls/build.gradle +++ b/library/hls/build.gradle @@ -29,8 +29,10 @@ dependencies { compileOnly 'org.checkerframework:checker-compat-qual:' + checkerframeworkCompatVersion compileOnly 'org.jetbrains.kotlin:kotlin-annotations-jvm:' + kotlinAnnotationsVersion implementation project(modulePrefix + 'library-core') + testImplementation project(modulePrefix + 'robolectricutils') testImplementation project(modulePrefix + 'testutils') testImplementation project(modulePrefix + 'testdata') + testImplementation 'com.squareup.okhttp3:mockwebserver:' + mockWebServerVersion testImplementation 'org.robolectric:robolectric:' + robolectricVersion } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/DefaultHlsExtractorFactory.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/DefaultHlsExtractorFactory.java index c8ef90742b3..a1251810ad6 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/DefaultHlsExtractorFactory.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/DefaultHlsExtractorFactory.java @@ -17,6 +17,7 @@ import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import android.annotation.SuppressLint; import android.net.Uri; import android.text.TextUtils; import androidx.annotation.Nullable; @@ -125,6 +126,8 @@ public BundledHlsMediaChunkExtractor createExtractor( return new BundledHlsMediaChunkExtractor(extractor, format, timestampAdjuster); } if (fileType == FileTypes.TS) { + // Fall back on TsExtractor to handle TS streams with an EXT-X-MAP tag. See + // https://github.com/google/ExoPlayer/issues/8219. fallBackExtractor = extractor; } } @@ -141,6 +144,7 @@ private static void addFileTypeIfNotPresent( fileTypes.add(fileType); } + @SuppressLint("SwitchIntDef") // HLS only supports a small subset of the defined file types. @Nullable private Extractor createExtractorByFileType( @FileTypes.Type int fileType, diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsChunkSource.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsChunkSource.java index 2ab4852339b..66cd100a632 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsChunkSource.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsChunkSource.java @@ -15,11 +15,15 @@ */ package com.google.android.exoplayer2.source.hls; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; import static java.lang.Math.max; import android.net.Uri; import android.os.SystemClock; +import android.util.Pair; +import androidx.annotation.IntDef; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.source.BehindLiveWindowException; @@ -33,18 +37,22 @@ import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist.Segment; import com.google.android.exoplayer2.source.hls.playlist.HlsPlaylistTracker; import com.google.android.exoplayer2.trackselection.BaseTrackSelection; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DataSpec; import com.google.android.exoplayer2.upstream.TransferListener; -import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.TimestampAdjuster; import com.google.android.exoplayer2.util.UriUtil; import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Iterables; import com.google.common.primitives.Ints; import java.io.IOException; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; import java.util.ArrayList; import java.util.Arrays; +import java.util.Collections; import java.util.List; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @@ -79,9 +87,31 @@ public void clear() { endOfStream = false; playlistUrl = null; } - } + /** + * Chunk publication state. One of {@link #CHUNK_PUBLICATION_STATE_PRELOAD}, {@link + * #CHUNK_PUBLICATION_STATE_PUBLISHED}, {@link #CHUNK_PUBLICATION_STATE_REMOVED}. + */ + @IntDef({ + CHUNK_PUBLICATION_STATE_PRELOAD, + CHUNK_PUBLICATION_STATE_PUBLISHED, + CHUNK_PUBLICATION_STATE_REMOVED + }) + @Retention(RetentionPolicy.SOURCE) + @interface ChunkPublicationState {} + + /** Indicates that the chunk is based on a preload hint. */ + public static final int CHUNK_PUBLICATION_STATE_PRELOAD = 0; + /** Indicates that the chunk is definitely published. */ + public static final int CHUNK_PUBLICATION_STATE_PUBLISHED = 1; + /** + * Indicates that the chunk has been removed from the playlist. + * + *

    See RFC 8216, Section 6.2.6 also. + */ + public static final int CHUNK_PUBLICATION_STATE_REMOVED = 2; + /** * The maximum number of keys that the key cache can hold. This value must be 2 or greater in * order to hold initialization segment and media segment keys simultaneously. @@ -107,8 +137,8 @@ public void clear() { // Note: The track group in the selection is typically *not* equal to trackGroup. This is due to // the way in which HlsSampleStreamWrapper generates track groups. Use only index based methods - // in TrackSelection to avoid unexpected behavior. - private TrackSelection trackSelection; + // in ExoTrackSelection to avoid unexpected behavior. + private ExoTrackSelection trackSelection; private long liveEdgeInPeriodTimeUs; private boolean seenExpectedPlaylistError; @@ -189,14 +219,14 @@ public TrackGroup getTrackGroup() { /** * Sets the current track selection. * - * @param trackSelection The {@link TrackSelection}. + * @param trackSelection The {@link ExoTrackSelection}. */ - public void setTrackSelection(TrackSelection trackSelection) { + public void setTrackSelection(ExoTrackSelection trackSelection) { this.trackSelection = trackSelection; } - /** Returns the current {@link TrackSelection}. */ - public TrackSelection getTrackSelection() { + /** Returns the current {@link ExoTrackSelection}. */ + public ExoTrackSelection getTrackSelection() { return trackSelection; } @@ -217,6 +247,53 @@ public void setIsTimestampMaster(boolean isTimestampMaster) { this.isTimestampMaster = isTimestampMaster; } + /** + * Returns the publication state of the given chunk. + * + * @param mediaChunk The media chunk for which to evaluate the publication state. + * @return Whether the media chunk is {@link #CHUNK_PUBLICATION_STATE_PRELOAD a preload chunk}, + * has been {@link #CHUNK_PUBLICATION_STATE_REMOVED removed} or is definitely {@link + * #CHUNK_PUBLICATION_STATE_PUBLISHED published}. + */ + @ChunkPublicationState + public int getChunkPublicationState(HlsMediaChunk mediaChunk) { + if (mediaChunk.partIndex == C.INDEX_UNSET) { + // Chunks based on full segments can't be removed and are always published. + return CHUNK_PUBLICATION_STATE_PUBLISHED; + } + Uri playlistUrl = playlistUrls[trackGroup.indexOf(mediaChunk.trackFormat)]; + HlsMediaPlaylist mediaPlaylist = + checkNotNull(playlistTracker.getPlaylistSnapshot(playlistUrl, /* isForPlayback= */ false)); + int segmentIndexInPlaylist = (int) (mediaChunk.chunkIndex - mediaPlaylist.mediaSequence); + if (segmentIndexInPlaylist < 0) { + // The parent segment of the previous chunk is not in the current playlist anymore. + return CHUNK_PUBLICATION_STATE_PUBLISHED; + } + List partsInCurrentPlaylist = + segmentIndexInPlaylist < mediaPlaylist.segments.size() + ? mediaPlaylist.segments.get(segmentIndexInPlaylist).parts + : mediaPlaylist.trailingParts; + if (mediaChunk.partIndex >= partsInCurrentPlaylist.size()) { + // In case the part hinted in the previous playlist has been wrongly assigned to the then full + // but not yet terminated segment, we discard it regardless whether the URI is different or + // not. While this is theoretically possible and unspecified, it appears to be an edge case + // which we can avoid with a small inefficiency of discarding in vain. We could allow this + // here but, if the chunk is not discarded, it could create unpredictable problems later, + // because the media sequence in previous.chunkIndex does not match to the actual media + // sequence in the new playlist. + return CHUNK_PUBLICATION_STATE_REMOVED; + } + HlsMediaPlaylist.Part newPart = partsInCurrentPlaylist.get(mediaChunk.partIndex); + if (newPart.isPreload) { + // The playlist did not change and the part in the new playlist is still a preload hint. + return CHUNK_PUBLICATION_STATE_PRELOAD; + } + Uri newUri = Uri.parse(UriUtil.resolve(mediaPlaylist.baseUri, newPart.url)); + return Util.areEqual(newUri, mediaChunk.dataSpec.uri) + ? CHUNK_PUBLICATION_STATE_PUBLISHED + : CHUNK_PUBLICATION_STATE_REMOVED; + } + /** * Returns the next chunk to load. * @@ -242,7 +319,7 @@ public void getNextChunk( List queue, boolean allowEndOfStream, HlsChunkHolder out) { - HlsMediaChunk previous = queue.isEmpty() ? null : queue.get(queue.size() - 1); + @Nullable HlsMediaChunk previous = queue.isEmpty() ? null : Iterables.getLast(queue); int oldTrackIndex = previous == null ? C.INDEX_UNSET : trackGroup.indexOf(previous.trackFormat); long bufferedDurationUs = loadPositionUs - playbackPositionUs; long timeToLiveEdgeUs = resolveTimeToLiveEdgeUs(playbackPositionUs); @@ -265,7 +342,6 @@ public void getNextChunk( trackSelection.updateSelectedTrack( playbackPositionUs, bufferedDurationUs, timeToLiveEdgeUs, queue, mediaChunkIterators); int selectedTrackIndex = trackSelection.getSelectedIndexInTrackGroup(); - boolean switchingTrack = oldTrackIndex != selectedTrackIndex; Uri selectedPlaylistUrl = playlistUrls[selectedTrackIndex]; if (!playlistTracker.isSnapshotValid(selectedPlaylistUrl)) { @@ -275,10 +351,11 @@ public void getNextChunk( // Retry when playlist is refreshed. return; } + @Nullable HlsMediaPlaylist mediaPlaylist = playlistTracker.getPlaylistSnapshot(selectedPlaylistUrl, /* isForPlayback= */ true); // playlistTracker snapshot is valid (checked by if() above), so mediaPlaylist must be non-null. - Assertions.checkNotNull(mediaPlaylist); + checkNotNull(mediaPlaylist); independentSegments = mediaPlaylist.hasIndependentSegments; updateLiveEdgeTimeUs(mediaPlaylist); @@ -286,22 +363,33 @@ public void getNextChunk( // Select the chunk. long startOfPlaylistInPeriodUs = mediaPlaylist.startTimeUs - playlistTracker.getInitialStartTimeUs(); - long chunkMediaSequence = - getChunkMediaSequence( + Pair nextMediaSequenceAndPartIndex = + getNextMediaSequenceAndPartIndex( previous, switchingTrack, mediaPlaylist, startOfPlaylistInPeriodUs, loadPositionUs); + long chunkMediaSequence = nextMediaSequenceAndPartIndex.first; + int partIndex = nextMediaSequenceAndPartIndex.second; if (chunkMediaSequence < mediaPlaylist.mediaSequence && previous != null && switchingTrack) { - // We try getting the next chunk without adapting in case that's the reason for falling - // behind the live window. - selectedTrackIndex = oldTrackIndex; - selectedPlaylistUrl = playlistUrls[selectedTrackIndex]; + // We try getting the next chunk without adapting in case that's the reason for falling + // behind the live window. + selectedTrackIndex = oldTrackIndex; + selectedPlaylistUrl = playlistUrls[selectedTrackIndex]; mediaPlaylist = playlistTracker.getPlaylistSnapshot(selectedPlaylistUrl, /* isForPlayback= */ true); // playlistTracker snapshot is valid (checked by if() above), so mediaPlaylist must be // non-null. - Assertions.checkNotNull(mediaPlaylist); - startOfPlaylistInPeriodUs = - mediaPlaylist.startTimeUs - playlistTracker.getInitialStartTimeUs(); - chunkMediaSequence = previous.getNextChunkIndex(); + checkNotNull(mediaPlaylist); + startOfPlaylistInPeriodUs = + mediaPlaylist.startTimeUs - playlistTracker.getInitialStartTimeUs(); + // Get the next segment/part without switching tracks. + Pair nextMediaSequenceAndPartIndexWithoutAdapting = + getNextMediaSequenceAndPartIndex( + previous, + /* switchingTrack= */ false, + mediaPlaylist, + startOfPlaylistInPeriodUs, + loadPositionUs); + chunkMediaSequence = nextMediaSequenceAndPartIndexWithoutAdapting.first; + partIndex = nextMediaSequenceAndPartIndexWithoutAdapting.second; } if (chunkMediaSequence < mediaPlaylist.mediaSequence) { @@ -309,41 +397,46 @@ public void getNextChunk( return; } - int segmentIndexInPlaylist = (int) (chunkMediaSequence - mediaPlaylist.mediaSequence); - int availableSegmentCount = mediaPlaylist.segments.size(); - if (segmentIndexInPlaylist >= availableSegmentCount) { - if (mediaPlaylist.hasEndTag) { - if (allowEndOfStream || availableSegmentCount == 0) { - out.endOfStream = true; - return; - } - segmentIndexInPlaylist = availableSegmentCount - 1; - } else /* Live */ { + @Nullable + SegmentBaseHolder segmentBaseHolder = + getNextSegmentHolder(mediaPlaylist, chunkMediaSequence, partIndex); + if (segmentBaseHolder == null) { + if (!mediaPlaylist.hasEndTag) { + // Reload the playlist in case of a live stream. out.playlistUrl = selectedPlaylistUrl; seenExpectedPlaylistError &= selectedPlaylistUrl.equals(expectedPlaylistUrl); expectedPlaylistUrl = selectedPlaylistUrl; return; + } else if (allowEndOfStream || mediaPlaylist.segments.isEmpty()) { + out.endOfStream = true; + return; } + // Use the last segment available in case of a VOD stream. + segmentBaseHolder = + new SegmentBaseHolder( + Iterables.getLast(mediaPlaylist.segments), + mediaPlaylist.mediaSequence + mediaPlaylist.segments.size() - 1, + /* partIndex= */ C.INDEX_UNSET); } - // We have a valid playlist snapshot, we can discard any playlist errors at this point. + + // We have a valid media segment, we can discard any playlist errors at this point. seenExpectedPlaylistError = false; expectedPlaylistUrl = null; - // Handle encryption. - HlsMediaPlaylist.Segment segment = mediaPlaylist.segments.get(segmentIndexInPlaylist); - - // Check if the segment or its initialization segment are fully encrypted. - Uri initSegmentKeyUri = getFullEncryptionKeyUri(mediaPlaylist, segment.initializationSegment); + // Check if the media segment or its initialization segment are fully encrypted. + @Nullable + Uri initSegmentKeyUri = + getFullEncryptionKeyUri(mediaPlaylist, segmentBaseHolder.segmentBase.initializationSegment); out.chunk = maybeCreateEncryptionChunkFor(initSegmentKeyUri, selectedTrackIndex); if (out.chunk != null) { return; } - Uri mediaSegmentKeyUri = getFullEncryptionKeyUri(mediaPlaylist, segment); + @Nullable + Uri mediaSegmentKeyUri = getFullEncryptionKeyUri(mediaPlaylist, segmentBaseHolder.segmentBase); out.chunk = maybeCreateEncryptionChunkFor(mediaSegmentKeyUri, selectedTrackIndex); if (out.chunk != null) { return; } - out.chunk = HlsMediaChunk.createInstance( extractorFactory, @@ -351,7 +444,7 @@ public void getNextChunk( playlistFormats[selectedTrackIndex], startOfPlaylistInPeriodUs, mediaPlaylist, - segmentIndexInPlaylist, + segmentBaseHolder, selectedPlaylistUrl, muxedCaptionFormats, trackSelection.getSelectionReason(), @@ -363,6 +456,41 @@ public void getNextChunk( /* initSegmentKey= */ keyCache.get(initSegmentKeyUri)); } + @Nullable + private static SegmentBaseHolder getNextSegmentHolder( + HlsMediaPlaylist mediaPlaylist, long nextMediaSequence, int nextPartIndex) { + int segmentIndexInPlaylist = (int) (nextMediaSequence - mediaPlaylist.mediaSequence); + if (segmentIndexInPlaylist == mediaPlaylist.segments.size()) { + int index = nextPartIndex != C.INDEX_UNSET ? nextPartIndex : 0; + return index < mediaPlaylist.trailingParts.size() + ? new SegmentBaseHolder(mediaPlaylist.trailingParts.get(index), nextMediaSequence, index) + : null; + } + + Segment mediaSegment = mediaPlaylist.segments.get(segmentIndexInPlaylist); + if (nextPartIndex == C.INDEX_UNSET) { + return new SegmentBaseHolder(mediaSegment, nextMediaSequence, /* partIndex= */ C.INDEX_UNSET); + } + + if (nextPartIndex < mediaSegment.parts.size()) { + // The requested part is available in the requested segment. + return new SegmentBaseHolder( + mediaSegment.parts.get(nextPartIndex), nextMediaSequence, nextPartIndex); + } else if (segmentIndexInPlaylist + 1 < mediaPlaylist.segments.size()) { + // The first part of the next segment is requested, but we can use the next full segment. + return new SegmentBaseHolder( + mediaPlaylist.segments.get(segmentIndexInPlaylist + 1), + nextMediaSequence + 1, + /* partIndex= */ C.INDEX_UNSET); + } else if (!mediaPlaylist.trailingParts.isEmpty()) { + // The part index is rolling over to the first trailing part. + return new SegmentBaseHolder( + mediaPlaylist.trailingParts.get(0), nextMediaSequence + 1, /* partIndex= */ 0); + } + // End of stream. + return null; + } + /** * Called when the {@link HlsSampleStreamWrapper} has finished loading a chunk obtained from this * source. @@ -373,8 +501,7 @@ public void onChunkLoadCompleted(Chunk chunk) { if (chunk instanceof EncryptionKeyChunk) { EncryptionKeyChunk encryptionKeyChunk = (EncryptionKeyChunk) chunk; scratchSpace = encryptionKeyChunk.getDataHolder(); - keyCache.put( - encryptionKeyChunk.dataSpec.uri, Assertions.checkNotNull(encryptionKeyChunk.getResult())); + keyCache.put(encryptionKeyChunk.dataSpec.uri, checkNotNull(encryptionKeyChunk.getResult())); } } @@ -438,23 +565,24 @@ public MediaChunkIterator[] createMediaChunkIterators( chunkIterators[i] = MediaChunkIterator.EMPTY; continue; } + @Nullable HlsMediaPlaylist playlist = playlistTracker.getPlaylistSnapshot(playlistUrl, /* isForPlayback= */ false); // Playlist snapshot is valid (checked by if() above) so playlist must be non-null. - Assertions.checkNotNull(playlist); + checkNotNull(playlist); long startOfPlaylistInPeriodUs = playlist.startTimeUs - playlistTracker.getInitialStartTimeUs(); boolean switchingTrack = trackIndex != oldTrackIndex; - long chunkMediaSequence = - getChunkMediaSequence( + Pair chunkMediaSequenceAndPartIndex = + getNextMediaSequenceAndPartIndex( previous, switchingTrack, playlist, startOfPlaylistInPeriodUs, loadPositionUs); - if (chunkMediaSequence < playlist.mediaSequence) { - chunkIterators[i] = MediaChunkIterator.EMPTY; - continue; - } - int chunkIndex = (int) (chunkMediaSequence - playlist.mediaSequence); + long chunkMediaSequence = chunkMediaSequenceAndPartIndex.first; + int partIndex = chunkMediaSequenceAndPartIndex.second; chunkIterators[i] = - new HlsMediaPlaylistSegmentIterator(playlist, startOfPlaylistInPeriodUs, chunkIndex); + new HlsMediaPlaylistSegmentIterator( + playlist.baseUri, + startOfPlaylistInPeriodUs, + getSegmentBaseList(playlist, chunkMediaSequence, partIndex)); } return chunkIterators; } @@ -495,10 +623,56 @@ public boolean shouldCancelLoad( return trackSelection.shouldCancelChunkLoad(playbackPositionUs, loadingChunk, queue); } + // Package methods. + + /** + * Returns a list with all segment bases in the playlist starting from {@code mediaSequence} and + * {@code partIndex} in the given playlist. The list may be empty if the starting point is not in + * the playlist. + */ + @VisibleForTesting + /* package */ static List getSegmentBaseList( + HlsMediaPlaylist playlist, long mediaSequence, int partIndex) { + int firstSegmentIndexInPlaylist = (int) (mediaSequence - playlist.mediaSequence); + if (firstSegmentIndexInPlaylist < 0 || playlist.segments.size() < firstSegmentIndexInPlaylist) { + // The first media sequence is not in the playlist. + return ImmutableList.of(); + } + List segmentBases = new ArrayList<>(); + if (firstSegmentIndexInPlaylist < playlist.segments.size()) { + if (partIndex != C.INDEX_UNSET) { + // The iterator starts with a part that belongs to a segment. + Segment firstSegment = playlist.segments.get(firstSegmentIndexInPlaylist); + if (partIndex == 0) { + // Use the full segment instead of the first part. + segmentBases.add(firstSegment); + } else if (partIndex < firstSegment.parts.size()) { + // Add the parts from the first requested segment. + segmentBases.addAll(firstSegment.parts.subList(partIndex, firstSegment.parts.size())); + } + firstSegmentIndexInPlaylist++; + } + partIndex = 0; + // Add all remaining segments. + segmentBases.addAll( + playlist.segments.subList(firstSegmentIndexInPlaylist, playlist.segments.size())); + } + + if (playlist.partTargetDurationUs != C.TIME_UNSET) { + // That's a low latency playlist. + partIndex = partIndex == C.INDEX_UNSET ? 0 : partIndex; + if (partIndex < playlist.trailingParts.size()) { + segmentBases.addAll( + playlist.trailingParts.subList(partIndex, playlist.trailingParts.size())); + } + } + return Collections.unmodifiableList(segmentBases); + } + // Private methods. /** - * Returns the media sequence number of the segment to load next in {@code mediaPlaylist}. + * Returns the media sequence number and part index to load next in the {@code mediaPlaylist}. * * @param previous The last (at least partially) loaded segment. * @param switchingTrack Whether the segment to load is not preceded by a segment in the same @@ -507,9 +681,9 @@ public boolean shouldCancelLoad( * @param startOfPlaylistInPeriodUs The start of {@code mediaPlaylist} relative to the period * start in microseconds. * @param loadPositionUs The current load position relative to the period start in microseconds. - * @return The media sequence of the segment to load. + * @return The media sequence and part index to load. */ - private long getChunkMediaSequence( + private Pair getNextMediaSequenceAndPartIndex( @Nullable HlsMediaChunk previous, boolean switchingTrack, HlsMediaPlaylist mediaPlaylist, @@ -521,17 +695,48 @@ private long getChunkMediaSequence( (previous == null || independentSegments) ? loadPositionUs : previous.startTimeUs; if (!mediaPlaylist.hasEndTag && targetPositionInPeriodUs >= endOfPlaylistInPeriodUs) { // If the playlist is too old to contain the chunk, we need to refresh it. - return mediaPlaylist.mediaSequence + mediaPlaylist.segments.size(); + return new Pair<>( + mediaPlaylist.mediaSequence + mediaPlaylist.segments.size(), + /* partIndex */ C.INDEX_UNSET); } long targetPositionInPlaylistUs = targetPositionInPeriodUs - startOfPlaylistInPeriodUs; - return Util.binarySearchFloor( + int segmentIndexInPlaylist = + Util.binarySearchFloor( mediaPlaylist.segments, /* value= */ targetPositionInPlaylistUs, /* inclusive= */ true, - /* stayInBounds= */ !playlistTracker.isLive() || previous == null) - + mediaPlaylist.mediaSequence; - } - return previous.isLoadCompleted() ? previous.getNextChunkIndex() : previous.chunkIndex; + /* stayInBounds= */ !playlistTracker.isLive() || previous == null); + long mediaSequence = segmentIndexInPlaylist + mediaPlaylist.mediaSequence; + int partIndex = C.INDEX_UNSET; + if (segmentIndexInPlaylist >= 0) { + // In case we are inside the live window, we try to pick a part if available. + Segment segment = mediaPlaylist.segments.get(segmentIndexInPlaylist); + List parts = + targetPositionInPlaylistUs < segment.relativeStartTimeUs + segment.durationUs + ? segment.parts + : mediaPlaylist.trailingParts; + for (int i = 0; i < parts.size(); i++) { + HlsMediaPlaylist.Part part = parts.get(i); + if (targetPositionInPlaylistUs < part.relativeStartTimeUs + part.durationUs) { + if (part.isIndependent) { + partIndex = i; + // Increase media sequence by one if the part is a trailing part. + mediaSequence += parts == mediaPlaylist.trailingParts ? 1 : 0; + } + break; + } + } + } + return new Pair<>(mediaSequence, partIndex); + } + // If loading has not completed, we return the previous chunk again. + return (previous.isLoadCompleted() + ? new Pair<>( + previous.partIndex == C.INDEX_UNSET + ? previous.getNextChunkIndex() + : previous.chunkIndex, + previous.partIndex == C.INDEX_UNSET ? C.INDEX_UNSET : previous.partIndex + 1) + : new Pair<>(previous.chunkIndex, previous.partIndex)); } private long resolveTimeToLiveEdgeUs(long playbackPositionUs) { @@ -574,18 +779,38 @@ private Chunk maybeCreateEncryptionChunkFor(@Nullable Uri keyUri, int selectedTr } @Nullable - private static Uri getFullEncryptionKeyUri(HlsMediaPlaylist playlist, @Nullable Segment segment) { - if (segment == null || segment.fullSegmentEncryptionKeyUri == null) { + private static Uri getFullEncryptionKeyUri( + HlsMediaPlaylist playlist, @Nullable HlsMediaPlaylist.SegmentBase segmentBase) { + if (segmentBase == null || segmentBase.fullSegmentEncryptionKeyUri == null) { return null; } - return UriUtil.resolveToUri(playlist.baseUri, segment.fullSegmentEncryptionKeyUri); + return UriUtil.resolveToUri(playlist.baseUri, segmentBase.fullSegmentEncryptionKeyUri); + } + + // Package classes. + + /* package */ static final class SegmentBaseHolder { + + public final HlsMediaPlaylist.SegmentBase segmentBase; + public final long mediaSequence; + public final int partIndex; + public final boolean isPreload; + + /** Creates a new instance. */ + public SegmentBaseHolder( + HlsMediaPlaylist.SegmentBase segmentBase, long mediaSequence, int partIndex) { + this.segmentBase = segmentBase; + this.mediaSequence = mediaSequence; + this.partIndex = partIndex; + this.isPreload = + segmentBase instanceof HlsMediaPlaylist.Part + && ((HlsMediaPlaylist.Part) segmentBase).isPreload; + } } // Private classes. - /** - * A {@link TrackSelection} to use for initialization. - */ + /** A {@link ExoTrackSelection} to use for initialization. */ private static final class InitializationTrackSelection extends BaseTrackSelection { private int selectedIndex; @@ -665,48 +890,52 @@ public byte[] getResult() { } - /** {@link MediaChunkIterator} wrapping a {@link HlsMediaPlaylist}. */ - private static final class HlsMediaPlaylistSegmentIterator extends BaseMediaChunkIterator { + @VisibleForTesting + /* package */ static final class HlsMediaPlaylistSegmentIterator extends BaseMediaChunkIterator { - private final HlsMediaPlaylist playlist; + private final List segmentBases; private final long startOfPlaylistInPeriodUs; + private final String playlistBaseUri; /** - * Creates iterator. + * Creates an iterator instance wrapping a list of {@link HlsMediaPlaylist.SegmentBase}. * - * @param playlist The {@link HlsMediaPlaylist} to wrap. + * @param playlistBaseUri The base URI of the {@link HlsMediaPlaylist}. * @param startOfPlaylistInPeriodUs The start time of the playlist in the period, in * microseconds. - * @param chunkIndex The index of the first available chunk in the playlist. + * @param segmentBases The list of {@link HlsMediaPlaylist.SegmentBase segment bases} to wrap. */ public HlsMediaPlaylistSegmentIterator( - HlsMediaPlaylist playlist, long startOfPlaylistInPeriodUs, int chunkIndex) { - super(/* fromIndex= */ chunkIndex, /* toIndex= */ playlist.segments.size() - 1); - this.playlist = playlist; + String playlistBaseUri, + long startOfPlaylistInPeriodUs, + List segmentBases) { + super(/* fromIndex= */ 0, segmentBases.size() - 1); + this.playlistBaseUri = playlistBaseUri; this.startOfPlaylistInPeriodUs = startOfPlaylistInPeriodUs; + this.segmentBases = segmentBases; } @Override public DataSpec getDataSpec() { checkInBounds(); - Segment segment = playlist.segments.get((int) getCurrentIndex()); - Uri chunkUri = UriUtil.resolveToUri(playlist.baseUri, segment.url); - return new DataSpec(chunkUri, segment.byteRangeOffset, segment.byteRangeLength); + HlsMediaPlaylist.SegmentBase segmentBase = segmentBases.get((int) getCurrentIndex()); + Uri chunkUri = UriUtil.resolveToUri(playlistBaseUri, segmentBase.url); + return new DataSpec(chunkUri, segmentBase.byteRangeOffset, segmentBase.byteRangeLength); } @Override public long getChunkStartTimeUs() { checkInBounds(); - Segment segment = playlist.segments.get((int) getCurrentIndex()); - return startOfPlaylistInPeriodUs + segment.relativeStartTimeUs; + return startOfPlaylistInPeriodUs + + segmentBases.get((int) getCurrentIndex()).relativeStartTimeUs; } @Override public long getChunkEndTimeUs() { checkInBounds(); - Segment segment = playlist.segments.get((int) getCurrentIndex()); - long segmentStartTimeInPeriodUs = startOfPlaylistInPeriodUs + segment.relativeStartTimeUs; - return segmentStartTimeInPeriodUs + segment.durationUs; + HlsMediaPlaylist.SegmentBase segmentBase = segmentBases.get((int) getCurrentIndex()); + long segmentStartTimeInPeriodUs = startOfPlaylistInPeriodUs + segmentBase.relativeStartTimeUs; + return segmentStartTimeInPeriodUs + segmentBase.durationUs; } } } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaChunk.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaChunk.java index 8de69a68d1d..30e8350982b 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaChunk.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaChunk.java @@ -15,6 +15,8 @@ */ package com.google.android.exoplayer2.source.hls; +import static com.google.android.exoplayer2.upstream.DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED; + import android.net.Uri; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; @@ -59,7 +61,7 @@ * @param format The chunk format. * @param startOfPlaylistInPeriodUs The position of the playlist in the period in microseconds. * @param mediaPlaylist The media playlist from which this chunk was obtained. - * @param segmentIndexInPlaylist The index of the segment in the media playlist. + * @param segmentBaseHolder The segment holder. * @param playlistUrl The url of the playlist from which this chunk was obtained. * @param muxedCaptionFormats List of muxed caption {@link Format}s. Null if no closed caption * information is available in the master playlist. @@ -79,7 +81,7 @@ public static HlsMediaChunk createInstance( Format format, long startOfPlaylistInPeriodUs, HlsMediaPlaylist mediaPlaylist, - int segmentIndexInPlaylist, + HlsChunkSource.SegmentBaseHolder segmentBaseHolder, Uri playlistUrl, @Nullable List muxedCaptionFormats, int trackSelectionReason, @@ -90,12 +92,14 @@ public static HlsMediaChunk createInstance( @Nullable byte[] mediaSegmentKey, @Nullable byte[] initSegmentKey) { // Media segment. - HlsMediaPlaylist.Segment mediaSegment = mediaPlaylist.segments.get(segmentIndexInPlaylist); + HlsMediaPlaylist.SegmentBase mediaSegment = segmentBaseHolder.segmentBase; DataSpec dataSpec = - new DataSpec( - UriUtil.resolveToUri(mediaPlaylist.baseUri, mediaSegment.url), - mediaSegment.byteRangeOffset, - mediaSegment.byteRangeLength); + new DataSpec.Builder() + .setUri(UriUtil.resolveToUri(mediaPlaylist.baseUri, mediaSegment.url)) + .setPosition(mediaSegment.byteRangeOffset) + .setLength(mediaSegment.byteRangeLength) + .setFlags(segmentBaseHolder.isPreload ? FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED : 0) + .build(); boolean mediaSegmentEncrypted = mediaSegmentKey != null; @Nullable byte[] mediaSegmentIv = @@ -136,10 +140,10 @@ public static HlsMediaChunk createInstance( playlistUrl.equals(previousChunk.playlistUrl) && previousChunk.loadCompleted; id3Decoder = previousChunk.id3Decoder; scratchId3Data = previousChunk.scratchId3Data; + boolean isIndependent = isIndependent(segmentBaseHolder, mediaPlaylist); boolean canContinueWithoutSplice = isFollowingChunk - || (mediaPlaylist.hasIndependentSegments - && segmentStartTimeInPeriodUs >= previousChunk.endTimeUs); + || (isIndependent && segmentStartTimeInPeriodUs >= previousChunk.endTimeUs); shouldSpliceIn = !canContinueWithoutSplice; previousExtractor = isFollowingChunk @@ -152,7 +156,6 @@ public static HlsMediaChunk createInstance( scratchId3Data = new ParsableByteArray(Id3Decoder.ID3_HEADER_LENGTH); shouldSpliceIn = false; } - return new HlsMediaChunk( extractorFactory, mediaDataSource, @@ -168,7 +171,9 @@ public static HlsMediaChunk createInstance( trackSelectionData, segmentStartTimeInPeriodUs, segmentEndTimeInPeriodUs, - /* chunkMediaSequence= */ mediaPlaylist.mediaSequence + segmentIndexInPlaylist, + segmentBaseHolder.mediaSequence, + segmentBaseHolder.partIndex, + /* isPublished= */ !segmentBaseHolder.isPreload, discontinuitySequenceNumber, mediaSegment.hasGapTag, isMasterTimestampSource, @@ -201,6 +206,9 @@ public static HlsMediaChunk createInstance( /** Whether samples for this chunk should be spliced into existing samples. */ public final boolean shouldSpliceIn; + /** The part index or {@link C#INDEX_UNSET} if the chunk is a full segment */ + public final int partIndex; + @Nullable private final DataSource initDataSource; @Nullable private final DataSpec initDataSpec; @Nullable private final HlsMediaChunkExtractor previousExtractor; @@ -226,6 +234,7 @@ public static HlsMediaChunk createInstance( private boolean loadCompleted; private ImmutableList sampleQueueFirstSampleIndices; private boolean extractorInvalidated; + private boolean isPublished; private HlsMediaChunk( HlsExtractorFactory extractorFactory, @@ -243,6 +252,8 @@ private HlsMediaChunk( long startTimeUs, long endTimeUs, long chunkMediaSequence, + int partIndex, + boolean isPublished, int discontinuitySequenceNumber, boolean hasGapTag, boolean isMasterTimestampSource, @@ -262,6 +273,8 @@ private HlsMediaChunk( endTimeUs, chunkMediaSequence); this.mediaSegmentEncrypted = mediaSegmentEncrypted; + this.partIndex = partIndex; + this.isPublished = isPublished; this.discontinuitySequenceNumber = discontinuitySequenceNumber; this.initDataSpec = initDataSpec; this.initDataSource = initDataSource; @@ -345,6 +358,22 @@ public void load() throws IOException { } } + /** + * Whether the chunk is a published chunk as opposed to a preload hint that may change when the + * playlist updates. + */ + public boolean isPublished() { + return isPublished; + } + + /** + * Sets the publish flag of the media chunk to indicate that it is not based on a part that is a + * preload hint in the playlist. + */ + public void publish() { + isPublished = true; + } + // Internal methods. @RequiresNonNull("output") @@ -470,12 +499,12 @@ private DefaultExtractorInput prepareExtraction(DataSource dataSource, DataSpec private long peekId3PrivTimestamp(ExtractorInput input) throws IOException { input.resetPeekPosition(); try { + scratchId3Data.reset(Id3Decoder.ID3_HEADER_LENGTH); input.peekFully(scratchId3Data.getData(), 0, Id3Decoder.ID3_HEADER_LENGTH); } catch (EOFException e) { // The input isn't long enough for there to be any ID3 data. return C.TIME_UNSET; } - scratchId3Data.reset(Id3Decoder.ID3_HEADER_LENGTH); int id = scratchId3Data.readUnsignedInt24(); if (id != Id3Decoder.ID3_TAG) { return C.TIME_UNSET; @@ -501,7 +530,8 @@ private long peekId3PrivTimestamp(ExtractorInput input) throws IOException { if (PRIV_TIMESTAMP_FRAME_OWNER.equals(privFrame.owner)) { System.arraycopy( privFrame.privateData, 0, scratchId3Data.getData(), 0, 8 /* timestamp size */); - scratchId3Data.reset(8); + scratchId3Data.setPosition(0); + scratchId3Data.setLimit(8); // The top 31 bits should be zeros, but explicitly zero them to wrap in the case that the // streaming provider forgot. See: https://github.com/google/ExoPlayer/pull/3495. return scratchId3Data.readLong() & 0x1FFFFFFFFL; @@ -549,4 +579,13 @@ private static DataSource buildDataSource( } return dataSource; } + + private static boolean isIndependent( + HlsChunkSource.SegmentBaseHolder segmentBaseHolder, HlsMediaPlaylist mediaPlaylist) { + if (segmentBaseHolder.segmentBase instanceof HlsMediaPlaylist.Part) { + return ((HlsMediaPlaylist.Part) segmentBaseHolder.segmentBase).isIndependent + || (segmentBaseHolder.partIndex == 0 && mediaPlaylist.hasIndependentSegments); + } + return mediaPlaylist.hasIndependentSegments; + } } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaPeriod.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaPeriod.java index 0089f68bf45..9d643ea9264 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaPeriod.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaPeriod.java @@ -39,7 +39,7 @@ import com.google.android.exoplayer2.source.hls.playlist.HlsMasterPlaylist.Rendition; import com.google.android.exoplayer2.source.hls.playlist.HlsMasterPlaylist.Variant; import com.google.android.exoplayer2.source.hls.playlist.HlsPlaylistTracker; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy; @@ -177,7 +177,7 @@ public TrackGroupArray getTrackGroups() { // null URLs, this method must be updated to calculate stream keys that are compatible with those // that may already be persisted for offline. @Override - public List getStreamKeys(List trackSelections) { + public List getStreamKeys(List trackSelections) { // See HlsMasterPlaylist.copy for interpretation of StreamKeys. HlsMasterPlaylist masterPlaylist = Assertions.checkNotNull(playlistTracker.getMasterPlaylist()); boolean hasVariants = !masterPlaylist.variants.isEmpty(); @@ -202,7 +202,7 @@ public List getStreamKeys(List trackSelections) { List streamKeys = new ArrayList<>(); boolean needsPrimaryTrackGroupSelection = false; boolean hasPrimaryTrackGroupSelection = false; - for (TrackSelection trackSelection : trackSelections) { + for (ExoTrackSelection trackSelection : trackSelections) { TrackGroup trackSelectionGroup = trackSelection.getTrackGroup(); int mainWrapperTrackGroupIndex = mainWrapperTrackGroups.indexOf(trackSelectionGroup); if (mainWrapperTrackGroupIndex != C.INDEX_UNSET) { @@ -258,7 +258,7 @@ public List getStreamKeys(List trackSelections) { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, @@ -286,7 +286,7 @@ public long selectTracks( // Select tracks for each child, copying the resulting streams back into a new streams array. SampleStream[] newStreams = new SampleStream[selections.length]; @NullableType SampleStream[] childStreams = new SampleStream[selections.length]; - @NullableType TrackSelection[] childSelections = new TrackSelection[selections.length]; + @NullableType ExoTrackSelection[] childSelections = new ExoTrackSelection[selections.length]; int newEnabledSampleStreamWrapperCount = 0; HlsSampleStreamWrapper[] newEnabledSampleStreamWrappers = new HlsSampleStreamWrapper[sampleStreamWrappers.length]; @@ -445,6 +445,9 @@ public void onContinueLoadingRequested(HlsSampleStreamWrapper sampleStreamWrappe @Override public void onPlaylistChanged() { + for (HlsSampleStreamWrapper streamWrapper : sampleStreamWrappers) { + streamWrapper.onPlaylistUpdated(); + } callback.onContinueLoadingRequested(this); } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaSource.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaSource.java index b58f3da928f..e5c233ef434 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaSource.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsMediaSource.java @@ -16,18 +16,20 @@ package com.google.android.exoplayer2.source.hls; import static com.google.android.exoplayer2.util.Assertions.checkNotNull; -import static java.lang.Math.max; import static java.lang.annotation.RetentionPolicy.SOURCE; import android.net.Uri; -import android.os.Handler; +import android.os.SystemClock; import androidx.annotation.IntDef; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.drm.DefaultDrmSessionManagerProvider; import com.google.android.exoplayer2.drm.DrmSessionEventListener; import com.google.android.exoplayer2.drm.DrmSessionManager; +import com.google.android.exoplayer2.drm.DrmSessionManagerProvider; import com.google.android.exoplayer2.extractor.Extractor; import com.google.android.exoplayer2.offline.StreamKey; import com.google.android.exoplayer2.source.BaseMediaSource; @@ -35,7 +37,6 @@ import com.google.android.exoplayer2.source.DefaultCompositeSequenceableLoaderFactory; import com.google.android.exoplayer2.source.MediaPeriod; import com.google.android.exoplayer2.source.MediaSource; -import com.google.android.exoplayer2.source.MediaSourceDrmHelper; import com.google.android.exoplayer2.source.MediaSourceEventListener; import com.google.android.exoplayer2.source.MediaSourceFactory; import com.google.android.exoplayer2.source.SequenceableLoader; @@ -94,19 +95,20 @@ public final class HlsMediaSource extends BaseMediaSource public static final class Factory implements MediaSourceFactory { private final HlsDataSourceFactory hlsDataSourceFactory; - private final MediaSourceDrmHelper mediaSourceDrmHelper; private HlsExtractorFactory extractorFactory; private HlsPlaylistParserFactory playlistParserFactory; private HlsPlaylistTracker.Factory playlistTrackerFactory; private CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory; - @Nullable private DrmSessionManager drmSessionManager; + private boolean usingCustomDrmSessionManagerProvider; + private DrmSessionManagerProvider drmSessionManagerProvider; private LoadErrorHandlingPolicy loadErrorHandlingPolicy; private boolean allowChunklessPreparation; @MetadataType private int metadataType; private boolean useSessionKeys; private List streamKeys; @Nullable private Object tag; + private long elapsedRealTimeOffsetMs; /** * Creates a new factory for {@link HlsMediaSource}s. @@ -127,7 +129,7 @@ public Factory(DataSource.Factory dataSourceFactory) { */ public Factory(HlsDataSourceFactory hlsDataSourceFactory) { this.hlsDataSourceFactory = checkNotNull(hlsDataSourceFactory); - mediaSourceDrmHelper = new MediaSourceDrmHelper(); + drmSessionManagerProvider = new DefaultDrmSessionManagerProvider(); playlistParserFactory = new DefaultHlsPlaylistParserFactory(); playlistTrackerFactory = DefaultHlsPlaylistTracker.FACTORY; extractorFactory = HlsExtractorFactory.DEFAULT; @@ -135,6 +137,7 @@ public Factory(HlsDataSourceFactory hlsDataSourceFactory) { compositeSequenceableLoaderFactory = new DefaultCompositeSequenceableLoaderFactory(); metadataType = METADATA_TYPE_ID3; streamKeys = Collections.emptyList(); + elapsedRealTimeOffsetMs = C.TIME_UNSET; } /** @@ -165,8 +168,6 @@ public Factory setExtractorFactory(@Nullable HlsExtractorFactory extractorFactor * Sets the {@link LoadErrorHandlingPolicy}. The default value is created by calling {@link * DefaultLoadErrorHandlingPolicy#DefaultLoadErrorHandlingPolicy()}. * - *

    Calling this method overrides any calls to {@link #setMinLoadableRetryCount(int)}. - * * @param loadErrorHandlingPolicy A {@link LoadErrorHandlingPolicy}. * @return This factory, for convenience. */ @@ -179,13 +180,6 @@ public Factory setLoadErrorHandlingPolicy( return this; } - /** @deprecated Use {@link #setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)} instead. */ - @Deprecated - public Factory setMinLoadableRetryCount(int minLoadableRetryCount) { - this.loadErrorHandlingPolicy = new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount); - return this; - } - /** * Sets the factory from which playlist parsers will be obtained. The default value is a {@link * DefaultHlsPlaylistParserFactory}. @@ -287,22 +281,44 @@ public Factory setUseSessionKeys(boolean useSessionKeys) { return this; } + @Override + public Factory setDrmSessionManagerProvider( + @Nullable DrmSessionManagerProvider drmSessionManagerProvider) { + if (drmSessionManagerProvider != null) { + this.drmSessionManagerProvider = drmSessionManagerProvider; + this.usingCustomDrmSessionManagerProvider = true; + } else { + this.drmSessionManagerProvider = new DefaultDrmSessionManagerProvider(); + this.usingCustomDrmSessionManagerProvider = false; + } + return this; + } + @Override public Factory setDrmSessionManager(@Nullable DrmSessionManager drmSessionManager) { - this.drmSessionManager = drmSessionManager; + if (drmSessionManager == null) { + setDrmSessionManagerProvider(null); + } else { + setDrmSessionManagerProvider(unusedMediaItem -> drmSessionManager); + } return this; } @Override public Factory setDrmHttpDataSourceFactory( @Nullable HttpDataSource.Factory drmHttpDataSourceFactory) { - mediaSourceDrmHelper.setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + if (!usingCustomDrmSessionManagerProvider) { + ((DefaultDrmSessionManagerProvider) drmSessionManagerProvider) + .setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + } return this; } @Override - public MediaSourceFactory setDrmUserAgent(@Nullable String userAgent) { - mediaSourceDrmHelper.setDrmUserAgent(userAgent); + public Factory setDrmUserAgent(@Nullable String userAgent) { + if (!usingCustomDrmSessionManagerProvider) { + ((DefaultDrmSessionManagerProvider) drmSessionManagerProvider).setDrmUserAgent(userAgent); + } return this; } @@ -319,20 +335,17 @@ public Factory setStreamKeys(@Nullable List streamKeys) { } /** - * @deprecated Use {@link #createMediaSource(MediaItem)} and {@link #addEventListener(Handler, - * MediaSourceEventListener)} instead. + * Sets the offset between {@link SystemClock#elapsedRealtime()} and the time since the Unix + * epoch. By default, is it set to {@link C#TIME_UNSET}. + * + * @param elapsedRealTimeOffsetMs The offset between {@link SystemClock#elapsedRealtime()} and + * the time since the Unix epoch, in milliseconds. + * @return This factory, for convenience. */ - @SuppressWarnings("deprecation") - @Deprecated - public HlsMediaSource createMediaSource( - Uri playlistUri, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - HlsMediaSource mediaSource = createMediaSource(playlistUri); - if (eventHandler != null && eventListener != null) { - mediaSource.addEventListener(eventHandler, eventListener); - } - return mediaSource; + @VisibleForTesting + /* package */ Factory setElapsedRealTimeOffsetMs(long elapsedRealTimeOffsetMs) { + this.elapsedRealTimeOffsetMs = elapsedRealTimeOffsetMs; + return this; } /** @deprecated Use {@link #createMediaSource(MediaItem)} instead. */ @@ -379,10 +392,11 @@ public HlsMediaSource createMediaSource(MediaItem mediaItem) { hlsDataSourceFactory, extractorFactory, compositeSequenceableLoaderFactory, - drmSessionManager != null ? drmSessionManager : mediaSourceDrmHelper.create(mediaItem), + drmSessionManagerProvider.get(mediaItem), loadErrorHandlingPolicy, playlistTrackerFactory.createTracker( hlsDataSourceFactory, loadErrorHandlingPolicy, playlistParserFactory), + elapsedRealTimeOffsetMs, allowChunklessPreparation, metadataType, useSessionKeys); @@ -395,7 +409,6 @@ public int[] getSupportedTypes() { } private final HlsExtractorFactory extractorFactory; - private final MediaItem mediaItem; private final MediaItem.PlaybackProperties playbackProperties; private final HlsDataSourceFactory dataSourceFactory; private final CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory; @@ -405,7 +418,10 @@ public int[] getSupportedTypes() { private final @MetadataType int metadataType; private final boolean useSessionKeys; private final HlsPlaylistTracker playlistTracker; + private final long elapsedRealTimeOffsetMs; + private final MediaItem mediaItem; + private MediaItem.LiveConfiguration liveConfiguration; @Nullable private TransferListener mediaTransferListener; private HlsMediaSource( @@ -416,17 +432,20 @@ private HlsMediaSource( DrmSessionManager drmSessionManager, LoadErrorHandlingPolicy loadErrorHandlingPolicy, HlsPlaylistTracker playlistTracker, + long elapsedRealTimeOffsetMs, boolean allowChunklessPreparation, @MetadataType int metadataType, boolean useSessionKeys) { this.playbackProperties = checkNotNull(mediaItem.playbackProperties); this.mediaItem = mediaItem; + this.liveConfiguration = mediaItem.liveConfiguration; this.dataSourceFactory = dataSourceFactory; this.extractorFactory = extractorFactory; this.compositeSequenceableLoaderFactory = compositeSequenceableLoaderFactory; this.drmSessionManager = drmSessionManager; this.loadErrorHandlingPolicy = loadErrorHandlingPolicy; this.playlistTracker = playlistTracker; + this.elapsedRealTimeOffsetMs = elapsedRealTimeOffsetMs; this.allowChunklessPreparation = allowChunklessPreparation; this.metadataType = metadataType; this.useSessionKeys = useSessionKeys; @@ -510,24 +529,26 @@ public void onPrimaryPlaylistRefreshed(HlsMediaPlaylist playlist) { HlsManifest manifest = new HlsManifest(checkNotNull(playlistTracker.getMasterPlaylist()), playlist); if (playlistTracker.isLive()) { + long liveEdgeOffsetUs = getLiveEdgeOffsetUs(playlist); + long targetLiveOffsetUs = + liveConfiguration.targetOffsetMs != C.TIME_UNSET + ? C.msToUs(liveConfiguration.targetOffsetMs) + : getTargetLiveOffsetUs(playlist, liveEdgeOffsetUs); + // Ensure target live offset is within the live window and greater than the live edge offset. + targetLiveOffsetUs = + Util.constrainValue( + targetLiveOffsetUs, liveEdgeOffsetUs, playlist.durationUs + liveEdgeOffsetUs); + maybeUpdateMediaItem(targetLiveOffsetUs); + long offsetFromInitialStartTimeUs = playlist.startTimeUs - playlistTracker.getInitialStartTimeUs(); long periodDurationUs = playlist.hasEndTag ? offsetFromInitialStartTimeUs + playlist.durationUs : C.TIME_UNSET; List segments = playlist.segments; - if (windowDefaultStartPositionUs == C.TIME_UNSET) { + if (!segments.isEmpty()) { + windowDefaultStartPositionUs = getWindowDefaultStartPosition(playlist, liveEdgeOffsetUs); + } else if (windowDefaultStartPositionUs == C.TIME_UNSET) { windowDefaultStartPositionUs = 0; - if (!segments.isEmpty()) { - int defaultStartSegmentIndex = max(0, segments.size() - 3); - // We attempt to set the default start position to be at least twice the target duration - // behind the live edge. - long minStartPositionUs = playlist.durationUs - playlist.targetDurationUs * 2; - while (defaultStartSegmentIndex > 0 - && segments.get(defaultStartSegmentIndex).relativeStartTimeUs > minStartPositionUs) { - defaultStartSegmentIndex--; - } - windowDefaultStartPositionUs = segments.get(defaultStartSegmentIndex).relativeStartTimeUs; - } } timeline = new SinglePeriodTimeline( @@ -540,9 +561,9 @@ public void onPrimaryPlaylistRefreshed(HlsMediaPlaylist playlist) { windowDefaultStartPositionUs, /* isSeekable= */ true, /* isDynamic= */ !playlist.hasEndTag, - /* isLive= */ true, manifest, - mediaItem); + mediaItem, + liveConfiguration); } else /* not live */ { if (windowDefaultStartPositionUs == C.TIME_UNSET) { windowDefaultStartPositionUs = 0; @@ -558,11 +579,52 @@ public void onPrimaryPlaylistRefreshed(HlsMediaPlaylist playlist) { windowDefaultStartPositionUs, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, manifest, - mediaItem); + mediaItem, + /* liveConfiguration= */ null); } refreshSourceInfo(timeline); } + private long getLiveEdgeOffsetUs(HlsMediaPlaylist playlist) { + return playlist.hasProgramDateTime + ? C.msToUs(Util.getNowUnixTimeMs(elapsedRealTimeOffsetMs)) - playlist.getEndTimeUs() + : 0; + } + + private long getWindowDefaultStartPosition(HlsMediaPlaylist playlist, long liveEdgeOffsetUs) { + List segments = playlist.segments; + int segmentIndex = segments.size() - 1; + long minStartPositionUs = + playlist.durationUs + liveEdgeOffsetUs - C.msToUs(liveConfiguration.targetOffsetMs); + while (segmentIndex > 0 + && segments.get(segmentIndex).relativeStartTimeUs > minStartPositionUs) { + segmentIndex--; + } + return segments.get(segmentIndex).relativeStartTimeUs; + } + + private void maybeUpdateMediaItem(long targetLiveOffsetUs) { + long targetLiveOffsetMs = C.usToMs(targetLiveOffsetUs); + if (targetLiveOffsetMs != liveConfiguration.targetOffsetMs) { + liveConfiguration = + mediaItem.buildUpon().setLiveTargetOffsetMs(targetLiveOffsetMs).build().liveConfiguration; + } + } + + private static long getTargetLiveOffsetUs(HlsMediaPlaylist playlist, long liveEdgeOffsetUs) { + HlsMediaPlaylist.ServerControl serverControl = playlist.serverControl; + // Select part hold back only if the playlist has a part target duration. + long offsetToEndOfPlaylistUs; + if (serverControl.partHoldBackUs != C.TIME_UNSET + && playlist.partTargetDurationUs != C.TIME_UNSET) { + offsetToEndOfPlaylistUs = serverControl.partHoldBackUs; + } else if (serverControl.holdBackUs != C.TIME_UNSET) { + offsetToEndOfPlaylistUs = serverControl.holdBackUs; + } else { + // Fallback, see RFC 8216, Section 4.4.3.8. + offsetToEndOfPlaylistUs = 3 * playlist.targetDurationUs; + } + return offsetToEndOfPlaylistUs + liveEdgeOffsetUs; + } } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsSampleStreamWrapper.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsSampleStreamWrapper.java index 2b1ec60607b..df1c598be56 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsSampleStreamWrapper.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsSampleStreamWrapper.java @@ -15,6 +15,8 @@ */ package com.google.android.exoplayer2.source.hls; +import static com.google.android.exoplayer2.source.hls.HlsChunkSource.CHUNK_PUBLICATION_STATE_PUBLISHED; +import static com.google.android.exoplayer2.source.hls.HlsChunkSource.CHUNK_PUBLICATION_STATE_REMOVED; import static java.lang.Math.max; import android.net.Uri; @@ -51,9 +53,10 @@ import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.source.chunk.Chunk; import com.google.android.exoplayer2.source.chunk.MediaChunkIterator; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.DataReader; +import com.google.android.exoplayer2.upstream.HttpDataSource; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy.LoadErrorInfo; import com.google.android.exoplayer2.upstream.Loader; @@ -328,7 +331,7 @@ public void unbindSampleQueue(int trackGroupIndex) { * part of the track selection. */ public boolean selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, @@ -355,11 +358,11 @@ public boolean selectTracks( : positionUs != lastSeekPositionUs); // Get the old (i.e. current before the loop below executes) primary track selection. The new // primary selection will equal the old one unless it's changed in the loop. - TrackSelection oldPrimaryTrackSelection = chunkSource.getTrackSelection(); - TrackSelection primaryTrackSelection = oldPrimaryTrackSelection; + ExoTrackSelection oldPrimaryTrackSelection = chunkSource.getTrackSelection(); + ExoTrackSelection primaryTrackSelection = oldPrimaryTrackSelection; // Select new tracks. for (int i = 0; i < selections.length; i++) { - TrackSelection selection = selections[i]; + ExoTrackSelection selection = selections[i]; if (selection == null) { continue; } @@ -504,6 +507,23 @@ public boolean seekToUs(long positionUs, boolean forceReset) { return true; } + /** Called when the playlist is updated. */ + public void onPlaylistUpdated() { + if (mediaChunks.isEmpty()) { + return; + } + HlsMediaChunk lastMediaChunk = Iterables.getLast(mediaChunks); + @HlsChunkSource.ChunkPublicationState + int chunkState = chunkSource.getChunkPublicationState(lastMediaChunk); + if (chunkState == CHUNK_PUBLICATION_STATE_PUBLISHED) { + lastMediaChunk.publish(); + } else if (chunkState == CHUNK_PUBLICATION_STATE_REMOVED + && !loadingFinished + && loader.isLoading()) { + loader.cancelLoading(); + } + } + public void release() { if (prepared) { // Discard as much as we can synchronously. We only do this if we're prepared, since otherwise @@ -576,6 +596,11 @@ && finishedReadingChunk(mediaChunks.get(discardToMediaChunkIndex))) { downstreamTrackFormat = trackFormat; } + if (!mediaChunks.isEmpty() && !mediaChunks.get(0).isPublished()) { + // Don't read into preload chunks until we can be sure they are permanently published. + return C.RESULT_NOTHING_READ; + } + int result = sampleQueues[sampleQueueIndex].read(formatHolder, buffer, requireFormat, loadingFinished); if (result == C.RESULT_FORMAT_READ) { @@ -605,6 +630,21 @@ public int skipData(int sampleQueueIndex, long positionUs) { SampleQueue sampleQueue = sampleQueues[sampleQueueIndex]; int skipCount = sampleQueue.getSkipCount(positionUs, loadingFinished); + + // Ensure we don't skip into preload chunks until we can be sure they are permanently published. + int readIndex = sampleQueue.getReadIndex(); + for (int i = 0; i < mediaChunks.size(); i++) { + HlsMediaChunk mediaChunk = mediaChunks.get(i); + int firstSampleIndex = mediaChunks.get(i).getFirstSampleIndex(sampleQueueIndex); + if (readIndex + skipCount <= firstSampleIndex) { + break; + } + if (!mediaChunk.isPublished()) { + skipCount = firstSampleIndex - readIndex; + break; + } + } + sampleQueue.skip(skipCount); return skipCount; } @@ -672,8 +712,8 @@ public boolean continueLoading(long positionUs) { /* allowEndOfStream= */ prepared || !chunkQueue.isEmpty(), nextChunkHolder); boolean endOfStream = nextChunkHolder.endOfStream; - Chunk loadable = nextChunkHolder.chunk; - Uri playlistUrlToLoad = nextChunkHolder.playlistUrl; + @Nullable Chunk loadable = nextChunkHolder.chunk; + @Nullable Uri playlistUrlToLoad = nextChunkHolder.playlistUrl; nextChunkHolder.clear(); if (endOfStream) { @@ -727,6 +767,16 @@ public void reevaluateBuffer(long positionUs) { return; } + int newQueueSize = readOnlyMediaChunks.size(); + while (newQueueSize > 0 + && chunkSource.getChunkPublicationState(readOnlyMediaChunks.get(newQueueSize - 1)) + == CHUNK_PUBLICATION_STATE_REMOVED) { + newQueueSize--; + } + if (newQueueSize < readOnlyMediaChunks.size()) { + discardUpstream(newQueueSize); + } + int preferredQueueSize = chunkSource.getPreferredQueueSize(positionUs, readOnlyMediaChunks); if (preferredQueueSize < mediaChunks.size()) { discardUpstream(preferredQueueSize); @@ -805,8 +855,19 @@ public LoadErrorAction onLoadError( long loadDurationMs, IOException error, int errorCount) { - long bytesLoaded = loadable.bytesLoaded(); boolean isMediaChunk = isMediaChunk(loadable); + if (isMediaChunk + && !((HlsMediaChunk) loadable).isPublished() + && error instanceof HttpDataSource.InvalidResponseCodeException) { + int responseCode = ((HttpDataSource.InvalidResponseCodeException) error).responseCode; + if (responseCode == 410 || responseCode == 404) { + // According to RFC 8216, Section 6.2.6 a server should respond with an HTTP 404 (Not found) + // for requests of hinted parts that are replaced and not available anymore. We've seen test + // streams with HTTP 410 (Gone) also. + return Loader.RETRY; + } + } + long bytesLoaded = loadable.bytesLoaded(); boolean exclusionSucceeded = false; LoadEventInfo loadEventInfo = new LoadEventInfo( @@ -980,7 +1041,7 @@ public TrackOutput track(int id, int type) { * * @param id The ID of the track. * @param type The type of the track, must be one of {@link #MAPPABLE_TYPES}. - * @return The the mapped {@link TrackOutput}, or null if it's not been created yet. + * @return The mapped {@link TrackOutput}, or null if it's not been created yet. */ @Nullable private TrackOutput getMappedTrackOutput(int id, int type) { diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/MediaParserHlsMediaChunkExtractor.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/MediaParserHlsMediaChunkExtractor.java new file mode 100644 index 00000000000..893bfc0a325 --- /dev/null +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/MediaParserHlsMediaChunkExtractor.java @@ -0,0 +1,287 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.hls; + +import static android.media.MediaParser.PARAMETER_TS_IGNORE_AAC_STREAM; +import static android.media.MediaParser.PARAMETER_TS_IGNORE_AVC_STREAM; +import static android.media.MediaParser.PARAMETER_TS_IGNORE_SPLICE_INFO_STREAM; +import static android.media.MediaParser.PARAMETER_TS_MODE; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_EAGERLY_EXPOSE_TRACK_TYPE; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_EXPOSE_CAPTION_FORMATS; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_IGNORE_TIMESTAMP_OFFSET; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_IN_BAND_CRYPTO_INFO; +import static com.google.android.exoplayer2.source.mediaparser.MediaParserUtil.PARAMETER_OVERRIDE_IN_BAND_CAPTION_DECLARATIONS; + +import android.annotation.SuppressLint; +import android.media.MediaFormat; +import android.media.MediaParser; +import android.media.MediaParser.OutputConsumer; +import android.media.MediaParser.SeekPoint; +import android.text.TextUtils; +import androidx.annotation.NonNull; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.extractor.ExtractorInput; +import com.google.android.exoplayer2.extractor.ExtractorOutput; +import com.google.android.exoplayer2.source.mediaparser.InputReaderAdapterV30; +import com.google.android.exoplayer2.source.mediaparser.MediaParserUtil; +import com.google.android.exoplayer2.source.mediaparser.OutputConsumerAdapterV30; +import com.google.android.exoplayer2.util.Assertions; +import com.google.android.exoplayer2.util.FileTypes; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.common.collect.ImmutableList; +import java.io.IOException; + +/** {@link HlsMediaChunkExtractor} implemented on top of the platform's {@link MediaParser}. */ +@RequiresApi(30) +public final class MediaParserHlsMediaChunkExtractor implements HlsMediaChunkExtractor { + + /** + * {@link HlsExtractorFactory} implementation that produces {@link + * MediaParserHlsMediaChunkExtractor} for all container formats except WebVTT, for which a {@link + * BundledHlsMediaChunkExtractor} is returned. + */ + public static final HlsExtractorFactory FACTORY = + (uri, + format, + muxedCaptionFormats, + timestampAdjuster, + responseHeaders, + sniffingExtractorInput) -> { + if (FileTypes.inferFileTypeFromMimeType(format.sampleMimeType) == FileTypes.WEBVTT) { + // The segment contains WebVTT. MediaParser does not support WebVTT parsing, so we use the + // bundled extractor. + return new BundledHlsMediaChunkExtractor( + new WebvttExtractor(format.language, timestampAdjuster), format, timestampAdjuster); + } + + boolean overrideInBandCaptionDeclarations = muxedCaptionFormats != null; + ImmutableList.Builder muxedCaptionMediaFormatsBuilder = + ImmutableList.builder(); + if (muxedCaptionFormats != null) { + // The manifest contains captions declarations. We use those to determine which captions + // will be exposed by MediaParser. + for (int i = 0; i < muxedCaptionFormats.size(); i++) { + muxedCaptionMediaFormatsBuilder.add( + MediaParserUtil.toCaptionsMediaFormat(muxedCaptionFormats.get(i))); + } + } else { + // The manifest does not declare any captions in the stream. Imitate the default HLS + // extractor factory and declare a 608 track by default. + muxedCaptionMediaFormatsBuilder.add( + MediaParserUtil.toCaptionsMediaFormat( + new Format.Builder().setSampleMimeType(MimeTypes.APPLICATION_CEA608).build())); + } + + ImmutableList muxedCaptionMediaFormats = + muxedCaptionMediaFormatsBuilder.build(); + + // TODO: Factor out code for optimizing the sniffing order across both factories. + OutputConsumerAdapterV30 outputConsumerAdapter = new OutputConsumerAdapterV30(); + outputConsumerAdapter.setMuxedCaptionFormats( + muxedCaptionFormats != null ? muxedCaptionFormats : ImmutableList.of()); + outputConsumerAdapter.setTimestampAdjuster(timestampAdjuster); + MediaParser mediaParser = + createMediaParserInstance( + outputConsumerAdapter, + format, + overrideInBandCaptionDeclarations, + muxedCaptionMediaFormats, + MediaParser.PARSER_NAME_FMP4, + MediaParser.PARSER_NAME_AC3, + MediaParser.PARSER_NAME_AC4, + MediaParser.PARSER_NAME_ADTS, + MediaParser.PARSER_NAME_MP3, + MediaParser.PARSER_NAME_TS); + + PeekingInputReader peekingInputReader = new PeekingInputReader(sniffingExtractorInput); + // The chunk extractor constructor requires an instance with a known parser name, so we + // advance once for MediaParser to sniff the content. + mediaParser.advance(peekingInputReader); + outputConsumerAdapter.setSelectedParserName(mediaParser.getParserName()); + + return new MediaParserHlsMediaChunkExtractor( + mediaParser, + outputConsumerAdapter, + format, + overrideInBandCaptionDeclarations, + muxedCaptionMediaFormats, + /* leadingBytesToSkip= */ peekingInputReader.totalPeekedBytes); + }; + + private final OutputConsumerAdapterV30 outputConsumerAdapter; + private final InputReaderAdapterV30 inputReaderAdapter; + private final MediaParser mediaParser; + private final Format format; + private final boolean overrideInBandCaptionDeclarations; + private final ImmutableList muxedCaptionMediaFormats; + private int pendingSkipBytes; + + /** + * Creates a new instance. + * + * @param mediaParser The {@link MediaParser} instance to use for extraction of segments. The + * provided instance must have completed sniffing, or must have been created by name. + * @param outputConsumerAdapter The {@link OutputConsumerAdapterV30} with which {@code + * mediaParser} was created. + * @param format The {@link Format} associated with the segment. + * @param overrideInBandCaptionDeclarations Whether to ignore any in-band caption track + * declarations in favor of using the {@code muxedCaptionMediaFormats} instead. If false, + * caption declarations found in the extracted media will be used, causing {@code + * muxedCaptionMediaFormats} to be ignored instead. + * @param muxedCaptionMediaFormats The list of in-band caption {@link MediaFormat MediaFormats} + * that {@link MediaParser} should expose. + * @param leadingBytesToSkip The number of bytes to skip from the start of the input before + * starting extraction. + */ + public MediaParserHlsMediaChunkExtractor( + MediaParser mediaParser, + OutputConsumerAdapterV30 outputConsumerAdapter, + Format format, + boolean overrideInBandCaptionDeclarations, + ImmutableList muxedCaptionMediaFormats, + int leadingBytesToSkip) { + this.mediaParser = mediaParser; + this.outputConsumerAdapter = outputConsumerAdapter; + this.overrideInBandCaptionDeclarations = overrideInBandCaptionDeclarations; + this.muxedCaptionMediaFormats = muxedCaptionMediaFormats; + this.format = format; + pendingSkipBytes = leadingBytesToSkip; + inputReaderAdapter = new InputReaderAdapterV30(); + } + + // ChunkExtractor implementation. + + @Override + public void init(ExtractorOutput extractorOutput) { + outputConsumerAdapter.setExtractorOutput(extractorOutput); + } + + @Override + public boolean read(ExtractorInput extractorInput) throws IOException { + extractorInput.skipFully(pendingSkipBytes); + pendingSkipBytes = 0; + inputReaderAdapter.setDataReader(extractorInput, extractorInput.getLength()); + return mediaParser.advance(inputReaderAdapter); + } + + @Override + public boolean isPackedAudioExtractor() { + String parserName = mediaParser.getParserName(); + return MediaParser.PARSER_NAME_AC3.equals(parserName) + || MediaParser.PARSER_NAME_AC4.equals(parserName) + || MediaParser.PARSER_NAME_ADTS.equals(parserName) + || MediaParser.PARSER_NAME_MP3.equals(parserName); + } + + @Override + public boolean isReusable() { + String parserName = mediaParser.getParserName(); + return MediaParser.PARSER_NAME_FMP4.equals(parserName) + || MediaParser.PARSER_NAME_TS.equals(parserName); + } + + @Override + public HlsMediaChunkExtractor recreate() { + Assertions.checkState(!isReusable()); + return new MediaParserHlsMediaChunkExtractor( + createMediaParserInstance( + outputConsumerAdapter, + format, + overrideInBandCaptionDeclarations, + muxedCaptionMediaFormats, + mediaParser.getParserName()), + outputConsumerAdapter, + format, + overrideInBandCaptionDeclarations, + muxedCaptionMediaFormats, + /* leadingBytesToSkip= */ 0); + } + + @Override + public void onTruncatedSegmentParsed() { + mediaParser.seek(SeekPoint.START); + } + + // Allow constants that are not part of the public MediaParser API. + @SuppressLint({"WrongConstant"}) + private static MediaParser createMediaParserInstance( + OutputConsumer outputConsumer, + Format format, + boolean overrideInBandCaptionDeclarations, + ImmutableList muxedCaptionMediaFormats, + String... parserNames) { + MediaParser mediaParser = + parserNames.length == 1 + ? MediaParser.createByName(parserNames[0], outputConsumer) + : MediaParser.create(outputConsumer, parserNames); + mediaParser.setParameter(PARAMETER_EXPOSE_CAPTION_FORMATS, muxedCaptionMediaFormats); + mediaParser.setParameter( + PARAMETER_OVERRIDE_IN_BAND_CAPTION_DECLARATIONS, overrideInBandCaptionDeclarations); + mediaParser.setParameter(PARAMETER_IN_BAND_CRYPTO_INFO, true); + mediaParser.setParameter(PARAMETER_EAGERLY_EXPOSE_TRACK_TYPE, true); + mediaParser.setParameter(PARAMETER_IGNORE_TIMESTAMP_OFFSET, true); + mediaParser.setParameter(PARAMETER_TS_IGNORE_SPLICE_INFO_STREAM, true); + mediaParser.setParameter(PARAMETER_TS_MODE, "hls"); + @Nullable String codecs = format.codecs; + if (!TextUtils.isEmpty(codecs)) { + // Sometimes AAC and H264 streams are declared in TS chunks even though they don't really + // exist. If we know from the codec attribute that they don't exist, then we can + // explicitly ignore them even if they're declared. + if (!MimeTypes.AUDIO_AAC.equals(MimeTypes.getAudioMediaMimeType(codecs))) { + mediaParser.setParameter(PARAMETER_TS_IGNORE_AAC_STREAM, true); + } + if (!MimeTypes.VIDEO_H264.equals(MimeTypes.getVideoMediaMimeType(codecs))) { + mediaParser.setParameter(PARAMETER_TS_IGNORE_AVC_STREAM, true); + } + } + return mediaParser; + } + + private static final class PeekingInputReader implements MediaParser.SeekableInputReader { + + private final ExtractorInput extractorInput; + private int totalPeekedBytes; + + private PeekingInputReader(ExtractorInput extractorInput) { + this.extractorInput = extractorInput; + } + + @Override + public int read(@NonNull byte[] buffer, int offset, int readLength) throws IOException { + int peekedBytes = extractorInput.peek(buffer, offset, readLength); + totalPeekedBytes += peekedBytes; + return peekedBytes; + } + + @Override + public long getPosition() { + return extractorInput.getPeekPosition(); + } + + @Override + public long getLength() { + return extractorInput.getLength(); + } + + @Override + public void seekToPosition(long position) { + // Seeking is not allowed when sniffing the content. + throw new UnsupportedOperationException(); + } + } +} diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistParserFactory.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistParserFactory.java index d185e2a3e83..3bb34a02689 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistParserFactory.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistParserFactory.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.source.hls.playlist; +import androidx.annotation.Nullable; import com.google.android.exoplayer2.upstream.ParsingLoadable; /** Default implementation for {@link HlsPlaylistParserFactory}. */ @@ -27,7 +28,7 @@ public ParsingLoadable.Parser createPlaylistParser() { @Override public ParsingLoadable.Parser createPlaylistParser( - HlsMasterPlaylist masterPlaylist) { - return new HlsPlaylistParser(masterPlaylist); + HlsMasterPlaylist masterPlaylist, @Nullable HlsMediaPlaylist previousMediaPlaylist) { + return new HlsPlaylistParser(masterPlaylist, previousMediaPlaylist); } } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistTracker.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistTracker.java index ccbcb986c1d..2a69f5c6af3 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistTracker.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistTracker.java @@ -15,6 +15,8 @@ */ package com.google.android.exoplayer2.source.hls.playlist; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Util.castNonNull; import static java.lang.Math.max; import android.net.Uri; @@ -28,8 +30,11 @@ import com.google.android.exoplayer2.source.MediaSourceEventListener.EventDispatcher; import com.google.android.exoplayer2.source.hls.HlsDataSourceFactory; import com.google.android.exoplayer2.source.hls.playlist.HlsMasterPlaylist.Variant; +import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist.Part; +import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist.RenditionReport; import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist.Segment; import com.google.android.exoplayer2.upstream.DataSource; +import com.google.android.exoplayer2.upstream.HttpDataSource; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy.LoadErrorInfo; import com.google.android.exoplayer2.upstream.Loader; @@ -37,6 +42,7 @@ import com.google.android.exoplayer2.upstream.ParsingLoadable; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.Iterables; import java.io.IOException; import java.util.ArrayList; import java.util.HashMap; @@ -62,7 +68,6 @@ public final class DefaultHlsPlaylistTracker private final List listeners; private final double playlistStuckTargetDurationCoefficient; - @Nullable private ParsingLoadable.Parser mediaPlaylistParser; @Nullable private EventDispatcher eventDispatcher; @Nullable private Loader initialPlaylistLoader; @Nullable private Handler playlistRefreshHandler; @@ -163,7 +168,7 @@ public void stop() { @Override public void addListener(PlaylistEventListener listener) { - Assertions.checkNotNull(listener); + checkNotNull(listener); listeners.add(listener); } @@ -181,7 +186,7 @@ public HlsMasterPlaylist getMasterPlaylist() { @Override @Nullable public HlsMediaPlaylist getPlaylistSnapshot(Uri url, boolean isForPlayback) { - HlsMediaPlaylist snapshot = playlistBundles.get(url).getPlaylistSnapshot(); + @Nullable HlsMediaPlaylist snapshot = playlistBundles.get(url).getPlaylistSnapshot(); if (snapshot != null && isForPlayback) { maybeSetPrimaryUrl(url); } @@ -237,10 +242,8 @@ public void onLoadCompleted( masterPlaylist = (HlsMasterPlaylist) result; } this.masterPlaylist = masterPlaylist; - mediaPlaylistParser = playlistParserFactory.createPlaylistParser(masterPlaylist); primaryMediaPlaylistUrl = masterPlaylist.variants.get(0).url; createBundles(masterPlaylist.mediaPlaylistUrls); - MediaPlaylistBundle primaryBundle = playlistBundles.get(primaryMediaPlaylistUrl); LoadEventInfo loadEventInfo = new LoadEventInfo( loadable.loadTaskId, @@ -250,6 +253,7 @@ public void onLoadCompleted( elapsedRealtimeMs, loadDurationMs, loadable.bytesLoaded()); + MediaPlaylistBundle primaryBundle = playlistBundles.get(primaryMediaPlaylistUrl); if (isMediaPlaylist) { // We don't need to load the playlist again. We can use the same result. primaryBundle.processLoadedPlaylist((HlsMediaPlaylist) result, loadEventInfo); @@ -316,10 +320,10 @@ private boolean maybeSelectNewPrimaryUrl() { int variantsSize = variants.size(); long currentTimeMs = SystemClock.elapsedRealtime(); for (int i = 0; i < variantsSize; i++) { - MediaPlaylistBundle bundle = playlistBundles.get(variants.get(i).url); + MediaPlaylistBundle bundle = checkNotNull(playlistBundles.get(variants.get(i).url)); if (currentTimeMs > bundle.excludeUntilMs) { primaryMediaPlaylistUrl = bundle.playlistUrl; - bundle.loadPlaylist(); + bundle.loadPlaylistInternal(getRequestUriForPrimaryChange(primaryMediaPlaylistUrl)); return true; } } @@ -335,7 +339,29 @@ private void maybeSetPrimaryUrl(Uri url) { return; } primaryMediaPlaylistUrl = url; - playlistBundles.get(primaryMediaPlaylistUrl).loadPlaylist(); + playlistBundles + .get(primaryMediaPlaylistUrl) + .loadPlaylistInternal(getRequestUriForPrimaryChange(url)); + } + + private Uri getRequestUriForPrimaryChange(Uri newPrimaryPlaylistUri) { + if (primaryMediaPlaylistSnapshot != null + && primaryMediaPlaylistSnapshot.serverControl.canBlockReload) { + @Nullable + RenditionReport renditionReport = + primaryMediaPlaylistSnapshot.renditionReports.get(newPrimaryPlaylistUri); + if (renditionReport != null) { + Uri.Builder uriBuilder = newPrimaryPlaylistUri.buildUpon(); + uriBuilder.appendQueryParameter( + MediaPlaylistBundle.BLOCK_MSN_PARAM, String.valueOf(renditionReport.lastMediaSequence)); + if (renditionReport.lastPartIndex != C.INDEX_UNSET) { + uriBuilder.appendQueryParameter( + MediaPlaylistBundle.BLOCK_PART_PARAM, String.valueOf(renditionReport.lastPartIndex)); + } + return uriBuilder.build(); + } + } + return newPrimaryPlaylistUri; } /** Returns whether any of the variants in the master playlist have the specified playlist URL. */ @@ -390,7 +416,7 @@ private boolean notifyPlaylistError(Uri playlistUrl, long exclusionDurationMs) { } private HlsMediaPlaylist getLatestPlaylistSnapshot( - HlsMediaPlaylist oldPlaylist, HlsMediaPlaylist loadedPlaylist) { + @Nullable HlsMediaPlaylist oldPlaylist, HlsMediaPlaylist loadedPlaylist) { if (!loadedPlaylist.isNewerThan(oldPlaylist)) { if (loadedPlaylist.hasEndTag) { // If the loaded playlist has an end tag but is not newer than the old playlist then we have @@ -408,7 +434,7 @@ private HlsMediaPlaylist getLatestPlaylistSnapshot( } private long getLoadedPlaylistStartTimeUs( - HlsMediaPlaylist oldPlaylist, HlsMediaPlaylist loadedPlaylist) { + @Nullable HlsMediaPlaylist oldPlaylist, HlsMediaPlaylist loadedPlaylist) { if (loadedPlaylist.hasProgramDateTime) { return loadedPlaylist.startTimeUs; } @@ -430,7 +456,7 @@ private long getLoadedPlaylistStartTimeUs( } private int getLoadedPlaylistDiscontinuitySequence( - HlsMediaPlaylist oldPlaylist, HlsMediaPlaylist loadedPlaylist) { + @Nullable HlsMediaPlaylist oldPlaylist, HlsMediaPlaylist loadedPlaylist) { if (loadedPlaylist.hasDiscontinuitySequence) { return loadedPlaylist.discontinuitySequence; } @@ -459,12 +485,15 @@ private static Segment getFirstOldOverlappingSegment( } /** Holds all information related to a specific Media Playlist. */ - private final class MediaPlaylistBundle - implements Loader.Callback>, Runnable { + private final class MediaPlaylistBundle implements Loader.Callback> { + + private static final String BLOCK_MSN_PARAM = "_HLS_msn"; + private static final String BLOCK_PART_PARAM = "_HLS_part"; + private static final String SKIP_PARAM = "_HLS_skip"; private final Uri playlistUrl; private final Loader mediaPlaylistLoader; - private final ParsingLoadable mediaPlaylistLoadable; + private final DataSource mediaPlaylistDataSource; @Nullable private HlsMediaPlaylist playlistSnapshot; private long lastSnapshotLoadMs; @@ -472,17 +501,12 @@ private final class MediaPlaylistBundle private long earliestNextLoadTimeMs; private long excludeUntilMs; private boolean loadPending; - private IOException playlistError; + @Nullable private IOException playlistError; public MediaPlaylistBundle(Uri playlistUrl) { this.playlistUrl = playlistUrl; mediaPlaylistLoader = new Loader("DefaultHlsPlaylistTracker:MediaPlaylist"); - mediaPlaylistLoadable = - new ParsingLoadable<>( - dataSourceFactory.createDataSource(C.DATA_TYPE_MANIFEST), - playlistUrl, - C.DATA_TYPE_MANIFEST, - mediaPlaylistParser); + mediaPlaylistDataSource = dataSourceFactory.createDataSource(C.DATA_TYPE_MANIFEST); } @Nullable @@ -502,23 +526,8 @@ public boolean isSnapshotValid() { || lastSnapshotLoadMs + snapshotValidityDurationMs > currentTimeMs; } - public void release() { - mediaPlaylistLoader.release(); - } - public void loadPlaylist() { - excludeUntilMs = 0; - if (loadPending || mediaPlaylistLoader.isLoading() || mediaPlaylistLoader.hasFatalError()) { - // Load already pending, in progress, or a fatal error has been encountered. Do nothing. - return; - } - long currentTimeMs = SystemClock.elapsedRealtime(); - if (currentTimeMs < earliestNextLoadTimeMs) { - loadPending = true; - playlistRefreshHandler.postDelayed(this, earliestNextLoadTimeMs - currentTimeMs); - } else { - loadPlaylistImmediately(); - } + loadPlaylistInternal(playlistUrl); } public void maybeThrowPlaylistRefreshError() throws IOException { @@ -528,12 +537,16 @@ public void maybeThrowPlaylistRefreshError() throws IOException { } } + public void release() { + mediaPlaylistLoader.release(); + } + // Loader.Callback implementation. @Override public void onLoadCompleted( ParsingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs) { - HlsPlaylist result = loadable.getResult(); + @Nullable HlsPlaylist result = loadable.getResult(); LoadEventInfo loadEventInfo = new LoadEventInfo( loadable.loadTaskId, @@ -589,6 +602,24 @@ public LoadErrorAction onLoadError( elapsedRealtimeMs, loadDurationMs, loadable.bytesLoaded()); + boolean isBlockingRequest = loadable.getUri().getQueryParameter(BLOCK_MSN_PARAM) != null; + boolean deltaUpdateFailed = error instanceof HlsPlaylistParser.DeltaUpdateException; + if (isBlockingRequest || deltaUpdateFailed) { + int responseCode = Integer.MAX_VALUE; + if (error instanceof HttpDataSource.InvalidResponseCodeException) { + responseCode = ((HttpDataSource.InvalidResponseCodeException) error).responseCode; + } + if (deltaUpdateFailed || responseCode == 400 || responseCode == 503) { + // Intercept failed delta updates and blocking requests producing a Bad Request (400) and + // Service Unavailable (503). In such cases, force a full, non-blocking request (see RFC + // 8216, section 6.2.5.2 and 6.3.7). + earliestNextLoadTimeMs = SystemClock.elapsedRealtime(); + loadPlaylist(); + castNonNull(eventDispatcher) + .loadError(loadEventInfo, loadable.type, error, /* wasCanceled= */ true); + return Loader.DONT_RETRY; + } + } MediaLoadData mediaLoadData = new MediaLoadData(loadable.type); LoadErrorInfo loadErrorInfo = new LoadErrorInfo(loadEventInfo, mediaLoadData, error, errorCount); @@ -620,17 +651,37 @@ public LoadErrorAction onLoadError( return loadErrorAction; } - // Runnable implementation. + // Internal methods. - @Override - public void run() { - loadPending = false; - loadPlaylistImmediately(); + private void loadPlaylistInternal(Uri playlistRequestUri) { + excludeUntilMs = 0; + if (loadPending || mediaPlaylistLoader.isLoading() || mediaPlaylistLoader.hasFatalError()) { + // Load already pending, in progress, or a fatal error has been encountered. Do nothing. + return; + } + long currentTimeMs = SystemClock.elapsedRealtime(); + if (currentTimeMs < earliestNextLoadTimeMs) { + loadPending = true; + playlistRefreshHandler.postDelayed( + () -> { + loadPending = false; + loadPlaylistImmediately(playlistRequestUri); + }, + earliestNextLoadTimeMs - currentTimeMs); + } else { + loadPlaylistImmediately(playlistRequestUri); + } } - // Internal methods. - - private void loadPlaylistImmediately() { + private void loadPlaylistImmediately(Uri playlistRequestUri) { + ParsingLoadable.Parser mediaPlaylistParser = + playlistParserFactory.createPlaylistParser(masterPlaylist, playlistSnapshot); + ParsingLoadable mediaPlaylistLoadable = + new ParsingLoadable<>( + mediaPlaylistDataSource, + playlistRequestUri, + C.DATA_TYPE_MANIFEST, + mediaPlaylistParser); long elapsedRealtime = mediaPlaylistLoader.startLoading( mediaPlaylistLoadable, @@ -644,7 +695,7 @@ private void loadPlaylistImmediately() { private void processLoadedPlaylist( HlsMediaPlaylist loadedPlaylist, LoadEventInfo loadEventInfo) { - HlsMediaPlaylist oldPlaylist = playlistSnapshot; + @Nullable HlsMediaPlaylist oldPlaylist = playlistSnapshot; long currentTimeMs = SystemClock.elapsedRealtime(); lastSnapshotLoadMs = currentTimeMs; playlistSnapshot = getLatestPlaylistSnapshot(oldPlaylist, loadedPlaylist); @@ -679,20 +730,53 @@ private void processLoadedPlaylist( } } } - // Do not allow the playlist to load again within the target duration if we obtained a new - // snapshot, or half the target duration otherwise. - earliestNextLoadTimeMs = - currentTimeMs - + C.usToMs( - playlistSnapshot != oldPlaylist - ? playlistSnapshot.targetDurationUs - : (playlistSnapshot.targetDurationUs / 2)); - // Schedule a load if this is the primary playlist and it doesn't have an end tag. Else the - // next load will be scheduled when refreshPlaylist is called, or when this playlist becomes - // the primary. - if (playlistUrl.equals(primaryMediaPlaylistUrl) && !playlistSnapshot.hasEndTag) { - loadPlaylist(); + long durationUntilNextLoadUs = 0L; + if (!playlistSnapshot.serverControl.canBlockReload) { + // If blocking requests are not supported, do not allow the playlist to load again within + // the target duration if we obtained a new snapshot, or half the target duration otherwise. + durationUntilNextLoadUs = + playlistSnapshot != oldPlaylist + ? playlistSnapshot.targetDurationUs + : (playlistSnapshot.targetDurationUs / 2); + } + earliestNextLoadTimeMs = currentTimeMs + C.usToMs(durationUntilNextLoadUs); + // Schedule a load if this is the primary playlist or a playlist of a low-latency stream and + // it doesn't have an end tag. Else the next load will be scheduled when refreshPlaylist is + // called, or when this playlist becomes the primary. + boolean scheduleLoad = + playlistSnapshot.partTargetDurationUs != C.TIME_UNSET + || playlistUrl.equals(primaryMediaPlaylistUrl); + if (scheduleLoad && !playlistSnapshot.hasEndTag) { + loadPlaylistInternal(getMediaPlaylistUriForReload()); + } + } + + private Uri getMediaPlaylistUriForReload() { + if (playlistSnapshot == null + || (playlistSnapshot.serverControl.skipUntilUs == C.TIME_UNSET + && !playlistSnapshot.serverControl.canBlockReload)) { + return playlistUrl; + } + Uri.Builder uriBuilder = playlistUrl.buildUpon(); + if (playlistSnapshot.serverControl.canBlockReload) { + long targetMediaSequence = + playlistSnapshot.mediaSequence + playlistSnapshot.segments.size(); + uriBuilder.appendQueryParameter(BLOCK_MSN_PARAM, String.valueOf(targetMediaSequence)); + if (playlistSnapshot.partTargetDurationUs != C.TIME_UNSET) { + List trailingParts = playlistSnapshot.trailingParts; + int targetPartIndex = trailingParts.size(); + if (!trailingParts.isEmpty() && Iterables.getLast(trailingParts).isPreload) { + // Ignore the preload part. + targetPartIndex--; + } + uriBuilder.appendQueryParameter(BLOCK_PART_PARAM, String.valueOf(targetPartIndex)); + } + } + if (playlistSnapshot.serverControl.skipUntilUs != C.TIME_UNSET) { + uriBuilder.appendQueryParameter( + SKIP_PARAM, playlistSnapshot.serverControl.canSkipDateRanges ? "v2" : "YES"); } + return uriBuilder.build(); } /** diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/FilteringHlsPlaylistParserFactory.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/FilteringHlsPlaylistParserFactory.java index 2d7ad5a78a2..cab93a1dc30 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/FilteringHlsPlaylistParserFactory.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/FilteringHlsPlaylistParserFactory.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.source.hls.playlist; +import androidx.annotation.Nullable; import com.google.android.exoplayer2.offline.FilteringManifestParser; import com.google.android.exoplayer2.offline.StreamKey; import com.google.android.exoplayer2.upstream.ParsingLoadable; @@ -48,8 +49,9 @@ public ParsingLoadable.Parser createPlaylistParser() { @Override public ParsingLoadable.Parser createPlaylistParser( - HlsMasterPlaylist masterPlaylist) { + HlsMasterPlaylist masterPlaylist, @Nullable HlsMediaPlaylist previousMediaPlaylist) { return new FilteringManifestParser<>( - hlsPlaylistParserFactory.createPlaylistParser(masterPlaylist), streamKeys); + hlsPlaylistParserFactory.createPlaylistParser(masterPlaylist, previousMediaPlaylist), + streamKeys); } } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsMediaPlaylist.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsMediaPlaylist.java index be771b92fc2..c4ab3fc662a 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsMediaPlaylist.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsMediaPlaylist.java @@ -15,73 +15,86 @@ */ package com.google.android.exoplayer2.source.hls.playlist; + +import android.net.Uri; import androidx.annotation.IntDef; import androidx.annotation.Nullable; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.drm.DrmInitData; import com.google.android.exoplayer2.offline.StreamKey; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Iterables; import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; -import java.util.Collections; +import java.util.ArrayList; import java.util.List; +import java.util.Map; /** Represents an HLS media playlist. */ public final class HlsMediaPlaylist extends HlsPlaylist { - /** Media segment reference. */ - @SuppressWarnings("ComparableType") - public static final class Segment implements Comparable { + /** Server control attributes. */ + public static final class ServerControl { /** - * The url of the segment. + * The skip boundary for delta updates in microseconds, or {@link C#TIME_UNSET} if delta updates + * are not supported. */ - public final String url; + public final long skipUntilUs; /** - * The media initialization section for this segment, as defined by #EXT-X-MAP. May be null if - * the media playlist does not define a media section for this segment. The same instance is - * used for all segments that share an EXT-X-MAP tag. + * Whether the playlist can produce delta updates that skip older #EXT-X-DATERANGE tags in + * addition to media segments. */ - @Nullable public final Segment initializationSegment; - /** The duration of the segment in microseconds, as defined by #EXTINF. */ - public final long durationUs; - /** The human readable title of the segment. */ - public final String title; + public final boolean canSkipDateRanges; /** - * The number of #EXT-X-DISCONTINUITY tags in the playlist before the segment. + * The server-recommended live offset in microseconds, or {@link C#TIME_UNSET} if none defined. */ - public final int relativeDiscontinuitySequence; + public final long holdBackUs; /** - * The start time of the segment in microseconds, relative to the start of the playlist. + * The server-recommended live offset in microseconds in low-latency mode, or {@link + * C#TIME_UNSET} if none defined. */ - public final long relativeStartTimeUs; - /** - * DRM initialization data for sample decryption, or null if the segment does not use CDM-DRM - * protection. - */ - @Nullable public final DrmInitData drmInitData; - /** - * The encryption identity key uri as defined by #EXT-X-KEY, or null if the segment does not use - * full segment encryption with identity key. - */ - @Nullable public final String fullSegmentEncryptionKeyUri; - /** - * The encryption initialization vector as defined by #EXT-X-KEY, or null if the segment is not - * encrypted. - */ - @Nullable public final String encryptionIV; - /** The segment's byte range offset, as defined by #EXT-X-BYTERANGE. */ - public final long byteRangeOffset; + public final long partHoldBackUs; + /** Whether the server supports blocking playlist reload. */ + public final boolean canBlockReload; + /** - * The segment's byte range length, as defined by #EXT-X-BYTERANGE, or {@link C#LENGTH_UNSET} if - * no byte range is specified. + * Creates a new instance. + * + * @param skipUntilUs See {@link #skipUntilUs}. + * @param canSkipDateRanges See {@link #canSkipDateRanges}. + * @param holdBackUs See {@link #holdBackUs}. + * @param partHoldBackUs See {@link #partHoldBackUs}. + * @param canBlockReload See {@link #canBlockReload}. */ - public final long byteRangeLength; + public ServerControl( + long skipUntilUs, + boolean canSkipDateRanges, + long holdBackUs, + long partHoldBackUs, + boolean canBlockReload) { + this.skipUntilUs = skipUntilUs; + this.canSkipDateRanges = canSkipDateRanges; + this.holdBackUs = holdBackUs; + this.partHoldBackUs = partHoldBackUs; + this.canBlockReload = canBlockReload; + } + } - /** Whether the segment is tagged with #EXT-X-GAP. */ - public final boolean hasGapTag; + /** Media segment reference. */ + @SuppressWarnings("ComparableType") + public static final class Segment extends SegmentBase { + + /** The human readable title of the segment. */ + public final String title; + /** The parts belonging to this segment. */ + public final List parts; /** + * Creates an instance to be used as init segment. + * * @param uri See {@link #url}. * @param byteRangeOffset See {@link #byteRangeOffset}. * @param byteRangeLength See {@link #byteRangeLength}. @@ -106,10 +119,13 @@ public Segment( encryptionIV, byteRangeOffset, byteRangeLength, - /* hasGapTag= */ false); + /* hasGapTag= */ false, + /* parts= */ ImmutableList.of()); } /** + * Creates an instance. + * * @param url See {@link #url}. * @param initializationSegment See {@link #initializationSegment}. * @param title See {@link #title}. @@ -122,6 +138,7 @@ public Segment( * @param byteRangeOffset See {@link #byteRangeOffset}. * @param byteRangeLength See {@link #byteRangeLength}. * @param hasGapTag See {@link #hasGapTag}. + * @param parts See {@link #parts}. */ public Segment( String url, @@ -135,10 +152,182 @@ public Segment( @Nullable String encryptionIV, long byteRangeOffset, long byteRangeLength, + boolean hasGapTag, + List parts) { + super( + url, + initializationSegment, + durationUs, + relativeDiscontinuitySequence, + relativeStartTimeUs, + drmInitData, + fullSegmentEncryptionKeyUri, + encryptionIV, + byteRangeOffset, + byteRangeLength, + hasGapTag); + this.title = title; + this.parts = ImmutableList.copyOf(parts); + } + + public Segment copyWith(long relativeStartTimeUs, int relativeDiscontinuitySequence) { + List updatedParts = new ArrayList<>(); + long relativePartStartTimeUs = relativeStartTimeUs; + for (int i = 0; i < parts.size(); i++) { + Part part = parts.get(i); + updatedParts.add(part.copyWith(relativePartStartTimeUs, relativeDiscontinuitySequence)); + relativePartStartTimeUs += part.durationUs; + } + return new Segment( + url, + initializationSegment, + title, + durationUs, + relativeDiscontinuitySequence, + relativeStartTimeUs, + drmInitData, + fullSegmentEncryptionKeyUri, + encryptionIV, + byteRangeOffset, + byteRangeLength, + hasGapTag, + updatedParts); + } + } + + /** A media part. */ + public static final class Part extends SegmentBase { + + /** Whether the part is independent. */ + public final boolean isIndependent; + /** Whether the part is a preloading part. */ + public final boolean isPreload; + + /** + * Creates an instance. + * + * @param url See {@link #url}. + * @param initializationSegment See {@link #initializationSegment}. + * @param durationUs See {@link #durationUs}. + * @param relativeDiscontinuitySequence See {@link #relativeDiscontinuitySequence}. + * @param relativeStartTimeUs See {@link #relativeStartTimeUs}. + * @param drmInitData See {@link #drmInitData}. + * @param fullSegmentEncryptionKeyUri See {@link #fullSegmentEncryptionKeyUri}. + * @param encryptionIV See {@link #encryptionIV}. + * @param byteRangeOffset See {@link #byteRangeOffset}. + * @param byteRangeLength See {@link #byteRangeLength}. + * @param hasGapTag See {@link #hasGapTag}. + * @param isIndependent See {@link #isIndependent}. + * @param isPreload See {@link #isPreload}. + */ + public Part( + String url, + @Nullable Segment initializationSegment, + long durationUs, + int relativeDiscontinuitySequence, + long relativeStartTimeUs, + @Nullable DrmInitData drmInitData, + @Nullable String fullSegmentEncryptionKeyUri, + @Nullable String encryptionIV, + long byteRangeOffset, + long byteRangeLength, + boolean hasGapTag, + boolean isIndependent, + boolean isPreload) { + super( + url, + initializationSegment, + durationUs, + relativeDiscontinuitySequence, + relativeStartTimeUs, + drmInitData, + fullSegmentEncryptionKeyUri, + encryptionIV, + byteRangeOffset, + byteRangeLength, + hasGapTag); + this.isIndependent = isIndependent; + this.isPreload = isPreload; + } + + public Part copyWith(long relativeStartTimeUs, int relativeDiscontinuitySequence) { + return new Part( + url, + initializationSegment, + durationUs, + relativeDiscontinuitySequence, + relativeStartTimeUs, + drmInitData, + fullSegmentEncryptionKeyUri, + encryptionIV, + byteRangeOffset, + byteRangeLength, + hasGapTag, + isIndependent, + isPreload); + } + } + + /** The base for a {@link Segment} or a {@link Part} required for playback. */ + @SuppressWarnings("ComparableType") + public static class SegmentBase implements Comparable { + /** The url of the segment. */ + public final String url; + /** + * The media initialization section for this segment, as defined by #EXT-X-MAP. May be null if + * the media playlist does not define a media initialization section for this segment. The same + * instance is used for all segments that share an EXT-X-MAP tag. + */ + @Nullable public final Segment initializationSegment; + /** The duration of the segment in microseconds, as defined by #EXTINF or #EXT-X-PART. */ + public final long durationUs; + /** The number of #EXT-X-DISCONTINUITY tags in the playlist before the segment. */ + public final int relativeDiscontinuitySequence; + /** The start time of the segment in microseconds, relative to the start of the playlist. */ + public final long relativeStartTimeUs; + /** + * DRM initialization data for sample decryption, or null if the segment does not use CDM-DRM + * protection. + */ + @Nullable public final DrmInitData drmInitData; + /** + * The encryption identity key uri as defined by #EXT-X-KEY, or null if the segment does not use + * full segment encryption with identity key. + */ + @Nullable public final String fullSegmentEncryptionKeyUri; + /** + * The encryption initialization vector as defined by #EXT-X-KEY, or null if the segment is not + * encrypted. + */ + @Nullable public final String encryptionIV; + /** + * The segment's byte range offset, as defined by #EXT-X-BYTERANGE, #EXT-X-PART or + * #EXT-X-PRELOAD-HINT. + */ + public final long byteRangeOffset; + /** + * The segment's byte range length, as defined by #EXT-X-BYTERANGE, #EXT-X-PART or + * #EXT-X-PRELOAD-HINT, or {@link C#LENGTH_UNSET} if no byte range is specified or the byte + * range is open-ended. + */ + public final long byteRangeLength; + /** Whether the segment is marked as a gap. */ + public final boolean hasGapTag; + + private SegmentBase( + String url, + @Nullable Segment initializationSegment, + long durationUs, + int relativeDiscontinuitySequence, + long relativeStartTimeUs, + @Nullable DrmInitData drmInitData, + @Nullable String fullSegmentEncryptionKeyUri, + @Nullable String encryptionIV, + long byteRangeOffset, + long byteRangeLength, boolean hasGapTag) { this.url = url; this.initializationSegment = initializationSegment; - this.title = title; this.durationUs = durationUs; this.relativeDiscontinuitySequence = relativeDiscontinuitySequence; this.relativeStartTimeUs = relativeStartTimeUs; @@ -155,7 +344,36 @@ public int compareTo(Long relativeStartTimeUs) { return this.relativeStartTimeUs > relativeStartTimeUs ? 1 : (this.relativeStartTimeUs < relativeStartTimeUs ? -1 : 0); } + } + + /** + * A rendition report for an alternative rendition defined in another media playlist. + * + *

    See RFC 8216, section 4.4.5.1.4. + */ + public static final class RenditionReport { + /** The URI of the media playlist of the reported rendition. */ + public final Uri playlistUri; + /** The last media sequence that is in the playlist of the reported rendition. */ + public final long lastMediaSequence; + /** + * The last part index that is in the playlist of the reported rendition, or {@link + * C#INDEX_UNSET} if the rendition does not contain partial segments. + */ + public final int lastPartIndex; + /** + * Creates a new instance. + * + * @param playlistUri See {@link #playlistUri}. + * @param lastMediaSequence See {@link #lastMediaSequence}. + * @param lastPartIndex See {@link #lastPartIndex}. + */ + public RenditionReport(Uri playlistUri, long lastMediaSequence, int lastPartIndex) { + this.playlistUri = playlistUri; + this.lastMediaSequence = lastMediaSequence; + this.lastPartIndex = lastPartIndex; + } } /** @@ -208,8 +426,11 @@ public int compareTo(Long relativeStartTimeUs) { */ public final long targetDurationUs; /** - * Whether the playlist contains the #EXT-X-ENDLIST tag. + * The target duration for segment parts, as defined by #EXT-X-PART-INF, or {@link C#TIME_UNSET} + * if undefined. */ + public final long partTargetDurationUs; + /** Whether the playlist contains the #EXT-X-ENDLIST tag. */ public final boolean hasEndTag; /** * Whether the playlist contains a #EXT-X-PROGRAM-DATE-TIME tag. @@ -225,9 +446,15 @@ public int compareTo(Long relativeStartTimeUs) { */ public final List segments; /** - * The total duration of the playlist in microseconds. + * The list of parts at the end of the playlist for which the segment is not in the playlist yet. */ + public final List trailingParts; + /** The rendition reports of alternative rendition playlists. */ + public final Map renditionReports; + /** The total duration of the playlist in microseconds. */ public final long durationUs; + /** The attributes of the #EXT-X-SERVER-CONTROL header. */ + public final ServerControl serverControl; /** * @param playlistType See {@link #playlistType}. @@ -242,9 +469,12 @@ public int compareTo(Long relativeStartTimeUs) { * @param targetDurationUs See {@link #targetDurationUs}. * @param hasIndependentSegments See {@link #hasIndependentSegments}. * @param hasEndTag See {@link #hasEndTag}. - * @param protectionSchemes See {@link #protectionSchemes}. * @param hasProgramDateTime See {@link #hasProgramDateTime}. + * @param protectionSchemes See {@link #protectionSchemes}. * @param segments See {@link #segments}. + * @param trailingParts See {@link #trailingParts}. + * @param serverControl See {@link #serverControl} + * @param renditionReports See {@link #renditionReports}. */ public HlsMediaPlaylist( @PlaylistType int playlistType, @@ -257,11 +487,15 @@ public HlsMediaPlaylist( long mediaSequence, int version, long targetDurationUs, + long partTargetDurationUs, boolean hasIndependentSegments, boolean hasEndTag, boolean hasProgramDateTime, @Nullable DrmInitData protectionSchemes, - List segments) { + List segments, + List trailingParts, + ServerControl serverControl, + Map renditionReports) { super(baseUri, tags, hasIndependentSegments); this.playlistType = playlistType; this.startTimeUs = startTimeUs; @@ -270,18 +504,25 @@ public HlsMediaPlaylist( this.mediaSequence = mediaSequence; this.version = version; this.targetDurationUs = targetDurationUs; + this.partTargetDurationUs = partTargetDurationUs; this.hasEndTag = hasEndTag; this.hasProgramDateTime = hasProgramDateTime; this.protectionSchemes = protectionSchemes; - this.segments = Collections.unmodifiableList(segments); - if (!segments.isEmpty()) { - Segment last = segments.get(segments.size() - 1); - durationUs = last.relativeStartTimeUs + last.durationUs; + this.segments = ImmutableList.copyOf(segments); + this.trailingParts = ImmutableList.copyOf(trailingParts); + this.renditionReports = ImmutableMap.copyOf(renditionReports); + if (!trailingParts.isEmpty()) { + Part lastPart = Iterables.getLast(trailingParts); + durationUs = lastPart.relativeStartTimeUs + lastPart.durationUs; + } else if (!segments.isEmpty()) { + Segment lastSegment = Iterables.getLast(segments); + durationUs = lastSegment.relativeStartTimeUs + lastSegment.durationUs; } else { durationUs = 0; } this.startOffsetUs = startOffsetUs == C.TIME_UNSET ? C.TIME_UNSET : startOffsetUs >= 0 ? startOffsetUs : durationUs + startOffsetUs; + this.serverControl = serverControl; } @Override @@ -295,7 +536,7 @@ public HlsMediaPlaylist copy(List streamKeys) { * @param other The playlist to compare. * @return Whether this playlist is newer than {@code other}. */ - public boolean isNewerThan(HlsMediaPlaylist other) { + public boolean isNewerThan(@Nullable HlsMediaPlaylist other) { if (other == null || mediaSequence > other.mediaSequence) { return true; } @@ -303,10 +544,14 @@ public boolean isNewerThan(HlsMediaPlaylist other) { return false; } // The media sequences are equal. - int segmentCount = segments.size(); - int otherSegmentCount = other.segments.size(); - return segmentCount > otherSegmentCount - || (segmentCount == otherSegmentCount && hasEndTag && !other.hasEndTag); + int segmentCountDifference = segments.size() - other.segments.size(); + if (segmentCountDifference != 0) { + return segmentCountDifference > 0; + } + int partCount = trailingParts.size(); + int otherPartCount = other.trailingParts.size(); + return partCount > otherPartCount + || (partCount == otherPartCount && hasEndTag && !other.hasEndTag); } /** @@ -337,11 +582,15 @@ public HlsMediaPlaylist copyWith(long startTimeUs, int discontinuitySequence) { mediaSequence, version, targetDurationUs, + partTargetDurationUs, hasIndependentSegments, hasEndTag, hasProgramDateTime, protectionSchemes, - segments); + segments, + trailingParts, + serverControl, + renditionReports); } /** @@ -363,11 +612,15 @@ public HlsMediaPlaylist copyWithEndTag() { mediaSequence, version, targetDurationUs, + partTargetDurationUs, hasIndependentSegments, /* hasEndTag= */ true, hasProgramDateTime, protectionSchemes, - segments); + segments, + trailingParts, + serverControl, + renditionReports); } } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParser.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParser.java index e6bdb0e03a1..62357ecdeab 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParser.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParser.java @@ -15,6 +15,10 @@ */ package com.google.android.exoplayer2.source.hls.playlist; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static com.google.android.exoplayer2.util.Util.castNonNull; + import android.net.Uri; import android.text.TextUtils; import android.util.Base64; @@ -31,6 +35,8 @@ import com.google.android.exoplayer2.source.hls.HlsTrackMetadataEntry.VariantInfo; import com.google.android.exoplayer2.source.hls.playlist.HlsMasterPlaylist.Rendition; import com.google.android.exoplayer2.source.hls.playlist.HlsMasterPlaylist.Variant; +import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist.Part; +import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist.RenditionReport; import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist.Segment; import com.google.android.exoplayer2.upstream.ParsingLoadable; import com.google.android.exoplayer2.util.Assertions; @@ -38,6 +44,7 @@ import com.google.android.exoplayer2.util.MimeTypes; import com.google.android.exoplayer2.util.UriUtil; import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.Iterables; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; @@ -62,6 +69,9 @@ */ public final class HlsPlaylistParser implements ParsingLoadable.Parser { + /** Exception thrown when merging a delta update fails. */ + public static final class DeltaUpdateException extends IOException {} + private static final String LOG_TAG = "HlsPlaylistParser"; private static final String PLAYLIST_HEADER = "#EXTM3U"; @@ -71,7 +81,10 @@ public final class HlsPlaylistParser implements ParsingLoadable.Parser variants, } private static HlsMediaPlaylist parseMediaPlaylist( - HlsMasterPlaylist masterPlaylist, LineIterator iterator, String baseUri) throws IOException { + HlsMasterPlaylist masterPlaylist, + @Nullable HlsMediaPlaylist previousMediaPlaylist, + LineIterator iterator, + String baseUri) + throws IOException { @HlsMediaPlaylist.PlaylistType int playlistType = HlsMediaPlaylist.PLAYLIST_TYPE_UNKNOWN; long startOffsetUs = C.TIME_UNSET; long mediaSequence = 0; int version = 1; // Default version == 1. long targetDurationUs = C.TIME_UNSET; + long partTargetDurationUs = C.TIME_UNSET; boolean hasIndependentSegmentsTag = masterPlaylist.hasIndependentSegments; boolean hasEndTag = false; @Nullable Segment initializationSegment = null; HashMap variableDefinitions = new HashMap<>(); HashMap urlToInferredInitSegment = new HashMap<>(); List segments = new ArrayList<>(); + List trailingParts = new ArrayList<>(); + @Nullable Part preloadPart = null; + Map renditionReports = new HashMap<>(); List tags = new ArrayList<>(); long segmentDurationUs = 0; @@ -590,16 +647,25 @@ private static HlsMediaPlaylist parseMediaPlaylist( long segmentStartTimeUs = 0; long segmentByteRangeOffset = 0; long segmentByteRangeLength = C.LENGTH_UNSET; + long partStartTimeUs = 0; + long partByteRangeOffset = 0; boolean isIFrameOnly = false; long segmentMediaSequence = 0; boolean hasGapTag = false; + HlsMediaPlaylist.ServerControl serverControl = + new HlsMediaPlaylist.ServerControl( + /* skipUntilUs= */ C.TIME_UNSET, + /* canSkipDateRanges= */ false, + /* holdBackUs= */ C.TIME_UNSET, + /* partHoldBackUs= */ C.TIME_UNSET, + /* canBlockReload= */ false); - DrmInitData playlistProtectionSchemes = null; - String fullSegmentEncryptionKeyUri = null; - String fullSegmentEncryptionIV = null; + @Nullable DrmInitData playlistProtectionSchemes = null; + @Nullable String fullSegmentEncryptionKeyUri = null; + @Nullable String fullSegmentEncryptionIV = null; TreeMap currentSchemeDatas = new TreeMap<>(); - String encryptionScheme = null; - DrmInitData cachedDrmInitData = null; + @Nullable String encryptionScheme = null; + @Nullable DrmInitData cachedDrmInitData = null; String line; while (iterator.hasNext()) { @@ -621,11 +687,16 @@ private static HlsMediaPlaylist parseMediaPlaylist( isIFrameOnly = true; } else if (line.startsWith(TAG_START)) { startOffsetUs = (long) (parseDoubleAttr(line, REGEX_TIME_OFFSET) * C.MICROS_PER_SECOND); + } else if (line.startsWith(TAG_SERVER_CONTROL)) { + serverControl = parseServerControl(line); + } else if (line.startsWith(TAG_PART_INF)) { + double partTargetDurationSeconds = parseDoubleAttr(line, REGEX_PART_TARGET_DURATION); + partTargetDurationUs = (long) (partTargetDurationSeconds * C.MICROS_PER_SECOND); } else if (line.startsWith(TAG_INIT_SEGMENT)) { String uri = parseStringAttr(line, REGEX_URI, variableDefinitions); String byteRange = parseOptionalStringAttr(line, REGEX_ATTR_BYTERANGE, variableDefinitions); if (byteRange != null) { - String[] splitByteRange = byteRange.split("@"); + String[] splitByteRange = Util.split(byteRange, "@"); segmentByteRangeLength = Long.parseLong(splitByteRange[0]); if (splitByteRange.length > 1) { segmentByteRangeOffset = Long.parseLong(splitByteRange[1]); @@ -671,6 +742,43 @@ private static HlsMediaPlaylist parseMediaPlaylist( segmentDurationUs = (long) (parseDoubleAttr(line, REGEX_MEDIA_DURATION) * C.MICROS_PER_SECOND); segmentTitle = parseOptionalStringAttr(line, REGEX_MEDIA_TITLE, "", variableDefinitions); + } else if (line.startsWith(TAG_SKIP)) { + int skippedSegmentCount = parseIntAttr(line, REGEX_SKIPPED_SEGMENTS); + checkState(previousMediaPlaylist != null && segments.isEmpty()); + int startIndex = (int) (mediaSequence - castNonNull(previousMediaPlaylist).mediaSequence); + int endIndex = startIndex + skippedSegmentCount; + if (startIndex < 0 || endIndex > previousMediaPlaylist.segments.size()) { + // Throw to force a reload if not all segments are available in the previous playlist. + throw new DeltaUpdateException(); + } + for (int i = startIndex; i < endIndex; i++) { + Segment segment = previousMediaPlaylist.segments.get(i); + if (mediaSequence != previousMediaPlaylist.mediaSequence) { + // If the media sequences of the playlists are not the same, we need to recreate the + // object with the updated relative start time and the relative discontinuity + // sequence. With identical playlist media sequences these values do not change. + int newRelativeDiscontinuitySequence = + previousMediaPlaylist.discontinuitySequence + - playlistDiscontinuitySequence + + segment.relativeDiscontinuitySequence; + segment = segment.copyWith(segmentStartTimeUs, newRelativeDiscontinuitySequence); + } + segments.add(segment); + segmentStartTimeUs += segment.durationUs; + partStartTimeUs = segmentStartTimeUs; + if (segment.byteRangeLength != C.LENGTH_UNSET) { + segmentByteRangeOffset = segment.byteRangeOffset + segment.byteRangeLength; + } + relativeDiscontinuitySequence = segment.relativeDiscontinuitySequence; + initializationSegment = segment.initializationSegment; + cachedDrmInitData = segment.drmInitData; + fullSegmentEncryptionKeyUri = segment.fullSegmentEncryptionKeyUri; + if (segment.encryptionIV == null + || !segment.encryptionIV.equals(Long.toHexString(segmentMediaSequence))) { + fullSegmentEncryptionIV = segment.encryptionIV; + } + segmentMediaSequence++; + } } else if (line.startsWith(TAG_KEY)) { String method = parseStringAttr(line, REGEX_METHOD, variableDefinitions); String keyFormat = @@ -703,7 +811,7 @@ private static HlsMediaPlaylist parseMediaPlaylist( } } else if (line.startsWith(TAG_BYTERANGE)) { String byteRange = parseStringAttr(line, REGEX_BYTERANGE, variableDefinitions); - String[] splitByteRange = byteRange.split("@"); + String[] splitByteRange = Util.split(byteRange, "@"); segmentByteRangeLength = Long.parseLong(splitByteRange[0]); if (splitByteRange.length > 1) { segmentByteRangeOffset = Long.parseLong(splitByteRange[1]); @@ -725,16 +833,117 @@ private static HlsMediaPlaylist parseMediaPlaylist( hasIndependentSegmentsTag = true; } else if (line.equals(TAG_ENDLIST)) { hasEndTag = true; - } else if (!line.startsWith("#")) { - String segmentEncryptionIV; - if (fullSegmentEncryptionKeyUri == null) { - segmentEncryptionIV = null; - } else if (fullSegmentEncryptionIV != null) { - segmentEncryptionIV = fullSegmentEncryptionIV; - } else { - segmentEncryptionIV = Long.toHexString(segmentMediaSequence); + } else if (line.startsWith(TAG_RENDITION_REPORT)) { + long defaultValue = mediaSequence + segments.size() - (trailingParts.isEmpty() ? 1 : 0); + long lastMediaSequence = parseOptionalLongAttr(line, REGEX_LAST_MSN, defaultValue); + List lastParts = + trailingParts.isEmpty() ? Iterables.getLast(segments).parts : trailingParts; + int defaultPartIndex = + partTargetDurationUs != C.TIME_UNSET ? lastParts.size() - 1 : C.INDEX_UNSET; + int lastPartIndex = parseOptionalIntAttr(line, REGEX_LAST_PART, defaultPartIndex); + String uri = parseStringAttr(line, REGEX_URI, variableDefinitions); + Uri playlistUri = Uri.parse(UriUtil.resolve(baseUri, uri)); + renditionReports.put( + playlistUri, new RenditionReport(playlistUri, lastMediaSequence, lastPartIndex)); + } else if (line.startsWith(TAG_PRELOAD_HINT)) { + if (preloadPart != null) { + continue; } - + String type = parseStringAttr(line, REGEX_PRELOAD_HINT_TYPE, variableDefinitions); + if (!TYPE_PART.equals(type)) { + continue; + } + String url = parseStringAttr(line, REGEX_URI, variableDefinitions); + long byteRangeStart = + parseOptionalLongAttr(line, REGEX_BYTERANGE_START, /* defaultValue= */ C.LENGTH_UNSET); + long byteRangeLength = + parseOptionalLongAttr(line, REGEX_BYTERANGE_LENGTH, /* defaultValue= */ C.LENGTH_UNSET); + @Nullable + String segmentEncryptionIV = + getSegmentEncryptionIV( + segmentMediaSequence, fullSegmentEncryptionKeyUri, fullSegmentEncryptionIV); + if (cachedDrmInitData == null && !currentSchemeDatas.isEmpty()) { + SchemeData[] schemeDatas = currentSchemeDatas.values().toArray(new SchemeData[0]); + cachedDrmInitData = new DrmInitData(encryptionScheme, schemeDatas); + if (playlistProtectionSchemes == null) { + playlistProtectionSchemes = getPlaylistProtectionSchemes(encryptionScheme, schemeDatas); + } + } + if (byteRangeStart == C.LENGTH_UNSET || byteRangeLength != C.LENGTH_UNSET) { + // Skip preload part if it is an unbounded range request. + preloadPart = + new Part( + url, + initializationSegment, + /* durationUs= */ 0, + relativeDiscontinuitySequence, + partStartTimeUs, + cachedDrmInitData, + fullSegmentEncryptionKeyUri, + segmentEncryptionIV, + byteRangeStart != C.LENGTH_UNSET ? byteRangeStart : 0, + byteRangeLength, + /* hasGapTag= */ false, + /* isIndependent= */ false, + /* isPreload= */ true); + } + } else if (line.startsWith(TAG_PART)) { + @Nullable + String segmentEncryptionIV = + getSegmentEncryptionIV( + segmentMediaSequence, fullSegmentEncryptionKeyUri, fullSegmentEncryptionIV); + String url = parseStringAttr(line, REGEX_URI, variableDefinitions); + long partDurationUs = + (long) (parseDoubleAttr(line, REGEX_ATTR_DURATION) * C.MICROS_PER_SECOND); + boolean isIndependent = + parseOptionalBooleanAttribute(line, REGEX_INDEPENDENT, /* defaultValue= */ false); + // The first part of a segment is always independent if the segments are independent. + isIndependent |= hasIndependentSegmentsTag && trailingParts.isEmpty(); + boolean isGap = parseOptionalBooleanAttribute(line, REGEX_GAP, /* defaultValue= */ false); + @Nullable + String byteRange = parseOptionalStringAttr(line, REGEX_ATTR_BYTERANGE, variableDefinitions); + long partByteRangeLength = C.LENGTH_UNSET; + if (byteRange != null) { + String[] splitByteRange = Util.split(byteRange, "@"); + partByteRangeLength = Long.parseLong(splitByteRange[0]); + if (splitByteRange.length > 1) { + partByteRangeOffset = Long.parseLong(splitByteRange[1]); + } + } + if (partByteRangeLength == C.LENGTH_UNSET) { + partByteRangeOffset = 0; + } + if (cachedDrmInitData == null && !currentSchemeDatas.isEmpty()) { + SchemeData[] schemeDatas = currentSchemeDatas.values().toArray(new SchemeData[0]); + cachedDrmInitData = new DrmInitData(encryptionScheme, schemeDatas); + if (playlistProtectionSchemes == null) { + playlistProtectionSchemes = getPlaylistProtectionSchemes(encryptionScheme, schemeDatas); + } + } + trailingParts.add( + new Part( + url, + initializationSegment, + partDurationUs, + relativeDiscontinuitySequence, + partStartTimeUs, + cachedDrmInitData, + fullSegmentEncryptionKeyUri, + segmentEncryptionIV, + partByteRangeOffset, + partByteRangeLength, + isGap, + isIndependent, + /* isPreload= */ false)); + partStartTimeUs += partDurationUs; + if (partByteRangeLength != C.LENGTH_UNSET) { + partByteRangeOffset += partByteRangeLength; + } + } else if (!line.startsWith("#")) { + @Nullable + String segmentEncryptionIV = + getSegmentEncryptionIV( + segmentMediaSequence, fullSegmentEncryptionKeyUri, fullSegmentEncryptionIV); segmentMediaSequence++; String segmentUri = replaceVariableReferences(line, variableDefinitions); @Nullable Segment inferredInitSegment = urlToInferredInitSegment.get(segmentUri); @@ -761,11 +970,7 @@ private static HlsMediaPlaylist parseMediaPlaylist( SchemeData[] schemeDatas = currentSchemeDatas.values().toArray(new SchemeData[0]); cachedDrmInitData = new DrmInitData(encryptionScheme, schemeDatas); if (playlistProtectionSchemes == null) { - SchemeData[] playlistSchemeDatas = new SchemeData[schemeDatas.length]; - for (int i = 0; i < schemeDatas.length; i++) { - playlistSchemeDatas[i] = schemeDatas[i].copyWithData(null); - } - playlistProtectionSchemes = new DrmInitData(encryptionScheme, playlistSchemeDatas); + playlistProtectionSchemes = getPlaylistProtectionSchemes(encryptionScheme, schemeDatas); } } @@ -782,10 +987,13 @@ private static HlsMediaPlaylist parseMediaPlaylist( segmentEncryptionIV, segmentByteRangeOffset, segmentByteRangeLength, - hasGapTag)); + hasGapTag, + trailingParts)); segmentStartTimeUs += segmentDurationUs; + partStartTimeUs = segmentStartTimeUs; segmentDurationUs = 0; segmentTitle = ""; + trailingParts = new ArrayList<>(); if (segmentByteRangeLength != C.LENGTH_UNSET) { segmentByteRangeOffset += segmentByteRangeLength; } @@ -793,6 +1001,11 @@ private static HlsMediaPlaylist parseMediaPlaylist( hasGapTag = false; } } + + if (preloadPart != null) { + trailingParts.add(preloadPart); + } + return new HlsMediaPlaylist( playlistType, baseUri, @@ -804,11 +1017,37 @@ private static HlsMediaPlaylist parseMediaPlaylist( mediaSequence, version, targetDurationUs, + partTargetDurationUs, hasIndependentSegmentsTag, hasEndTag, /* hasProgramDateTime= */ playlistStartTimeUs != 0, playlistProtectionSchemes, - segments); + segments, + trailingParts, + serverControl, + renditionReports); + } + + private static DrmInitData getPlaylistProtectionSchemes( + @Nullable String encryptionScheme, SchemeData[] schemeDatas) { + SchemeData[] playlistSchemeDatas = new SchemeData[schemeDatas.length]; + for (int i = 0; i < schemeDatas.length; i++) { + playlistSchemeDatas[i] = schemeDatas[i].copyWithData(null); + } + return new DrmInitData(encryptionScheme, playlistSchemeDatas); + } + + @Nullable + private static String getSegmentEncryptionIV( + long segmentMediaSequence, + @Nullable String fullSegmentEncryptionKeyUri, + @Nullable String fullSegmentEncryptionIV) { + if (fullSegmentEncryptionKeyUri == null) { + return null; + } else if (fullSegmentEncryptionIV != null) { + return fullSegmentEncryptionIV; + } + return Long.toHexString(segmentMediaSequence); } @C.SelectionFlags @@ -873,6 +1112,33 @@ private static SchemeData parseDrmSchemeData( return null; } + private static HlsMediaPlaylist.ServerControl parseServerControl(String line) { + double skipUntilSeconds = + parseOptionalDoubleAttr(line, REGEX_CAN_SKIP_UNTIL, /* defaultValue= */ C.TIME_UNSET); + long skipUntilUs = + skipUntilSeconds == C.TIME_UNSET + ? C.TIME_UNSET + : (long) (skipUntilSeconds * C.MICROS_PER_SECOND); + boolean canSkipDateRanges = + parseOptionalBooleanAttribute(line, REGEX_CAN_SKIP_DATE_RANGES, /* defaultValue= */ false); + double holdBackSeconds = + parseOptionalDoubleAttr(line, REGEX_HOLD_BACK, /* defaultValue= */ C.TIME_UNSET); + long holdBackUs = + holdBackSeconds == C.TIME_UNSET + ? C.TIME_UNSET + : (long) (holdBackSeconds * C.MICROS_PER_SECOND); + double partHoldBackSeconds = parseOptionalDoubleAttr(line, REGEX_PART_HOLD_BACK, C.TIME_UNSET); + long partHoldBackUs = + partHoldBackSeconds == C.TIME_UNSET + ? C.TIME_UNSET + : (long) (partHoldBackSeconds * C.MICROS_PER_SECOND); + boolean canBlockReload = + parseOptionalBooleanAttribute(line, REGEX_CAN_BLOCK_RELOAD, /* defaultValue= */ false); + + return new HlsMediaPlaylist.ServerControl( + skipUntilUs, canSkipDateRanges, holdBackUs, partHoldBackUs, canBlockReload); + } + private static String parseEncryptionScheme(String method) { return METHOD_SAMPLE_AES_CENC.equals(method) || METHOD_SAMPLE_AES_CTR.equals(method) ? C.CENC_TYPE_cenc @@ -886,7 +1152,7 @@ private static int parseIntAttr(String line, Pattern pattern) throws ParserExcep private static int parseOptionalIntAttr(String line, Pattern pattern, int defaultValue) { Matcher matcher = pattern.matcher(line); if (matcher.find()) { - return Integer.parseInt(Assertions.checkNotNull(matcher.group(1))); + return Integer.parseInt(checkNotNull(matcher.group(1))); } return defaultValue; } @@ -895,6 +1161,14 @@ private static long parseLongAttr(String line, Pattern pattern) throws ParserExc return Long.parseLong(parseStringAttr(line, pattern, Collections.emptyMap())); } + private static long parseOptionalLongAttr(String line, Pattern pattern, long defaultValue) { + Matcher matcher = pattern.matcher(line); + if (matcher.find()) { + return Long.parseLong(checkNotNull(matcher.group(1))); + } + return defaultValue; + } + private static double parseDoubleAttr(String line, Pattern pattern) throws ParserException { return Double.parseDouble(parseStringAttr(line, pattern, Collections.emptyMap())); } @@ -921,13 +1195,20 @@ private static String parseStringAttr( @PolyNull String defaultValue, Map variableDefinitions) { Matcher matcher = pattern.matcher(line); - @PolyNull - String value = matcher.find() ? Assertions.checkNotNull(matcher.group(1)) : defaultValue; + @PolyNull String value = matcher.find() ? checkNotNull(matcher.group(1)) : defaultValue; return variableDefinitions.isEmpty() || value == null ? value : replaceVariableReferences(value, variableDefinitions); } + private static double parseOptionalDoubleAttr(String line, Pattern pattern, double defaultValue) { + Matcher matcher = pattern.matcher(line); + if (matcher.find()) { + return Double.parseDouble(checkNotNull(matcher.group(1))); + } + return defaultValue; + } + private static String replaceVariableReferences( String string, Map variableDefinitions) { Matcher matcher = REGEX_VARIABLE_REFERENCE.matcher(string); @@ -977,7 +1258,7 @@ public boolean hasNext() throws IOException { return true; } if (!extraLines.isEmpty()) { - next = Assertions.checkNotNull(extraLines.poll()); + next = checkNotNull(extraLines.poll()); return true; } while ((next = reader.readLine()) != null) { @@ -999,7 +1280,5 @@ public String next() throws IOException { throw new NoSuchElementException(); } } - } - } diff --git a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParserFactory.java b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParserFactory.java index 814060bf7df..face3b277e5 100644 --- a/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParserFactory.java +++ b/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParserFactory.java @@ -15,6 +15,7 @@ */ package com.google.android.exoplayer2.source.hls.playlist; +import androidx.annotation.Nullable; import com.google.android.exoplayer2.upstream.ParsingLoadable; /** Factory for {@link HlsPlaylist} parsers. */ @@ -32,7 +33,10 @@ public interface HlsPlaylistParserFactory { * {@code masterPlaylist}. * * @param masterPlaylist The master playlist that referenced any parsed media playlists. + * @param previousMediaPlaylist The previous media playlist or null if there is no previous media + * playlist. * @return A parser for HLS playlists. */ - ParsingLoadable.Parser createPlaylistParser(HlsMasterPlaylist masterPlaylist); + ParsingLoadable.Parser createPlaylistParser( + HlsMasterPlaylist masterPlaylist, @Nullable HlsMediaPlaylist previousMediaPlaylist); } diff --git a/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaPeriodTest.java b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaPeriodTest.java index a6c42f97549..4ed8fc1ee3f 100644 --- a/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaPeriodTest.java +++ b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaPeriodTest.java @@ -90,7 +90,7 @@ public void getSteamKeys_isCompatibleWithHlsMasterPlaylistFilter() { .withParameters(/* windowIndex= */ 0, mediaPeriodId, /* mediaTimeOffsetMs= */ 0), mock(Allocator.class), mock(CompositeSequenceableLoaderFactory.class), - /* allowChunklessPreparation =*/ true, + /* allowChunklessPreparation= */ true, HlsMediaSource.METADATA_TYPE_ID3, /* useSessionKeys= */ false); }; diff --git a/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaPlaylistSegmentIteratorTest.java b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaPlaylistSegmentIteratorTest.java new file mode 100644 index 00000000000..d4b4b1b4aa2 --- /dev/null +++ b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaPlaylistSegmentIteratorTest.java @@ -0,0 +1,222 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.hls; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.fail; + +import android.net.Uri; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist; +import com.google.android.exoplayer2.source.hls.playlist.HlsPlaylistParser; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.android.exoplayer2.upstream.DataSpec; +import com.google.common.collect.Iterables; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link HlsChunkSource.HlsMediaPlaylistSegmentIterator}. */ +@RunWith(AndroidJUnit4.class) +public class HlsMediaPlaylistSegmentIteratorTest { + + public static final String LOW_LATENCY_SEGMENTS_AND_PARTS = + "media/m3u8/live_low_latency_segments_and_parts"; + public static final String SEGMENTS_ONLY = "media/m3u8/live_low_latency_segments_only"; + + @Test + public void create_withMediaSequenceBehindLiveWindow_isEmpty() { + HlsMediaPlaylist mediaPlaylist = getHlsMediaPlaylist(LOW_LATENCY_SEGMENTS_AND_PARTS); + + HlsChunkSource.HlsMediaPlaylistSegmentIterator hlsMediaPlaylistSegmentIterator = + new HlsChunkSource.HlsMediaPlaylistSegmentIterator( + mediaPlaylist.baseUri, + /* startOfPlaylistInPeriodUs= */ 0, + HlsChunkSource.getSegmentBaseList( + mediaPlaylist, mediaPlaylist.mediaSequence - 1, /* partIndex= */ C.INDEX_UNSET)); + + assertThat(hlsMediaPlaylistSegmentIterator.next()).isFalse(); + } + + @Test + public void create_withMediaSequenceBeforeTrailingPartSegment_isEmpty() { + HlsMediaPlaylist mediaPlaylist = getHlsMediaPlaylist(LOW_LATENCY_SEGMENTS_AND_PARTS); + + HlsChunkSource.HlsMediaPlaylistSegmentIterator hlsMediaPlaylistSegmentIterator = + new HlsChunkSource.HlsMediaPlaylistSegmentIterator( + mediaPlaylist.baseUri, + /* startOfPlaylistInPeriodUs= */ 0, + HlsChunkSource.getSegmentBaseList( + mediaPlaylist, + mediaPlaylist.mediaSequence + mediaPlaylist.segments.size() + 1, + /* partIndex= */ C.INDEX_UNSET)); + + assertThat(hlsMediaPlaylistSegmentIterator.next()).isFalse(); + } + + @Test + public void create_withPartIndexBeforeLastTrailingPartSegment_isEmpty() { + HlsMediaPlaylist mediaPlaylist = getHlsMediaPlaylist(LOW_LATENCY_SEGMENTS_AND_PARTS); + + HlsChunkSource.HlsMediaPlaylistSegmentIterator hlsMediaPlaylistSegmentIterator = + new HlsChunkSource.HlsMediaPlaylistSegmentIterator( + mediaPlaylist.baseUri, + /* startOfPlaylistInPeriodUs= */ 0, + HlsChunkSource.getSegmentBaseList( + mediaPlaylist, + mediaPlaylist.mediaSequence + mediaPlaylist.segments.size(), + /* partIndex= */ 3)); + + assertThat(hlsMediaPlaylistSegmentIterator.next()).isFalse(); + } + + @Test + public void next_conventionalLiveStartIteratorAtSecondSegment_correctElements() { + HlsMediaPlaylist mediaPlaylist = getHlsMediaPlaylist(SEGMENTS_ONLY); + HlsChunkSource.HlsMediaPlaylistSegmentIterator hlsMediaPlaylistSegmentIterator = + new HlsChunkSource.HlsMediaPlaylistSegmentIterator( + mediaPlaylist.baseUri, + /* startOfPlaylistInPeriodUs= */ 0, + HlsChunkSource.getSegmentBaseList( + mediaPlaylist, /* mediaSequence= */ 11, /* partIndex= */ C.INDEX_UNSET)); + + List datasSpecs = new ArrayList<>(); + while (hlsMediaPlaylistSegmentIterator.next()) { + datasSpecs.add(hlsMediaPlaylistSegmentIterator.getDataSpec()); + } + + assertThat(datasSpecs).hasSize(5); + assertThat(datasSpecs.get(0).uri.toString()).isEqualTo("fileSequence11.ts"); + assertThat(Iterables.getLast(datasSpecs).uri.toString()).isEqualTo("fileSequence15.ts"); + } + + @Test + public void next_startIteratorAtFirstSegment_correctElements() { + HlsMediaPlaylist mediaPlaylist = getHlsMediaPlaylist(LOW_LATENCY_SEGMENTS_AND_PARTS); + HlsChunkSource.HlsMediaPlaylistSegmentIterator hlsMediaPlaylistSegmentIterator = + new HlsChunkSource.HlsMediaPlaylistSegmentIterator( + mediaPlaylist.baseUri, + /* startOfPlaylistInPeriodUs= */ 0, + HlsChunkSource.getSegmentBaseList( + mediaPlaylist, /* mediaSequence= */ 10, /* partIndex= */ C.INDEX_UNSET)); + + List datasSpecs = new ArrayList<>(); + while (hlsMediaPlaylistSegmentIterator.next()) { + datasSpecs.add(hlsMediaPlaylistSegmentIterator.getDataSpec()); + } + + assertThat(datasSpecs).hasSize(9); + // The iterator starts with 6 segments. + assertThat(datasSpecs.get(0).uri.toString()).isEqualTo("fileSequence10.ts"); + // Followed by trailing parts. + assertThat(datasSpecs.get(6).uri.toString()).isEqualTo("fileSequence16.0.ts"); + // The preload part is the last. + assertThat(Iterables.getLast(datasSpecs).uri.toString()).isEqualTo("fileSequence16.2.ts"); + } + + @Test + public void next_startIteratorAtFirstPartInaSegment_usesFullSegment() { + HlsMediaPlaylist mediaPlaylist = getHlsMediaPlaylist(LOW_LATENCY_SEGMENTS_AND_PARTS); + HlsChunkSource.HlsMediaPlaylistSegmentIterator hlsMediaPlaylistSegmentIterator = + new HlsChunkSource.HlsMediaPlaylistSegmentIterator( + mediaPlaylist.baseUri, + /* startOfPlaylistInPeriodUs= */ 0, + HlsChunkSource.getSegmentBaseList( + mediaPlaylist, /* mediaSequence= */ 14, /* partIndex= */ 0)); + + List datasSpecs = new ArrayList<>(); + while (hlsMediaPlaylistSegmentIterator.next()) { + datasSpecs.add(hlsMediaPlaylistSegmentIterator.getDataSpec()); + } + + assertThat(datasSpecs).hasSize(5); + // The iterator starts with 6 segments. + assertThat(datasSpecs.get(0).uri.toString()).isEqualTo("fileSequence14.ts"); + assertThat(datasSpecs.get(1).uri.toString()).isEqualTo("fileSequence15.ts"); + // Followed by trailing parts. + assertThat(datasSpecs.get(2).uri.toString()).isEqualTo("fileSequence16.0.ts"); + assertThat(datasSpecs.get(3).uri.toString()).isEqualTo("fileSequence16.1.ts"); + // The preload part is the last. + assertThat(Iterables.getLast(datasSpecs).uri.toString()).isEqualTo("fileSequence16.2.ts"); + } + + @Test + public void next_startIteratorAtTrailingPart_correctElements() { + HlsMediaPlaylist mediaPlaylist = getHlsMediaPlaylist(LOW_LATENCY_SEGMENTS_AND_PARTS); + HlsChunkSource.HlsMediaPlaylistSegmentIterator hlsMediaPlaylistSegmentIterator = + new HlsChunkSource.HlsMediaPlaylistSegmentIterator( + mediaPlaylist.baseUri, + /* startOfPlaylistInPeriodUs= */ 0, + HlsChunkSource.getSegmentBaseList( + mediaPlaylist, /* mediaSequence= */ 16, /* partIndex= */ 1)); + + List datasSpecs = new ArrayList<>(); + while (hlsMediaPlaylistSegmentIterator.next()) { + datasSpecs.add(hlsMediaPlaylistSegmentIterator.getDataSpec()); + } + + assertThat(datasSpecs).hasSize(2); + // The iterator starts with 2 parts. + assertThat(datasSpecs.get(0).uri.toString()).isEqualTo("fileSequence16.1.ts"); + // The preload part is the last. + assertThat(Iterables.getLast(datasSpecs).uri.toString()).isEqualTo("fileSequence16.2.ts"); + } + + @Test + public void next_startIteratorAtPartWithinSegment_correctElements() { + HlsMediaPlaylist mediaPlaylist = getHlsMediaPlaylist(LOW_LATENCY_SEGMENTS_AND_PARTS); + HlsChunkSource.HlsMediaPlaylistSegmentIterator hlsMediaPlaylistSegmentIterator = + new HlsChunkSource.HlsMediaPlaylistSegmentIterator( + mediaPlaylist.baseUri, + /* startOfPlaylistInPeriodUs= */ 0, + HlsChunkSource.getSegmentBaseList( + mediaPlaylist, /* mediaSequence= */ 14, /* partIndex= */ 1)); + + List datasSpecs = new ArrayList<>(); + while (hlsMediaPlaylistSegmentIterator.next()) { + datasSpecs.add(hlsMediaPlaylistSegmentIterator.getDataSpec()); + } + + assertThat(datasSpecs).hasSize(7); + // The iterator starts with 11 parts. + assertThat(datasSpecs.get(0).uri.toString()).isEqualTo("fileSequence14.1.ts"); + assertThat(datasSpecs.get(1).uri.toString()).isEqualTo("fileSequence14.2.ts"); + assertThat(datasSpecs.get(2).uri.toString()).isEqualTo("fileSequence14.3.ts"); + // Use a segment in between if possible. + assertThat(datasSpecs.get(3).uri.toString()).isEqualTo("fileSequence15.ts"); + // Then parts again. + assertThat(datasSpecs.get(4).uri.toString()).isEqualTo("fileSequence16.0.ts"); + assertThat(datasSpecs.get(5).uri.toString()).isEqualTo("fileSequence16.1.ts"); + assertThat(datasSpecs.get(6).uri.toString()).isEqualTo("fileSequence16.2.ts"); + } + + private static HlsMediaPlaylist getHlsMediaPlaylist(String file) { + try { + return (HlsMediaPlaylist) + new HlsPlaylistParser() + .parse( + Uri.EMPTY, + TestUtil.getInputStream(ApplicationProvider.getApplicationContext(), file)); + } catch (IOException e) { + fail(e.getMessage()); + } + return null; + } +} diff --git a/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaSourceTest.java b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaSourceTest.java index 7001417186d..fd2744280aa 100644 --- a/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaSourceTest.java +++ b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/HlsMediaSourceTest.java @@ -15,15 +15,33 @@ */ package com.google.android.exoplayer2.source.hls; +import static com.google.android.exoplayer2.robolectric.RobolectricUtil.runMainLooperUntil; import static com.google.common.truth.Truth.assertThat; import static org.mockito.Mockito.mock; +import android.net.Uri; +import android.os.SystemClock; import androidx.annotation.Nullable; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.ParserException; +import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.offline.StreamKey; +import com.google.android.exoplayer2.source.MediaSource; +import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist; +import com.google.android.exoplayer2.source.hls.playlist.HlsPlaylistParser; +import com.google.android.exoplayer2.testutil.FakeDataSet; +import com.google.android.exoplayer2.testutil.FakeDataSource; import com.google.android.exoplayer2.upstream.DataSource; +import com.google.android.exoplayer2.util.Util; +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.util.ArrayList; import java.util.Collections; +import java.util.List; +import java.util.concurrent.TimeoutException; +import java.util.concurrent.atomic.AtomicReference; import org.junit.Test; import org.junit.runner.RunWith; @@ -40,11 +58,11 @@ public void factorySetTag_nullMediaItemTag_setsMediaItemTag() { HlsMediaSource.Factory factory = new HlsMediaSource.Factory(mock(DataSource.Factory.class)).setTag(tag); - MediaItem dashMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); + MediaItem hlsMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); - assertThat(dashMediaItem.playbackProperties).isNotNull(); - assertThat(dashMediaItem.playbackProperties.uri).isEqualTo(mediaItem.playbackProperties.uri); - assertThat(dashMediaItem.playbackProperties.tag).isEqualTo(tag); + assertThat(hlsMediaItem.playbackProperties).isNotNull(); + assertThat(hlsMediaItem.playbackProperties.uri).isEqualTo(mediaItem.playbackProperties.uri); + assertThat(hlsMediaItem.playbackProperties.tag).isEqualTo(tag); } // Tests backwards compatibility @@ -58,11 +76,11 @@ public void factorySetTag_nonNullMediaItemTag_doesNotOverrideMediaItemTag() { HlsMediaSource.Factory factory = new HlsMediaSource.Factory(mock(DataSource.Factory.class)).setTag(factoryTag); - MediaItem dashMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); + MediaItem hlsMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); - assertThat(dashMediaItem.playbackProperties).isNotNull(); - assertThat(dashMediaItem.playbackProperties.uri).isEqualTo(mediaItem.playbackProperties.uri); - assertThat(dashMediaItem.playbackProperties.tag).isEqualTo(mediaItemTag); + assertThat(hlsMediaItem.playbackProperties).isNotNull(); + assertThat(hlsMediaItem.playbackProperties.uri).isEqualTo(mediaItem.playbackProperties.uri); + assertThat(hlsMediaItem.playbackProperties.tag).isEqualTo(mediaItemTag); } // Tests backwards compatibility @@ -104,11 +122,11 @@ public void factorySetStreamKeys_emptyMediaItemStreamKeys_setsMediaItemStreamKey new HlsMediaSource.Factory(mock(DataSource.Factory.class)) .setStreamKeys(Collections.singletonList(streamKey)); - MediaItem dashMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); + MediaItem hlsMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); - assertThat(dashMediaItem.playbackProperties).isNotNull(); - assertThat(dashMediaItem.playbackProperties.uri).isEqualTo(mediaItem.playbackProperties.uri); - assertThat(dashMediaItem.playbackProperties.streamKeys).containsExactly(streamKey); + assertThat(hlsMediaItem.playbackProperties).isNotNull(); + assertThat(hlsMediaItem.playbackProperties.uri).isEqualTo(mediaItem.playbackProperties.uri); + assertThat(hlsMediaItem.playbackProperties.streamKeys).containsExactly(streamKey); } // Tests backwards compatibility @@ -126,10 +144,369 @@ public void factorySetStreamKeys_withMediaItemStreamKeys_doesNotOverrideMediaIte .setStreamKeys( Collections.singletonList(new StreamKey(/* groupIndex= */ 1, /* trackIndex= */ 0))); - MediaItem dashMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); + MediaItem hlsMediaItem = factory.createMediaSource(mediaItem).getMediaItem(); - assertThat(dashMediaItem.playbackProperties).isNotNull(); - assertThat(dashMediaItem.playbackProperties.uri).isEqualTo(mediaItem.playbackProperties.uri); - assertThat(dashMediaItem.playbackProperties.streamKeys).containsExactly(mediaItemStreamKey); + assertThat(hlsMediaItem.playbackProperties).isNotNull(); + assertThat(hlsMediaItem.playbackProperties.uri).isEqualTo(mediaItem.playbackProperties.uri); + assertThat(hlsMediaItem.playbackProperties.streamKeys).containsExactly(mediaItemStreamKey); + } + + @Test + public void loadPlaylist_noTargetLiveOffsetDefined_fallbackToThreeTargetDuration() + throws TimeoutException, ParserException { + String playlistUri = "fake://foo.bar/media0/playlist.m3u8"; + // The playlist has a duration of 16 seconds but not hold back or part hold back. + String playlist = + "#EXTM3U\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-01T00:00:00.0+00:00\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:4.00000,\n" + + "fileSequence0.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence1.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence2.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence3.ts\n" + + "#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=24"; + // The playlist finishes 1 second before the current time, therefore there's a live edge + // offset of 1 second. + SystemClock.setCurrentTimeMillis(Util.parseXsDateTime("2020-01-01T00:00:17.0+00:00")); + HlsMediaSource.Factory factory = createHlsMediaSourceFactory(playlistUri, playlist); + MediaItem mediaItem = MediaItem.fromUri(playlistUri); + HlsMediaSource mediaSource = factory.createMediaSource(mediaItem); + + Timeline timeline = prepareAndWaitForTimeline(mediaSource); + + Timeline.Window window = timeline.getWindow(0, new Timeline.Window()); + // The target live offset is picked from target duration (3 * 4 = 12 seconds) and then expressed + // in relation to the live edge (12 + 1 seconds). + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(13000); + assertThat(window.defaultPositionUs).isEqualTo(4000000); + } + + @Test + public void loadPlaylist_holdBackInPlaylist_targetLiveOffsetFromHoldBack() + throws TimeoutException, ParserException { + String playlistUri = "fake://foo.bar/media0/playlist.m3u8"; + // The playlist has a duration of 16 seconds and a hold back of 12 seconds. + String playlist = + "#EXTM3U\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-01T00:00:00.0+00:00\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:4.00000,\n" + + "fileSequence0.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence1.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence2.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence3.ts\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK=12"; + // The playlist finishes 1 second before the the current time, therefore there's a live edge + // offset of 1 second. + SystemClock.setCurrentTimeMillis(Util.parseXsDateTime("2020-01-01T00:00:17.0+00:00")); + HlsMediaSource.Factory factory = createHlsMediaSourceFactory(playlistUri, playlist); + MediaItem mediaItem = MediaItem.fromUri(playlistUri); + HlsMediaSource mediaSource = factory.createMediaSource(mediaItem); + + Timeline timeline = prepareAndWaitForTimeline(mediaSource); + + Timeline.Window window = timeline.getWindow(0, new Timeline.Window()); + // The target live offset is picked from hold back and then expressed in relation to the live + // edge (+1 seconds). + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(13000); + assertThat(window.defaultPositionUs).isEqualTo(4000000); + } + + @Test + public void + loadPlaylist_partHoldBackWithoutPartInformationInPlaylist_targetLiveOffsetFromHoldBack() + throws TimeoutException, ParserException { + String playlistUri = "fake://foo.bar/media0/playlist.m3u8"; + // The playlist has a part hold back but not EXT-X-PART-INF. We should pick up the hold back. + // The duration of the playlist is 16 seconds so that the defined hold back is within the live + // window. + String playlist = + "#EXTM3U\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-01T00:00:00.0+00:00\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:4.00000,\n" + + "fileSequence0.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence1.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence2.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence3.ts\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK=12,PART-HOLD-BACK=3"; + // The playlist finishes 1 second before the the current time. + SystemClock.setCurrentTimeMillis(Util.parseXsDateTime("2020-01-01T00:00:17.0+00:00")); + HlsMediaSource.Factory factory = createHlsMediaSourceFactory(playlistUri, playlist); + MediaItem mediaItem = MediaItem.fromUri(playlistUri); + HlsMediaSource mediaSource = factory.createMediaSource(mediaItem); + + Timeline timeline = prepareAndWaitForTimeline(mediaSource); + + Timeline.Window window = timeline.getWindow(0, new Timeline.Window()); + // The target live offset is picked from hold back and then expressed in relation to the live + // edge (+1 seconds). + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(13000); + assertThat(window.defaultPositionUs).isEqualTo(4000000); + } + + @Test + public void + loadPlaylist_partHoldBackWithPartInformationInPlaylist_targetLiveOffsetFromPartHoldBack() + throws TimeoutException, ParserException { + String playlistUri = "fake://foo.bar/media0/playlist.m3u8"; + // The playlist has a duration of 4 seconds, part hold back and EXT-X-PART-INF defined. + String playlist = + "#EXTM3U\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-01T00:00:00.0+00:00\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:4.00000,\n" + + "fileSequence0.ts\n" + + "#EXT-X-PART-INF:PART-TARGET=0.5\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK=12,PART-HOLD-BACK=3"; + // The playlist finishes 1 second before the current time. + SystemClock.setCurrentTimeMillis(Util.parseXsDateTime("2020-01-01T00:00:05.0+00:00")); + HlsMediaSource.Factory factory = createHlsMediaSourceFactory(playlistUri, playlist); + MediaItem mediaItem = MediaItem.fromUri(playlistUri); + HlsMediaSource mediaSource = factory.createMediaSource(mediaItem); + + Timeline timeline = prepareAndWaitForTimeline(mediaSource); + + Timeline.Window window = timeline.getWindow(0, new Timeline.Window()); + // The target live offset is picked from part hold back and then expressed in relation to the + // live edge (+1 seconds). + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(4000); + assertThat(window.defaultPositionUs).isEqualTo(0); + } + + @Test + public void loadPlaylist_targetLiveOffsetInMediaItem_targetLiveOffsetPickedFromMediaItem() + throws TimeoutException, ParserException { + String playlistUri = "fake://foo.bar/media0/playlist.m3u8"; + // The playlist has a hold back of 12 seconds and a part hold back of 3 seconds. + String playlist = + "#EXTM3U\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-01T00:00:00.0+00:00\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:4.00000,\n" + + "fileSequence0.ts\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK=12,PART-HOLD-BACK=3"; + // The playlist finishes 1 second before the the current time. This should not affect the target + // live offset set in the media item. + SystemClock.setCurrentTimeMillis(Util.parseXsDateTime("2020-01-01T00:00:05.0+00:00")); + HlsMediaSource.Factory factory = createHlsMediaSourceFactory(playlistUri, playlist); + MediaItem mediaItem = + new MediaItem.Builder().setUri(playlistUri).setLiveTargetOffsetMs(1000).build(); + HlsMediaSource mediaSource = factory.createMediaSource(mediaItem); + + Timeline timeline = prepareAndWaitForTimeline(mediaSource); + + Timeline.Window window = timeline.getWindow(0, new Timeline.Window()); + // The target live offset is picked from the media item and not adjusted. + assertThat(window.liveConfiguration).isEqualTo(mediaItem.liveConfiguration); + assertThat(window.defaultPositionUs).isEqualTo(0); + } + + @Test + public void loadPlaylist_targetLiveOffsetLargerThanLiveWindow_targetLiveOffsetIsWithinLiveWindow() + throws TimeoutException, ParserException { + String playlistUri = "fake://foo.bar/media0/playlist.m3u8"; + // The playlist has a duration of 8 seconds and a hold back of 12 seconds. + String playlist = + "#EXTM3U\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-01T00:00:00.0+00:00\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:4.00000,\n" + + "fileSequence0.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence1.ts\n" + + "#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=24"; + // The playlist finishes 1 second before the live edge, therefore the live window duration is + // 9 seconds (8 + 1). + SystemClock.setCurrentTimeMillis(Util.parseXsDateTime("2020-01-01T00:00:09.0+00:00")); + HlsMediaSource.Factory factory = createHlsMediaSourceFactory(playlistUri, playlist); + MediaItem mediaItem = + new MediaItem.Builder().setUri(playlistUri).setLiveTargetOffsetMs(20_000).build(); + HlsMediaSource mediaSource = factory.createMediaSource(mediaItem); + + Timeline timeline = prepareAndWaitForTimeline(mediaSource); + + Timeline.Window window = timeline.getWindow(0, new Timeline.Window()); + assertThat(mediaItem.liveConfiguration.targetOffsetMs) + .isGreaterThan(C.usToMs(window.durationUs)); + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(9000); + } + + @Test + public void + loadPlaylist_withoutProgramDateTime_targetLiveOffsetFromPlaylistNotAdjustedToLiveEdge() + throws TimeoutException { + String playlistUri = "fake://foo.bar/media0/playlist.m3u8"; + // The playlist has a duration of 16 seconds and a hold back of 12 seconds. + String playlist = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:4.00000,\n" + + "fileSequence0.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence1.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence2.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence3.ts\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK=12"; + // The playlist finishes 8 seconds before the current time. + SystemClock.setCurrentTimeMillis(20000); + HlsMediaSource.Factory factory = createHlsMediaSourceFactory(playlistUri, playlist); + MediaItem mediaItem = new MediaItem.Builder().setUri(playlistUri).build(); + HlsMediaSource mediaSource = factory.createMediaSource(mediaItem); + + Timeline timeline = prepareAndWaitForTimeline(mediaSource); + + Timeline.Window window = timeline.getWindow(0, new Timeline.Window()); + // The target live offset is not adjusted to the live edge because the list does not have + // program date time. + assertThat(window.liveConfiguration.targetOffsetMs).isEqualTo(12000); + assertThat(window.defaultPositionUs).isEqualTo(4000000); + } + + @Test + public void refreshPlaylist_targetLiveOffsetRemainsInWindow() + throws TimeoutException, IOException { + String playlistUri1 = "fake://foo.bar/media0/playlist1.m3u8"; + // The playlist has a duration of 16 seconds and a hold back of 12 seconds. + String playlist1 = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:4.00000,\n" + + "fileSequence0.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence1.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence2.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence3.ts\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK:12"; + // The second playlist defines a different hold back. + String playlistUri2 = "fake://foo.bar/media0/playlist2.m3u8"; + String playlist2 = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:4\n" + + "#EXTINF:4.00000,\n" + + "fileSequence4.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence5.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence6.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence7.ts\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK:14"; + // The third playlist has a duration of 8 seconds. + String playlistUri3 = "fake://foo.bar/media0/playlist3.m3u8"; + String playlist3 = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:4\n" + + "#EXTINF:4.00000,\n" + + "fileSequence8.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence9.ts\n" + + "#EXTINF:4.00000,\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK:12"; + // The third playlist has a duration of 16 seconds but the target live offset should remain at + // 8 seconds. + String playlistUri4 = "fake://foo.bar/media0/playlist4.m3u8"; + String playlist4 = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:3\n" + + "#EXT-X-MEDIA-SEQUENCE:4\n" + + "#EXTINF:4.00000,\n" + + "fileSequence10.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence11.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence12.ts\n" + + "#EXTINF:4.00000,\n" + + "fileSequence13.ts\n" + + "#EXTINF:4.00000,\n" + + "#EXT-X-SERVER-CONTROL:HOLD-BACK:12"; + + HlsMediaSource.Factory factory = createHlsMediaSourceFactory(playlistUri1, playlist1); + MediaItem mediaItem = new MediaItem.Builder().setUri(playlistUri1).build(); + HlsMediaSource mediaSource = factory.createMediaSource(mediaItem); + HlsMediaPlaylist secondPlaylist = parseHlsMediaPlaylist(playlistUri2, playlist2); + HlsMediaPlaylist thirdPlaylist = parseHlsMediaPlaylist(playlistUri3, playlist3); + HlsMediaPlaylist fourthPlaylist = parseHlsMediaPlaylist(playlistUri4, playlist4); + List timelines = new ArrayList<>(); + MediaSource.MediaSourceCaller mediaSourceCaller = (source, timeline) -> timelines.add(timeline); + + mediaSource.prepareSource(mediaSourceCaller, null); + runMainLooperUntil(() -> timelines.size() == 1); + mediaSource.onPrimaryPlaylistRefreshed(secondPlaylist); + runMainLooperUntil(() -> timelines.size() == 2); + mediaSource.onPrimaryPlaylistRefreshed(thirdPlaylist); + runMainLooperUntil(() -> timelines.size() == 3); + mediaSource.onPrimaryPlaylistRefreshed(fourthPlaylist); + runMainLooperUntil(() -> timelines.size() == 4); + + Timeline.Window window = new Timeline.Window(); + assertThat(timelines.get(0).getWindow(0, window).liveConfiguration.targetOffsetMs) + .isEqualTo(12000); + assertThat(timelines.get(1).getWindow(0, window).liveConfiguration.targetOffsetMs) + .isEqualTo(12000); + assertThat(timelines.get(2).getWindow(0, window).liveConfiguration.targetOffsetMs) + .isEqualTo(8000); + assertThat(timelines.get(3).getWindow(0, window).liveConfiguration.targetOffsetMs) + .isEqualTo(8000); + } + + private static HlsMediaSource.Factory createHlsMediaSourceFactory( + String playlistUri, String playlist) { + FakeDataSet fakeDataSet = new FakeDataSet().setData(playlistUri, Util.getUtf8Bytes(playlist)); + return new HlsMediaSource.Factory( + dataType -> new FakeDataSource.Factory().setFakeDataSet(fakeDataSet).createDataSource()) + .setElapsedRealTimeOffsetMs(0); + } + + /** Prepares the media source and waits until the timeline is updated. */ + private static Timeline prepareAndWaitForTimeline(HlsMediaSource mediaSource) + throws TimeoutException { + AtomicReference receivedTimeline = new AtomicReference<>(); + mediaSource.prepareSource( + (source, timeline) -> receivedTimeline.set(timeline), /* mediaTransferListener= */ null); + runMainLooperUntil(() -> receivedTimeline.get() != null); + return receivedTimeline.get(); + } + + private static HlsMediaPlaylist parseHlsMediaPlaylist(String playlistUri, String playlist) + throws IOException { + return (HlsMediaPlaylist) + new HlsPlaylistParser() + .parse(Uri.parse(playlistUri), new ByteArrayInputStream(Util.getUtf8Bytes(playlist))); } } diff --git a/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistTrackerTest.java b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistTrackerTest.java new file mode 100644 index 00000000000..80060b15f85 --- /dev/null +++ b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistTrackerTest.java @@ -0,0 +1,450 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.source.hls.playlist; + +import static com.google.common.truth.Truth.assertThat; + +import android.net.Uri; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.robolectric.RobolectricUtil; +import com.google.android.exoplayer2.source.MediaSourceEventListener; +import com.google.android.exoplayer2.testutil.TestUtil; +import com.google.android.exoplayer2.upstream.DataSource; +import com.google.android.exoplayer2.upstream.DefaultHttpDataSource; +import com.google.android.exoplayer2.upstream.DefaultLoadErrorHandlingPolicy; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.TimeoutException; +import java.util.concurrent.atomic.AtomicInteger; +import okhttp3.HttpUrl; +import okhttp3.mockwebserver.MockResponse; +import okhttp3.mockwebserver.MockWebServer; +import okio.Buffer; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link DefaultHlsPlaylistTracker}. */ +@RunWith(AndroidJUnit4.class) +public class DefaultHlsPlaylistTrackerTest { + + private static final String SAMPLE_M3U8_LIVE_MASTER = "media/m3u8/live_low_latency_master"; + private static final String SAMPLE_M3U8_LIVE_MASTER_MEDIA_URI_WITH_PARAM = + "media/m3u8/live_low_latency_master_media_uri_with_param"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL = + "media/m3u8/live_low_latency_media_can_skip_until"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL_FULL_RELOAD_AFTER_ERROR = + "media/m3u8/live_low_latency_media_can_skip_until_full_reload_after_error"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_DATERANGES = + "media/m3u8/live_low_latency_media_can_skip_dateranges"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_SKIPPED = + "media/m3u8/live_low_latency_media_can_skip_skipped"; + private static final String + SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_SKIPPED_MEDIA_SEQUENCE_NO_OVERLAPPING = + "media/m3u8/live_low_latency_media_can_skip_skipped_media_sequence_no_overlapping"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_NOT_SKIP = + "media/m3u8/live_low_latency_media_can_not_skip"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_NOT_SKIP_NEXT = + "media/m3u8/live_low_latency_media_can_not_skip_next"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD = + "media/m3u8/live_low_latency_media_can_block_reload"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_NEXT = + "media/m3u8/live_low_latency_media_can_block_reload_next"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY = + "media/m3u8/live_low_latency_media_can_block_reload_low_latency"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_NEXT = + "media/m3u8/live_low_latency_media_can_block_reload_low_latency_next"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_FULL_SEGMENT = + "media/m3u8/live_low_latency_media_can_block_reload_low_latency_full_segment"; + private static final String + SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_FULL_SEGMENT_NEXT = + "media/m3u8/live_low_latency_media_can_block_reload_low_latency_full_segment_next"; + private static final String + SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_FULL_SEGMENT_PRELOAD = + "media/m3u8/live_low_latency_media_can_block_reload_low_latency_full_segment_preload"; + private static final String + SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_FULL_SEGMENT_PRELOAD_NEXT = + "media/m3u8/live_low_latency_media_can_block_reload_low_latency_full_segment_preload_next"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL_AND_BLOCK_RELOAD = + "media/m3u8/live_low_latency_media_can_skip_until_and_block_reload"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL_AND_BLOCK_RELOAD_NEXT = + "media/m3u8/live_low_latency_media_can_skip_until_and_block_reload_next"; + private static final String SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL_AND_BLOCK_RELOAD_NEXT_SKIPPED = + "media/m3u8/live_low_latency_media_can_skip_until_and_block_reload_next_skipped"; + + private MockWebServer mockWebServer; + private int enqueueCounter; + private int assertedRequestCounter; + + @Before + public void setUp() { + mockWebServer = new MockWebServer(); + enqueueCounter = 0; + assertedRequestCounter = 0; + } + + @After + public void tearDown() throws IOException { + assertThat(assertedRequestCounter).isEqualTo(enqueueCounter); + mockWebServer.shutdown(); + } + + @Test + public void start_playlistCanNotSkip_requestsFullUpdate() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] {"master.m3u8", "/media0/playlist.m3u8", "/media0/playlist.m3u8"}, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_NOT_SKIP), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_NOT_SKIP_NEXT)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + HlsMediaPlaylist firstFullPlaylist = mediaPlaylists.get(0); + assertThat(firstFullPlaylist.mediaSequence).isEqualTo(10); + assertThat(firstFullPlaylist.segments.get(0).url).isEqualTo("fileSequence10.ts"); + assertThat(firstFullPlaylist.segments.get(5).url).isEqualTo("fileSequence15.ts"); + assertThat(firstFullPlaylist.segments).hasSize(6); + HlsMediaPlaylist secondFullPlaylist = mediaPlaylists.get(1); + assertThat(secondFullPlaylist.mediaSequence).isEqualTo(11); + assertThat(secondFullPlaylist.segments.get(0).url).isEqualTo("fileSequence11.ts"); + assertThat(secondFullPlaylist.segments.get(5).url).isEqualTo("fileSequence16.ts"); + assertThat(secondFullPlaylist.segments).hasSize(6); + assertThat(secondFullPlaylist.segments).containsNoneIn(firstFullPlaylist.segments); + } + + @Test + public void start_playlistCanSkip_requestsDeltaUpdateAndExpandsSkippedSegments() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", "/media0/playlist.m3u8", "/media0/playlist.m3u8?_HLS_skip=YES" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_SKIPPED)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + HlsMediaPlaylist initialPlaylistWithAllSegments = mediaPlaylists.get(0); + assertThat(initialPlaylistWithAllSegments.mediaSequence).isEqualTo(10); + assertThat(initialPlaylistWithAllSegments.segments).hasSize(6); + HlsMediaPlaylist mergedPlaylist = mediaPlaylists.get(1); + assertThat(mergedPlaylist.mediaSequence).isEqualTo(11); + assertThat(mergedPlaylist.segments).hasSize(6); + // First 2 segments of the merged playlist need to be copied from the previous playlist. + assertThat(mergedPlaylist.segments.get(0).url) + .isEqualTo(initialPlaylistWithAllSegments.segments.get(1).url); + assertThat(mergedPlaylist.segments.get(0).relativeStartTimeUs).isEqualTo(0); + assertThat(mergedPlaylist.segments.get(1).url) + .isEqualTo(initialPlaylistWithAllSegments.segments.get(2).url); + assertThat(mergedPlaylist.segments.get(1).relativeStartTimeUs).isEqualTo(4000000); + } + + @Test + public void start_playlistCanSkip_missingSegments_reloadsWithoutSkipping() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", + "/media0/playlist.m3u8", + "/media0/playlist.m3u8?_HLS_skip=YES", + "/media0/playlist.m3u8" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_SKIPPED_MEDIA_SEQUENCE_NO_OVERLAPPING), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL_FULL_RELOAD_AFTER_ERROR)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + HlsMediaPlaylist initialPlaylistWithAllSegments = mediaPlaylists.get(0); + assertThat(initialPlaylistWithAllSegments.mediaSequence).isEqualTo(10); + assertThat(initialPlaylistWithAllSegments.segments).hasSize(6); + HlsMediaPlaylist mergedPlaylist = mediaPlaylists.get(1); + assertThat(mergedPlaylist.mediaSequence).isEqualTo(20); + assertThat(mergedPlaylist.segments).hasSize(6); + } + + @Test + public void start_playlistCanSkipDataRanges_requestsDeltaUpdateV2() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", "/media0/playlist.m3u8", "/media0/playlist.m3u8?_HLS_skip=v2" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_DATERANGES), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_SKIPPED)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + // Finding the media sequence of the second playlist request asserts that the second request has + // been made with the correct uri parameter appended. + assertThat(mediaPlaylists.get(1).mediaSequence).isEqualTo(11); + } + + @Test + public void start_playlistCanSkipAndUriWithParams_preservesOriginalParams() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", + "/media0/playlist.m3u8?param1=1¶m2=2", + "/media0/playlist.m3u8?param1=1¶m2=2&_HLS_skip=YES" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER_MEDIA_URI_WITH_PARAM), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_SKIPPED)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + // Finding the media sequence of the second playlist request asserts that the second request has + // been made with the original uri parameters preserved and the additional param concatenated + // correctly. + assertThat(mediaPlaylists.get(1).mediaSequence).isEqualTo(11); + } + + @Test + public void start_playlistCanBlockReload_requestBlockingReloadWithCorrectMediaSequence() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", "/media0/playlist.m3u8", "/media0/playlist.m3u8?_HLS_msn=14" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_NEXT)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + assertThat(mediaPlaylists.get(0).mediaSequence).isEqualTo(10); + assertThat(mediaPlaylists.get(1).mediaSequence).isEqualTo(11); + } + + @Test + public void + start_playlistCanBlockReloadLowLatency_requestBlockingReloadWithCorrectMediaSequenceAndPart() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", + "/media0/playlist.m3u8", + "/media0/playlist.m3u8?_HLS_msn=14&_HLS_part=1" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_NEXT)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + assertThat(mediaPlaylists.get(0).mediaSequence).isEqualTo(10); + assertThat(mediaPlaylists.get(0).segments).hasSize(4); + assertThat(mediaPlaylists.get(0).trailingParts).hasSize(2); + assertThat(mediaPlaylists.get(1).mediaSequence).isEqualTo(10); + assertThat(mediaPlaylists.get(1).segments).hasSize(4); + assertThat(mediaPlaylists.get(1).trailingParts).hasSize(3); + } + + @Test + public void start_playlistCanBlockReloadLowLatencyFullSegment_correctMsnAndPartParams() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", + "/media0/playlist.m3u8", + "/media0/playlist.m3u8?_HLS_msn=14&_HLS_part=0" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_FULL_SEGMENT), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_FULL_SEGMENT_NEXT)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + assertThat(mediaPlaylists.get(0).mediaSequence).isEqualTo(10); + assertThat(mediaPlaylists.get(0).segments).hasSize(4); + assertThat(mediaPlaylists.get(0).trailingParts).isEmpty(); + assertThat(mediaPlaylists.get(1).mediaSequence).isEqualTo(10); + assertThat(mediaPlaylists.get(1).segments).hasSize(4); + assertThat(mediaPlaylists.get(1).trailingParts).hasSize(1); + } + + @Test + public void start_playlistCanBlockReloadLowLatencyFullSegmentWithPreloadPart_ignoresPreloadPart() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", + "/media0/playlist.m3u8", + "/media0/playlist.m3u8?_HLS_msn=14&_HLS_part=0" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse( + SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_FULL_SEGMENT_PRELOAD), + getMockResponse( + SAMPLE_M3U8_LIVE_MEDIA_CAN_BLOCK_RELOAD_LOW_LATENCY_FULL_SEGMENT_PRELOAD_NEXT)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 2); + + assertRequestUrlsCalled(httpUrls); + assertThat(mediaPlaylists.get(0).mediaSequence).isEqualTo(10); + assertThat(mediaPlaylists.get(0).segments).hasSize(4); + assertThat(mediaPlaylists.get(0).trailingParts).hasSize(1); + assertThat(mediaPlaylists.get(1).mediaSequence).isEqualTo(10); + assertThat(mediaPlaylists.get(1).segments).hasSize(4); + assertThat(mediaPlaylists.get(1).trailingParts).hasSize(2); + } + + @Test + public void start_httpBadRequest_forcesFullNonBlockingPlaylistRequest() + throws IOException, TimeoutException, InterruptedException { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/master.m3u8", + "/media0/playlist.m3u8", + "/media0/playlist.m3u8?_HLS_msn=16&_HLS_skip=YES", + "/media0/playlist.m3u8", + "/media0/playlist.m3u8?_HLS_msn=17&_HLS_skip=YES" + }, + getMockResponse(SAMPLE_M3U8_LIVE_MASTER), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL_AND_BLOCK_RELOAD), + new MockResponse().setResponseCode(400), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL_AND_BLOCK_RELOAD_NEXT), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_SKIP_UNTIL_AND_BLOCK_RELOAD_NEXT_SKIPPED)); + + List mediaPlaylists = + runPlaylistTrackerAndCollectMediaPlaylists( + /* dataSourceFactory= */ new DefaultHttpDataSource.Factory(), + Uri.parse(mockWebServer.url("/master.m3u8").toString()), + /* awaitedMediaPlaylistCount= */ 3); + + assertRequestUrlsCalled(httpUrls); + assertThat(mediaPlaylists.get(0).mediaSequence).isEqualTo(10); + assertThat(mediaPlaylists.get(1).mediaSequence).isEqualTo(11); + assertThat(mediaPlaylists.get(2).mediaSequence).isEqualTo(12); + } + + private List enqueueWebServerResponses(String[] paths, MockResponse... mockResponses) { + assertThat(paths).hasLength(mockResponses.length); + for (MockResponse mockResponse : mockResponses) { + enqueueCounter++; + mockWebServer.enqueue(mockResponse); + } + List urls = new ArrayList<>(); + for (String path : paths) { + urls.add(mockWebServer.url(path)); + } + return urls; + } + + private void assertRequestUrlsCalled(List httpUrls) throws InterruptedException { + for (HttpUrl url : httpUrls) { + assertedRequestCounter++; + assertThat(url.toString()).endsWith(mockWebServer.takeRequest().getPath()); + } + } + + private static List runPlaylistTrackerAndCollectMediaPlaylists( + DataSource.Factory dataSourceFactory, Uri masterPlaylistUri, int awaitedMediaPlaylistCount) + throws TimeoutException { + + DefaultHlsPlaylistTracker defaultHlsPlaylistTracker = + new DefaultHlsPlaylistTracker( + dataType -> dataSourceFactory.createDataSource(), + new DefaultLoadErrorHandlingPolicy(), + new DefaultHlsPlaylistParserFactory()); + + List mediaPlaylists = new ArrayList<>(); + AtomicInteger playlistCounter = new AtomicInteger(); + defaultHlsPlaylistTracker.start( + masterPlaylistUri, + new MediaSourceEventListener.EventDispatcher(), + mediaPlaylist -> { + mediaPlaylists.add(mediaPlaylist); + playlistCounter.addAndGet(1); + }); + + RobolectricUtil.runMainLooperUntil(() -> playlistCounter.get() >= awaitedMediaPlaylistCount); + + defaultHlsPlaylistTracker.stop(); + return mediaPlaylists; + } + + private static MockResponse getMockResponse(String assetFile) throws IOException { + return new MockResponse().setResponseCode(200).setBody(new Buffer().write(getBytes(assetFile))); + } + + private static byte[] getBytes(String filename) throws IOException { + return TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), filename); + } +} diff --git a/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/playlist/HlsMediaPlaylistParserTest.java b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/playlist/HlsMediaPlaylistParserTest.java index 42b51056cf7..b97c940c95c 100644 --- a/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/playlist/HlsMediaPlaylistParserTest.java +++ b/library/hls/src/test/java/com/google/android/exoplayer2/source/hls/playlist/HlsMediaPlaylistParserTest.java @@ -19,12 +19,15 @@ import static org.junit.Assert.fail; import android.net.Uri; +import android.util.Base64; import androidx.annotation.Nullable; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ParserException; +import com.google.android.exoplayer2.extractor.mp4.PsshAtomUtil; import com.google.android.exoplayer2.source.hls.playlist.HlsMediaPlaylist.Segment; import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.Iterables; import java.io.ByteArrayInputStream; import java.io.IOException; import java.io.InputStream; @@ -45,7 +48,7 @@ public void parseMediaPlaylist() throws Exception { "#EXTM3U\n" + "#EXT-X-VERSION:3\n" + "#EXT-X-PLAYLIST-TYPE:VOD\n" - + "#EXT-X-START:TIME-OFFSET=-25" + + "#EXT-X-START:TIME-OFFSET=-25\n" + "#EXT-X-TARGETDURATION:8\n" + "#EXT-X-MEDIA-SEQUENCE:2679\n" + "#EXT-X-DISCONTINUITY-SEQUENCE:4\n" @@ -86,6 +89,8 @@ public void parseMediaPlaylist() throws Exception { assertThat(mediaPlaylist.version).isEqualTo(3); assertThat(mediaPlaylist.hasEndTag).isTrue(); assertThat(mediaPlaylist.protectionSchemes).isNull(); + assertThat(mediaPlaylist.targetDurationUs).isEqualTo(8000000); + assertThat(mediaPlaylist.partTargetDurationUs).isEqualTo(C.TIME_UNSET); List segments = mediaPlaylist.segments; assertThat(segments).isNotNull(); assertThat(segments).hasSize(5); @@ -219,6 +224,7 @@ public void parseSampleAesCtrMethod() throws Exception { + "https://priv.example.com/2.ts\n" + "#EXT-X-ENDLIST\n"; InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + HlsMediaPlaylist playlist = (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); assertThat(playlist.protectionSchemes.schemeType).isEqualTo(C.CENC_TYPE_cenc); @@ -226,6 +232,717 @@ public void parseSampleAesCtrMethod() throws Exception { assertThat(playlist.protectionSchemes.get(0).hasData()).isFalse(); } + @Test + public void parseMediaPlaylist_withPartMediaInformation_succeeds() throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-SERVER-CONTROL:PART-HOLD-BACK=1.234\n" + + "#EXT-X-PART-INF:PART-TARGET=0.5\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-PROGRAM-DATE-TIME:2019-02-14T02:13:36.106Z\n" + + "#EXT-X-MAP:URI=\"init.mp4\"\n" + + "#EXTINF:4.00008,\n" + + "fileSequence266.mp4"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.partTargetDurationUs).isEqualTo(500000); + } + + @Test + public void parseMediaPlaylist_withoutServerControl_serverControlDefaultValues() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXTINF:8,\n" + + "https://priv.example.com/1.ts\n" + + "#EXT-X-ENDLIST\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + assertThat(playlist.serverControl.canBlockReload).isFalse(); + assertThat(playlist.serverControl.partHoldBackUs).isEqualTo(C.TIME_UNSET); + assertThat(playlist.serverControl.holdBackUs).isEqualTo(C.TIME_UNSET); + assertThat(playlist.serverControl.skipUntilUs).isEqualTo(C.TIME_UNSET); + assertThat(playlist.serverControl.canSkipDateRanges).isFalse(); + } + + @Test + public void parseMediaPlaylist_withServerControl_succeeds() throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-SERVER-CONTROL:CAN-BLOCK-RELOAD=YES,HOLD-BACK=18.5,PART-HOLD-BACK=1.234," + + "CAN-SKIP-UNTIL=24.0,CAN-SKIP-DATERANGES=YES\n" + + "#EXT-X-PART-INF:PART-TARGET=0.5\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-PROGRAM-DATE-TIME:2019-02-14T02:13:36.106Z\n" + + "#EXT-X-MAP:URI=\"init.mp4\"\n" + + "#EXTINF:4.00008,\n" + + "fileSequence266.mp4"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.serverControl.canBlockReload).isTrue(); + assertThat(playlist.serverControl.partHoldBackUs).isEqualTo(1234000); + assertThat(playlist.serverControl.holdBackUs).isEqualTo(18500000); + assertThat(playlist.serverControl.skipUntilUs).isEqualTo(24000000); + assertThat(playlist.serverControl.canSkipDateRanges).isTrue(); + } + + @Test + public void parseMediaPlaylist_withSkippedSegments_correctlyMergedSegments() throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String previousPlaylistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-DISCONTINUITY-SEQUENCE:1234\n" + + "#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=24.0\n" + + "#EXT-X-MEDIA-SEQUENCE:263\n" + + "#EXTINF:4.00008,\n" + + "fileSequence264.mp4\n" + + "#EXT-X-DISCONTINUITY\n" + + "#EXTINF:4.00008,\n" + + "fileSequence265.mp4\n" + + "#EXTINF:4.00008,\n" + + "fileSequence266.mp4"; + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-DISCONTINUITY-SEQUENCE:1234\n" + + "#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=24.0\n" + + "#EXT-X-MEDIA-SEQUENCE:265\n" + + "#EXT-X-SKIP:SKIPPED-SEGMENTS=1\n" + + "#EXTINF:4.00008,\n" + + "fileSequence266.mp4" + + "#EXTINF:4.00008,\n" + + "fileSequence267.mp4\n"; + InputStream previousInputStream = + new ByteArrayInputStream(Util.getUtf8Bytes(previousPlaylistString)); + HlsMediaPlaylist previousPlaylist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, previousInputStream); + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) + new HlsPlaylistParser(HlsMasterPlaylist.EMPTY, previousPlaylist) + .parse(playlistUri, inputStream); + + assertThat(playlist.segments).hasSize(3); + assertThat(playlist.segments.get(1).relativeStartTimeUs).isEqualTo(4000079); + assertThat(previousPlaylist.segments.get(0).relativeDiscontinuitySequence).isEqualTo(0); + assertThat(previousPlaylist.segments.get(1).relativeDiscontinuitySequence).isEqualTo(1); + assertThat(previousPlaylist.segments.get(2).relativeDiscontinuitySequence).isEqualTo(1); + assertThat(playlist.segments.get(0).relativeDiscontinuitySequence).isEqualTo(1); + assertThat(playlist.segments.get(1).relativeDiscontinuitySequence).isEqualTo(1); + assertThat(playlist.segments.get(2).relativeDiscontinuitySequence).isEqualTo(1); + } + + @Test + public void parseMediaPlaylist_withSkippedSegments_correctlyMergedParts() throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String previousPlaylistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-DISCONTINUITY-SEQUENCE:1234\n" + + "#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=24.0\n" + + "#EXT-X-MEDIA-SEQUENCE:264\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part264.1.ts\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part264.2.ts\"\n" + + "#EXTINF:4.00008,\n" + + "fileSequence264.mp4\n" + + "#EXT-X-DISCONTINUITY\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part265.1.ts\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part265.2.ts\"\n" + + "#EXTINF:4.00008,\n" + + "fileSequence265.mp4\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part266.1.ts\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part266.2.ts\"\n" + + "#EXTINF:4.00008,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\""; + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-DISCONTINUITY-SEQUENCE:1234\n" + + "#EXT-X-SERVER-CONTROL:CAN-SKIP-UNTIL=24.0\n" + + "#EXT-X-MEDIA-SEQUENCE:265\n" + + "#EXT-X-SKIP:SKIPPED-SEGMENTS=2\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\""; + InputStream previousInputStream = + new ByteArrayInputStream(Util.getUtf8Bytes(previousPlaylistString)); + HlsMediaPlaylist previousPlaylist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, previousInputStream); + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) + new HlsPlaylistParser(HlsMasterPlaylist.EMPTY, previousPlaylist) + .parse(playlistUri, inputStream); + + assertThat(playlist.segments).hasSize(2); + assertThat(playlist.segments.get(0).relativeStartTimeUs).isEqualTo(0); + assertThat(playlist.segments.get(0).parts.get(0).relativeStartTimeUs).isEqualTo(0); + assertThat(playlist.segments.get(0).parts.get(0).relativeDiscontinuitySequence).isEqualTo(1); + assertThat(playlist.segments.get(0).parts.get(1).relativeStartTimeUs).isEqualTo(2000000); + assertThat(playlist.segments.get(0).parts.get(1).relativeDiscontinuitySequence).isEqualTo(1); + assertThat(playlist.segments.get(1).relativeStartTimeUs).isEqualTo(4000079); + assertThat(playlist.segments.get(1).parts.get(0).relativeStartTimeUs).isEqualTo(4000079); + assertThat(playlist.segments.get(1).parts.get(1).relativeDiscontinuitySequence).isEqualTo(1); + assertThat(playlist.segments.get(1).parts.get(1).relativeStartTimeUs).isEqualTo(6000079); + assertThat(playlist.segments.get(1).parts.get(1).relativeDiscontinuitySequence).isEqualTo(1); + assertThat(playlist.trailingParts.get(0).relativeStartTimeUs).isEqualTo(8000158); + assertThat(playlist.trailingParts.get(0).relativeDiscontinuitySequence).isEqualTo(1); + } + + @Test + public void parseMediaPlaylist_withParts_parsesPartWithAllAttributes() throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-PART:DURATION=2.00000,GAP=YES," + + "INDEPENDENT=YES,URI=\"part267.1.ts\"\n" + + "#EXT-X-PART:DURATION=2.00000,BYTERANGE=\"1000@1234\",URI=\"part267.2.ts\"\n" + + "#EXTINF:4.00000,\n" + + "fileSequence267.ts\n" + + "#EXT-X-PART:DURATION=2.00000, BYTERANGE=\"1000@1234\",URI=\"part268.1.ts\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part268.2.ts\", BYTERANGE=\"1000\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments.get(0).parts).isEmpty(); + assertThat(playlist.segments.get(1).parts).hasSize(2); + assertThat(playlist.trailingParts).hasSize(2); + + HlsMediaPlaylist.Part firstPart = playlist.segments.get(1).parts.get(0); + assertThat(firstPart.byteRangeLength).isEqualTo(C.LENGTH_UNSET); + assertThat(firstPart.byteRangeOffset).isEqualTo(0); + assertThat(firstPart.durationUs).isEqualTo(2_000_000); + assertThat(firstPart.relativeStartTimeUs).isEqualTo(playlist.segments.get(0).durationUs); + assertThat(firstPart.isIndependent).isTrue(); + assertThat(firstPart.isPreload).isFalse(); + assertThat(firstPart.hasGapTag).isTrue(); + assertThat(firstPart.url).isEqualTo("part267.1.ts"); + HlsMediaPlaylist.Part secondPart = playlist.segments.get(1).parts.get(1); + assertThat(secondPart.byteRangeLength).isEqualTo(1000); + assertThat(secondPart.byteRangeOffset).isEqualTo(1234); + // Assert trailing parts. + HlsMediaPlaylist.Part thirdPart = playlist.trailingParts.get(0); + // Assert tailing parts. + assertThat(thirdPart.byteRangeLength).isEqualTo(1000); + assertThat(thirdPart.byteRangeOffset).isEqualTo(1234); + assertThat(thirdPart.relativeStartTimeUs).isEqualTo(8_000_000); + HlsMediaPlaylist.Part lastPart = playlist.trailingParts.get(1); + assertThat(lastPart.relativeStartTimeUs).isEqualTo(10_000_000); + assertThat(lastPart.hasGapTag).isFalse(); + assertThat(lastPart.isIndependent).isFalse(); + assertThat(lastPart.byteRangeLength).isEqualTo(1000); + assertThat(lastPart.byteRangeOffset).isEqualTo(2234); + } + + @Test + public void parseMediaPlaylist_withPartAndAesPlayReadyKey_correctDrmInitData() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-KEY:METHOD=SAMPLE-AES," + + "KEYFORMAT=\"com.microsoft.playready\"," + + "URI=\"data:text/plain;base64,RG9uJ3QgeW91IGdldCB0aXJlZCBvZiBkb2luZyB0aGlzPw==\"\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.ts\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments.get(0).parts).isEmpty(); + assertThat(playlist.protectionSchemes.schemeType).isEqualTo("cbcs"); + HlsMediaPlaylist.Part part = playlist.trailingParts.get(0); + assertThat(part.drmInitData.schemeType).isEqualTo("cbcs"); + assertThat(part.drmInitData.schemeDataCount).isEqualTo(1); + assertThat(part.drmInitData.get(0).uuid).isEqualTo(C.PLAYREADY_UUID); + assertThat(part.drmInitData.get(0).data) + .isEqualTo( + PsshAtomUtil.buildPsshAtom( + C.PLAYREADY_UUID, + Base64.decode("RG9uJ3QgeW91IGdldCB0aXJlZCBvZiBkb2luZyB0aGlzPw==", Base64.DEFAULT))); + } + + @Test + public void parseMediaPlaylist_withPartAndAesPlayReadyWithOutPrecedingSegment_correctDrmInitData() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-KEY:METHOD=SAMPLE-AES," + + "KEYFORMAT=\"com.microsoft.playready\"," + + "URI=\"data:text/plain;base64,RG9uJ3QgeW91IGdldCB0aXJlZCBvZiBkb2luZyB0aGlzPw==\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments).isEmpty(); + assertThat(playlist.protectionSchemes.schemeType).isEqualTo("cbcs"); + HlsMediaPlaylist.Part part = playlist.trailingParts.get(0); + assertThat(part.drmInitData.schemeType).isEqualTo("cbcs"); + assertThat(part.drmInitData.schemeDataCount).isEqualTo(1); + assertThat(part.drmInitData.get(0).uuid).isEqualTo(C.PLAYREADY_UUID); + assertThat(part.drmInitData.get(0).data) + .isEqualTo( + PsshAtomUtil.buildPsshAtom( + C.PLAYREADY_UUID, + Base64.decode("RG9uJ3QgeW91IGdldCB0aXJlZCBvZiBkb2luZyB0aGlzPw==", Base64.DEFAULT))); + } + + @Test + public void parseMediaPlaylist_withPartAndAes128_partHasDrmKeyUriAndIV() throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-KEY:METHOD=AES-128,KEYFORMAT=\"identity\"" + + ", IV=0x410C8AC18AA42EFA18B5155484F5FC34,URI=\"fake://foo.bar/uri\"\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.ts\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments.get(0).parts).isEmpty(); + HlsMediaPlaylist.Part part = playlist.trailingParts.get(0); + assertThat(playlist.protectionSchemes).isNull(); + assertThat(part.drmInitData).isNull(); + assertThat(part.fullSegmentEncryptionKeyUri).isEqualTo("fake://foo.bar/uri"); + assertThat(part.encryptionIV).isEqualTo("0x410C8AC18AA42EFA18B5155484F5FC34"); + } + + @Test + public void parseMediaPlaylist_withPartAndAes128WithoutPrecedingSegment_partHasDrmKeyUriAndIV() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-KEY:METHOD=AES-128,KEYFORMAT=\"identity\"" + + ", IV=0x410C8AC18AA42EFA18B5155484F5FC34,URI=\"fake://foo.bar/uri\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments).isEmpty(); + HlsMediaPlaylist.Part part = playlist.trailingParts.get(0); + assertThat(playlist.protectionSchemes).isNull(); + assertThat(part.drmInitData).isNull(); + assertThat(part.fullSegmentEncryptionKeyUri).isEqualTo("fake://foo.bar/uri"); + assertThat(part.encryptionIV).isEqualTo("0x410C8AC18AA42EFA18B5155484F5FC34"); + } + + @Test + public void parseMediaPlaylist_withPreloadHintTypePart_hasPreloadPartWithAllAttributes() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-MAP:URI=\"map.mp4\"\n" + + "#EXT-X-KEY:METHOD=AES-128,KEYFORMAT=\"identity\"" + + ", IV=0x410C8AC18AA42EFA18B5155484F5FC34,URI=\"fake://foo.bar/uri\"\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\"\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,BYTERANGE-START=1234,BYTERANGE-LENGTH=1000," + + "URI=\"filePart267.2.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments.get(0).parts).isEmpty(); + assertThat(playlist.trailingParts).hasSize(2); + HlsMediaPlaylist.Part preloadPart = playlist.trailingParts.get(1); + assertThat(preloadPart.durationUs).isEqualTo(0L); + assertThat(preloadPart.url).isEqualTo("filePart267.2.ts"); + assertThat(preloadPart.byteRangeLength).isEqualTo(1000); + assertThat(preloadPart.byteRangeOffset).isEqualTo(1234); + assertThat(preloadPart.initializationSegment.url).isEqualTo("map.mp4"); + assertThat(preloadPart.encryptionIV).isEqualTo("0x410C8AC18AA42EFA18B5155484F5FC34"); + assertThat(preloadPart.isPreload).isTrue(); + } + + @Test + public void parseMediaPlaylist_withMultiplePreloadHintTypeParts_picksOnlyFirstPreloadPart() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\"\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,URI=\"filePart267.2.ts\"\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,URI=\"filePart267.3.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.trailingParts).hasSize(2); + assertThat(playlist.trailingParts.get(1).url).isEqualTo("filePart267.2.ts"); + assertThat(playlist.trailingParts.get(1).isPreload).isTrue(); + } + + @Test + public void parseMediaPlaylist_withUnboundedPreloadHintTypePart_ignoresPreloadPart() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.1.ts\"\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,URI=\"filePart267.2.ts,BYTERANGE-START=0\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.trailingParts).hasSize(1); + assertThat(Iterables.getLast(playlist.trailingParts).url).isEqualTo("part267.1.ts"); + assertThat(Iterables.getLast(playlist.trailingParts).isPreload).isFalse(); + } + + @Test + public void parseMediaPlaylist_withPreloadHintTypePartAndAesPlayReadyKey_inheritsDrmInitData() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-KEY:METHOD=SAMPLE-AES," + + "KEYFORMAT=\"com.microsoft.playready\"," + + "URI=\"data:text/plain;base64,RG9uJ3QgeW91IGdldCB0aXJlZCBvZiBkb2luZyB0aGlzPw==\"\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,URI=\"filePart267.2.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments.get(0).parts).isEmpty(); + assertThat(playlist.trailingParts).hasSize(1); + assertThat(playlist.protectionSchemes.schemeDataCount).isEqualTo(1); + HlsMediaPlaylist.Part preloadPart = playlist.trailingParts.get(0); + assertThat(preloadPart.drmInitData.schemeType).isEqualTo("cbcs"); + assertThat(preloadPart.drmInitData.schemeDataCount).isEqualTo(1); + assertThat(preloadPart.drmInitData.get(0).uuid).isEqualTo(C.PLAYREADY_UUID); + assertThat(preloadPart.drmInitData.get(0).data) + .isEqualTo( + PsshAtomUtil.buildPsshAtom( + C.PLAYREADY_UUID, + Base64.decode("RG9uJ3QgeW91IGdldCB0aXJlZCBvZiBkb2luZyB0aGlzPw==", Base64.DEFAULT))); + } + + @Test + public void parseMediaPlaylist_withPreloadHintTypePartAndNewAesPlayReadyKey_correctDrmInitData() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-KEY:METHOD=SAMPLE-AES," + + "KEYFORMAT=\"com.microsoft.playready\"," + + "URI=\"data:text/plain;base64,RG9uJ3QgeW91IGdldCB0aXJlZCBvZiBkb2luZyB0aGlzPw==\"\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,URI=\"filePart267.2.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments.get(0).parts).isEmpty(); + assertThat(playlist.trailingParts).hasSize(1); + assertThat(playlist.protectionSchemes.schemeDataCount).isEqualTo(1); + HlsMediaPlaylist.Part preloadPart = playlist.trailingParts.get(0); + assertThat(preloadPart.drmInitData.schemeType).isEqualTo("cbcs"); + assertThat(preloadPart.drmInitData.schemeDataCount).isEqualTo(1); + assertThat(preloadPart.drmInitData.get(0).uuid).isEqualTo(C.PLAYREADY_UUID); + assertThat(preloadPart.drmInitData.get(0).data) + .isEqualTo( + PsshAtomUtil.buildPsshAtom( + C.PLAYREADY_UUID, + Base64.decode("RG9uJ3QgeW91IGdldCB0aXJlZCBvZiBkb2luZyB0aGlzPw==", Base64.DEFAULT))); + } + + @Test + public void parseMediaPlaylist_withPreloadHintTypePartAndAes128_partHasDrmKeyUriAndIV() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-KEY:METHOD=AES-128,KEYFORMAT=\"identity\"" + + ", IV=0x410C8AC18AA42EFA18B5155484F5FC34,URI=\"fake://foo.bar/uri\"\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,URI=\"filePart267.2.ts\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.segments.get(0).parts).isEmpty(); + assertThat(playlist.trailingParts).hasSize(1); + HlsMediaPlaylist.Part preloadPart = playlist.trailingParts.get(0); + assertThat(preloadPart.drmInitData).isNull(); + assertThat(preloadPart.fullSegmentEncryptionKeyUri).isEqualTo("fake://foo.bar/uri"); + assertThat(preloadPart.encryptionIV).isEqualTo("0x410C8AC18AA42EFA18B5155484F5FC34"); + } + + @Test + public void parseMediaPlaylist_withRenditionReportWithoutPartTargetDuration_lastPartIndexUnset() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-RENDITION-REPORT:URI=\"/rendition0.m3u8\",LAST-MSN=100\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.renditionReports).hasSize(1); + HlsMediaPlaylist.RenditionReport report0 = + playlist.renditionReports.get(Uri.parse("https://example.com/rendition0.m3u8")); + assertThat(report0.lastMediaSequence).isEqualTo(100); + assertThat(report0.lastPartIndex).isEqualTo(C.INDEX_UNSET); + } + + @Test + public void + parseMediaPlaylist_withRenditionReportWithoutPartTargetDurationWithoutLastMsn_sameLastMsnAsCurrentPlaylist() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-RENDITION-REPORT:URI=\"/rendition0.m3u8\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.renditionReports).hasSize(1); + HlsMediaPlaylist.RenditionReport report0 = + playlist.renditionReports.get(Uri.parse("https://example.com/rendition0.m3u8")); + assertThat(report0.lastMediaSequence).isEqualTo(266); + assertThat(report0.lastPartIndex).isEqualTo(C.INDEX_UNSET); + } + + @Test + public void parseMediaPlaylist_withRenditionReportLowLatency_parseAllAttributes() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-PART-INF:PART-TARGET=1\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-RENDITION-REPORT:URI=\"/rendition0.m3u8\",LAST-MSN=100,LAST-PART=2\n" + + "#EXT-X-RENDITION-REPORT:" + + "URI=\"http://foo.bar/rendition2.m3u8\",LAST-MSN=1000,LAST-PART=3\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.renditionReports).hasSize(2); + HlsMediaPlaylist.RenditionReport report0 = + playlist.renditionReports.get(Uri.parse("https://example.com/rendition0.m3u8")); + assertThat(report0.lastMediaSequence).isEqualTo(100); + assertThat(report0.lastPartIndex).isEqualTo(2); + HlsMediaPlaylist.RenditionReport report2 = + playlist.renditionReports.get(Uri.parse("http://foo.bar/rendition2.m3u8")); + assertThat(report2.lastMediaSequence).isEqualTo(1000); + assertThat(report2.lastPartIndex).isEqualTo(3); + } + + @Test + public void + parseMediaPlaylist_withRenditionReportLowLatencyWithoutLastMsn_sameMsnAsCurrentPlaylist() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-PART-INF:PART-TARGET=1\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.0.ts\"\n" + + "#EXT-X-RENDITION-REPORT:URI=\"/rendition0.m3u8\",LAST-PART=2\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.renditionReports).hasSize(1); + HlsMediaPlaylist.RenditionReport report0 = + playlist.renditionReports.get(Uri.parse("https://example.com/rendition0.m3u8")); + assertThat(report0.lastMediaSequence).isEqualTo(267); + assertThat(report0.lastPartIndex).isEqualTo(2); + } + + @Test + public void + parseMediaPlaylist_withRenditionReportLowLatencyWithoutLastPartIndex_sameLastPartIndexAsCurrentPlaylist() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-PART-INF:PART-TARGET=1\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.0.ts\"\n" + + "#EXT-X-RENDITION-REPORT:URI=\"/rendition0.m3u8\",LAST-MSN=100\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.renditionReports).hasSize(1); + HlsMediaPlaylist.RenditionReport report0 = + playlist.renditionReports.get(Uri.parse("https://example.com/rendition0.m3u8")); + assertThat(report0.lastMediaSequence).isEqualTo(100); + assertThat(report0.lastPartIndex).isEqualTo(0); + } + + @Test + public void + parseMediaPlaylist_withRenditionReportLowLatencyWithoutLastPartIndex_ignoredPreloadPart() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-PART-INF:PART-TARGET=1\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part267.0.ts\"\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,URI=\"filePart267.1.ts\"\n" + + "#EXT-X-RENDITION-REPORT:URI=\"/rendition0.m3u8\",LAST-MSN=100\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.trailingParts).hasSize(2); + assertThat(playlist.renditionReports).hasSize(1); + HlsMediaPlaylist.RenditionReport report0 = + playlist.renditionReports.get(Uri.parse("https://example.com/rendition0.m3u8")); + assertThat(report0.lastMediaSequence).isEqualTo(100); + assertThat(report0.lastPartIndex).isEqualTo(0); + } + + @Test + public void parseMediaPlaylist_withRenditionReportLowLatencyFullSegment_rollingPartIndexUriParam() + throws IOException { + Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:4\n" + + "#EXT-X-PART-INF:PART-TARGET=1\n" + + "#EXT-X-VERSION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:266\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part266.0.ts\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part266.1.ts\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part266.2.ts\"\n" + + "#EXT-X-PART:DURATION=2.00000,URI=\"part266.3.ts\"\n" + + "#EXTINF:4.00000,\n" + + "fileSequence266.mp4\n" + + "#EXT-X-PRELOAD-HINT:TYPE=PART,URI=\"filePart267.0.ts\"\n" + + "#EXT-X-RENDITION-REPORT:URI=\"/rendition0.m3u8\"\n"; + InputStream inputStream = new ByteArrayInputStream(Util.getUtf8Bytes(playlistString)); + + HlsMediaPlaylist playlist = + (HlsMediaPlaylist) new HlsPlaylistParser().parse(playlistUri, inputStream); + + assertThat(playlist.renditionReports).hasSize(1); + HlsMediaPlaylist.RenditionReport report0 = + playlist.renditionReports.get(Uri.parse("https://example.com/rendition0.m3u8")); + assertThat(report0.lastMediaSequence).isEqualTo(266); + assertThat(report0.lastPartIndex).isEqualTo(3); + } + @Test public void multipleExtXKeysForSingleSegment() throws Exception { Uri playlistUri = Uri.parse("https://example.com/test.m3u8"); @@ -496,7 +1213,9 @@ public void masterPlaylistAttributeInheritance() throws IOException { /* variableDefinitions= */ Collections.emptyMap(), /* sessionKeyDrmInitData= */ Collections.emptyList()); HlsMediaPlaylist playlistWithInheritance = - (HlsMediaPlaylist) new HlsPlaylistParser(masterPlaylist).parse(playlistUri, inputStream); + (HlsMediaPlaylist) + new HlsPlaylistParser(masterPlaylist, /* previousMediaPlaylist= */ null) + .parse(playlistUri, inputStream); assertThat(playlistWithInheritance.hasIndependentSegments).isTrue(); } @@ -558,7 +1277,9 @@ public void inheritedVariableSubstitution() throws IOException { variableDefinitions, /* sessionKeyDrmInitData= */ Collections.emptyList()); HlsMediaPlaylist playlist = - (HlsMediaPlaylist) new HlsPlaylistParser(masterPlaylist).parse(playlistUri, inputStream); + (HlsMediaPlaylist) + new HlsPlaylistParser(masterPlaylist, /* previousMediaPlaylist= */ null) + .parse(playlistUri, inputStream); for (int i = 1; i <= 4; i++) { assertThat(playlist.segments.get(i - 1).url).isEqualTo("long_path" + i + ".ts"); } diff --git a/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/DefaultSsChunkSource.java b/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/DefaultSsChunkSource.java index 868cea7fd0c..be9aed4393e 100644 --- a/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/DefaultSsChunkSource.java +++ b/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/DefaultSsChunkSource.java @@ -34,7 +34,7 @@ import com.google.android.exoplayer2.source.chunk.MediaChunkIterator; import com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifest; import com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifest.StreamElement; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.DataSource; import com.google.android.exoplayer2.upstream.DataSpec; import com.google.android.exoplayer2.upstream.LoaderErrorThrower; @@ -61,7 +61,7 @@ public SsChunkSource createChunkSource( LoaderErrorThrower manifestLoaderErrorThrower, SsManifest manifest, int elementIndex, - TrackSelection trackSelection, + ExoTrackSelection trackSelection, @Nullable TransferListener transferListener) { DataSource dataSource = dataSourceFactory.createDataSource(); if (transferListener != null) { @@ -78,7 +78,7 @@ public SsChunkSource createChunkSource( private final ChunkExtractor[] chunkExtractors; private final DataSource dataSource; - private TrackSelection trackSelection; + private ExoTrackSelection trackSelection; private SsManifest manifest; private int currentManifestChunkOffset; @@ -95,7 +95,7 @@ public DefaultSsChunkSource( LoaderErrorThrower manifestLoaderErrorThrower, SsManifest manifest, int streamElementIndex, - TrackSelection trackSelection, + ExoTrackSelection trackSelection, DataSource dataSource) { this.manifestLoaderErrorThrower = manifestLoaderErrorThrower; this.manifest = manifest; @@ -163,7 +163,7 @@ public void updateManifest(SsManifest newManifest) { } @Override - public void updateTrackSelection(TrackSelection trackSelection) { + public void updateTrackSelection(ExoTrackSelection trackSelection) { this.trackSelection = trackSelection; } diff --git a/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsChunkSource.java b/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsChunkSource.java index 111393140e5..875b1379c97 100644 --- a/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsChunkSource.java +++ b/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsChunkSource.java @@ -18,13 +18,11 @@ import androidx.annotation.Nullable; import com.google.android.exoplayer2.source.chunk.ChunkSource; import com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifest; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.LoaderErrorThrower; import com.google.android.exoplayer2.upstream.TransferListener; -/** - * A {@link ChunkSource} for SmoothStreaming. - */ +/** A {@link ChunkSource} for SmoothStreaming. */ public interface SsChunkSource extends ChunkSource { /** Factory for {@link SsChunkSource}s. */ @@ -45,7 +43,7 @@ SsChunkSource createChunkSource( LoaderErrorThrower manifestLoaderErrorThrower, SsManifest manifest, int streamElementIndex, - TrackSelection trackSelection, + ExoTrackSelection trackSelection, @Nullable TransferListener transferListener); } @@ -61,5 +59,5 @@ SsChunkSource createChunkSource( * * @param trackSelection The new track selection instance. Must be equivalent to the previous one. */ - void updateTrackSelection(TrackSelection trackSelection); + void updateTrackSelection(ExoTrackSelection trackSelection); } diff --git a/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsMediaPeriod.java b/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsMediaPeriod.java index b6e21cd870b..ae96b941d26 100644 --- a/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsMediaPeriod.java +++ b/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsMediaPeriod.java @@ -31,7 +31,7 @@ import com.google.android.exoplayer2.source.TrackGroupArray; import com.google.android.exoplayer2.source.chunk.ChunkSampleStream; import com.google.android.exoplayer2.source.smoothstreaming.manifest.SsManifest; -import com.google.android.exoplayer2.trackselection.TrackSelection; +import com.google.android.exoplayer2.trackselection.ExoTrackSelection; import com.google.android.exoplayer2.upstream.Allocator; import com.google.android.exoplayer2.upstream.LoadErrorHandlingPolicy; import com.google.android.exoplayer2.upstream.LoaderErrorThrower; @@ -123,7 +123,7 @@ public TrackGroupArray getTrackGroups() { @Override public long selectTracks( - @NullableType TrackSelection[] selections, + @NullableType ExoTrackSelection[] selections, boolean[] mayRetainStreamFlags, @NullableType SampleStream[] streams, boolean[] streamResetFlags, @@ -156,10 +156,10 @@ public long selectTracks( } @Override - public List getStreamKeys(List trackSelections) { + public List getStreamKeys(List trackSelections) { List streamKeys = new ArrayList<>(); for (int selectionIndex = 0; selectionIndex < trackSelections.size(); selectionIndex++) { - TrackSelection trackSelection = trackSelections.get(selectionIndex); + ExoTrackSelection trackSelection = trackSelections.get(selectionIndex); int streamElementIndex = trackGroups.indexOf(trackSelection.getTrackGroup()); for (int i = 0; i < trackSelection.length(); i++) { streamKeys.add(new StreamKey(streamElementIndex, trackSelection.getIndexInTrackGroup(i))); @@ -232,16 +232,12 @@ public void onContinueLoadingRequested(ChunkSampleStream sampleSt // Private methods. - private ChunkSampleStream buildSampleStream(TrackSelection selection, - long positionUs) { + private ChunkSampleStream buildSampleStream( + ExoTrackSelection selection, long positionUs) { int streamElementIndex = trackGroups.indexOf(selection.getTrackGroup()); SsChunkSource chunkSource = chunkSourceFactory.createChunkSource( - manifestLoaderErrorThrower, - manifest, - streamElementIndex, - selection, - transferListener); + manifestLoaderErrorThrower, manifest, streamElementIndex, selection, transferListener); return new ChunkSampleStream<>( manifest.streamElements[streamElementIndex].type, null, diff --git a/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsMediaSource.java b/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsMediaSource.java index a2ebb06936c..bd6f5df197c 100644 --- a/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsMediaSource.java +++ b/library/smoothstreaming/src/main/java/com/google/android/exoplayer2/source/smoothstreaming/SsMediaSource.java @@ -27,8 +27,10 @@ import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.MediaItem; import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.drm.DefaultDrmSessionManagerProvider; import com.google.android.exoplayer2.drm.DrmSessionEventListener; import com.google.android.exoplayer2.drm.DrmSessionManager; +import com.google.android.exoplayer2.drm.DrmSessionManagerProvider; import com.google.android.exoplayer2.offline.FilteringManifestParser; import com.google.android.exoplayer2.offline.StreamKey; import com.google.android.exoplayer2.source.BaseMediaSource; @@ -38,7 +40,6 @@ import com.google.android.exoplayer2.source.MediaLoadData; import com.google.android.exoplayer2.source.MediaPeriod; import com.google.android.exoplayer2.source.MediaSource; -import com.google.android.exoplayer2.source.MediaSourceDrmHelper; import com.google.android.exoplayer2.source.MediaSourceEventListener; import com.google.android.exoplayer2.source.MediaSourceEventListener.EventDispatcher; import com.google.android.exoplayer2.source.MediaSourceFactory; @@ -78,11 +79,11 @@ public final class SsMediaSource extends BaseMediaSource public static final class Factory implements MediaSourceFactory { private final SsChunkSource.Factory chunkSourceFactory; - private final MediaSourceDrmHelper mediaSourceDrmHelper; @Nullable private final DataSource.Factory manifestDataSourceFactory; private CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory; - @Nullable private DrmSessionManager drmSessionManager; + private boolean usingCustomDrmSessionManagerProvider; + private DrmSessionManagerProvider drmSessionManagerProvider; private LoadErrorHandlingPolicy loadErrorHandlingPolicy; private long livePresentationDelayMs; @Nullable private ParsingLoadable.Parser manifestParser; @@ -106,14 +107,14 @@ public Factory(DataSource.Factory dataSourceFactory) { * @param manifestDataSourceFactory A factory for {@link DataSource} instances that will be used * to load (and refresh) the manifest. May be {@code null} if the factory will only ever be * used to create create media sources with sideloaded manifests via {@link - * #createMediaSource(SsManifest, Handler, MediaSourceEventListener)}. + * #createMediaSource(SsManifest, MediaItem)}. */ public Factory( SsChunkSource.Factory chunkSourceFactory, @Nullable DataSource.Factory manifestDataSourceFactory) { this.chunkSourceFactory = checkNotNull(chunkSourceFactory); this.manifestDataSourceFactory = manifestDataSourceFactory; - mediaSourceDrmHelper = new MediaSourceDrmHelper(); + drmSessionManagerProvider = new DefaultDrmSessionManagerProvider(); loadErrorHandlingPolicy = new DefaultLoadErrorHandlingPolicy(); livePresentationDelayMs = DEFAULT_LIVE_PRESENTATION_DELAY_MS; compositeSequenceableLoaderFactory = new DefaultCompositeSequenceableLoaderFactory(); @@ -130,18 +131,10 @@ public Factory setTag(@Nullable Object tag) { return this; } - /** @deprecated Use {@link #setLoadErrorHandlingPolicy(LoadErrorHandlingPolicy)} instead. */ - @Deprecated - public Factory setMinLoadableRetryCount(int minLoadableRetryCount) { - return setLoadErrorHandlingPolicy(new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount)); - } - /** * Sets the {@link LoadErrorHandlingPolicy}. The default value is created by calling {@link * DefaultLoadErrorHandlingPolicy#DefaultLoadErrorHandlingPolicy()}. * - *

    Calling this method overrides any calls to {@link #setMinLoadableRetryCount(int)}. - * * @param loadErrorHandlingPolicy A {@link LoadErrorHandlingPolicy}. * @return This factory, for convenience. */ @@ -199,22 +192,44 @@ public Factory setCompositeSequenceableLoaderFactory( return this; } + @Override + public Factory setDrmSessionManagerProvider( + @Nullable DrmSessionManagerProvider drmSessionManagerProvider) { + if (drmSessionManagerProvider != null) { + this.drmSessionManagerProvider = drmSessionManagerProvider; + this.usingCustomDrmSessionManagerProvider = true; + } else { + this.drmSessionManagerProvider = new DefaultDrmSessionManagerProvider(); + this.usingCustomDrmSessionManagerProvider = false; + } + return this; + } + @Override public Factory setDrmSessionManager(@Nullable DrmSessionManager drmSessionManager) { - this.drmSessionManager = drmSessionManager; + if (drmSessionManager == null) { + setDrmSessionManagerProvider(null); + } else { + setDrmSessionManagerProvider(unusedMediaItem -> drmSessionManager); + } return this; } @Override public Factory setDrmHttpDataSourceFactory( @Nullable HttpDataSource.Factory drmHttpDataSourceFactory) { - mediaSourceDrmHelper.setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + if (!usingCustomDrmSessionManagerProvider) { + ((DefaultDrmSessionManagerProvider) drmSessionManagerProvider) + .setDrmHttpDataSourceFactory(drmHttpDataSourceFactory); + } return this; } @Override public Factory setDrmUserAgent(@Nullable String userAgent) { - mediaSourceDrmHelper.setDrmUserAgent(userAgent); + if (!usingCustomDrmSessionManagerProvider) { + ((DefaultDrmSessionManagerProvider) drmSessionManagerProvider).setDrmUserAgent(userAgent); + } return this; } @@ -285,44 +300,11 @@ public SsMediaSource createMediaSource(SsManifest manifest, MediaItem mediaItem) /* manifestParser= */ null, chunkSourceFactory, compositeSequenceableLoaderFactory, - drmSessionManager != null ? drmSessionManager : mediaSourceDrmHelper.create(mediaItem), + drmSessionManagerProvider.get(mediaItem), loadErrorHandlingPolicy, livePresentationDelayMs); } - /** - * @deprecated Use {@link #createMediaSource(SsManifest)} and {@link #addEventListener(Handler, - * MediaSourceEventListener)} instead. - */ - @Deprecated - public SsMediaSource createMediaSource( - SsManifest manifest, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - SsMediaSource mediaSource = createMediaSource(manifest); - if (eventHandler != null && eventListener != null) { - mediaSource.addEventListener(eventHandler, eventListener); - } - return mediaSource; - } - - /** - * @deprecated Use {@link #createMediaSource(MediaItem)} and {@link #addEventListener(Handler, - * MediaSourceEventListener)} instead. - */ - @SuppressWarnings("deprecation") - @Deprecated - public SsMediaSource createMediaSource( - Uri manifestUri, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - SsMediaSource mediaSource = createMediaSource(manifestUri); - if (eventHandler != null && eventListener != null) { - mediaSource.addEventListener(eventHandler, eventListener); - } - return mediaSource; - } - /** * Returns a new {@link SsMediaSource} using the current parameters. * @@ -362,7 +344,7 @@ public SsMediaSource createMediaSource(MediaItem mediaItem) { manifestParser, chunkSourceFactory, compositeSequenceableLoaderFactory, - drmSessionManager != null ? drmSessionManager : mediaSourceDrmHelper.create(mediaItem), + drmSessionManagerProvider.get(mediaItem), loadErrorHandlingPolicy, livePresentationDelayMs); } @@ -412,162 +394,6 @@ public int[] getSupportedTypes() { private Handler manifestRefreshHandler; - /** - * Constructs an instance to play a given {@link SsManifest}, which must not be live. - * - * @param manifest The manifest. {@link SsManifest#isLive} must be false. - * @param chunkSourceFactory A factory for {@link SsChunkSource} instances. - * @param eventHandler A handler for events. May be null if delivery of events is not required. - * @param eventListener A listener of events. May be null if delivery of events is not required. - * @deprecated Use {@link Factory} instead. - */ - @Deprecated - @SuppressWarnings("deprecation") - public SsMediaSource( - SsManifest manifest, - SsChunkSource.Factory chunkSourceFactory, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - manifest, - chunkSourceFactory, - DefaultLoadErrorHandlingPolicy.DEFAULT_MIN_LOADABLE_RETRY_COUNT, - eventHandler, - eventListener); - } - - /** - * Constructs an instance to play a given {@link SsManifest}, which must not be live. - * - * @param manifest The manifest. {@link SsManifest#isLive} must be false. - * @param chunkSourceFactory A factory for {@link SsChunkSource} instances. - * @param minLoadableRetryCount The minimum number of times to retry if a loading error occurs. - * @param eventHandler A handler for events. May be null if delivery of events is not required. - * @param eventListener A listener of events. May be null if delivery of events is not required. - * @deprecated Use {@link Factory} instead. - */ - @Deprecated - public SsMediaSource( - SsManifest manifest, - SsChunkSource.Factory chunkSourceFactory, - int minLoadableRetryCount, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - new MediaItem.Builder().setUri(Uri.EMPTY).setMimeType(MimeTypes.APPLICATION_SS).build(), - manifest, - /* manifestDataSourceFactory= */ null, - /* manifestParser= */ null, - chunkSourceFactory, - new DefaultCompositeSequenceableLoaderFactory(), - DrmSessionManager.getDummyDrmSessionManager(), - new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount), - DEFAULT_LIVE_PRESENTATION_DELAY_MS); - if (eventHandler != null && eventListener != null) { - addEventListener(eventHandler, eventListener); - } - } - - /** - * Constructs an instance to play the manifest at a given {@link Uri}, which may be live or - * on-demand. - * - * @param manifestUri The manifest {@link Uri}. - * @param manifestDataSourceFactory A factory for {@link DataSource} instances that will be used - * to load (and refresh) the manifest. - * @param chunkSourceFactory A factory for {@link SsChunkSource} instances. - * @param eventHandler A handler for events. May be null if delivery of events is not required. - * @param eventListener A listener of events. May be null if delivery of events is not required. - * @deprecated Use {@link Factory} instead. - */ - @Deprecated - @SuppressWarnings("deprecation") - public SsMediaSource( - Uri manifestUri, - DataSource.Factory manifestDataSourceFactory, - SsChunkSource.Factory chunkSourceFactory, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - manifestUri, - manifestDataSourceFactory, - chunkSourceFactory, - DefaultLoadErrorHandlingPolicy.DEFAULT_MIN_LOADABLE_RETRY_COUNT, - DEFAULT_LIVE_PRESENTATION_DELAY_MS, - eventHandler, - eventListener); - } - - /** - * Constructs an instance to play the manifest at a given {@link Uri}, which may be live or - * on-demand. - * - * @param manifestUri The manifest {@link Uri}. - * @param manifestDataSourceFactory A factory for {@link DataSource} instances that will be used - * to load (and refresh) the manifest. - * @param chunkSourceFactory A factory for {@link SsChunkSource} instances. - * @param minLoadableRetryCount The minimum number of times to retry if a loading error occurs. - * @param livePresentationDelayMs For live playbacks, the duration in milliseconds by which the - * default start position should precede the end of the live window. - * @param eventHandler A handler for events. May be null if delivery of events is not required. - * @param eventListener A listener of events. May be null if delivery of events is not required. - * @deprecated Use {@link Factory} instead. - */ - @Deprecated - @SuppressWarnings("deprecation") - public SsMediaSource( - Uri manifestUri, - DataSource.Factory manifestDataSourceFactory, - SsChunkSource.Factory chunkSourceFactory, - int minLoadableRetryCount, - long livePresentationDelayMs, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this(manifestUri, manifestDataSourceFactory, new SsManifestParser(), chunkSourceFactory, - minLoadableRetryCount, livePresentationDelayMs, eventHandler, eventListener); - } - - /** - * Constructs an instance to play the manifest at a given {@link Uri}, which may be live or - * on-demand. - * - * @param manifestUri The manifest {@link Uri}. - * @param manifestDataSourceFactory A factory for {@link DataSource} instances that will be used - * to load (and refresh) the manifest. - * @param manifestParser A parser for loaded manifest data. - * @param chunkSourceFactory A factory for {@link SsChunkSource} instances. - * @param minLoadableRetryCount The minimum number of times to retry if a loading error occurs. - * @param livePresentationDelayMs For live playbacks, the duration in milliseconds by which the - * default start position should precede the end of the live window. - * @param eventHandler A handler for events. May be null if delivery of events is not required. - * @param eventListener A listener of events. May be null if delivery of events is not required. - * @deprecated Use {@link Factory} instead. - */ - @Deprecated - public SsMediaSource( - Uri manifestUri, - DataSource.Factory manifestDataSourceFactory, - ParsingLoadable.Parser manifestParser, - SsChunkSource.Factory chunkSourceFactory, - int minLoadableRetryCount, - long livePresentationDelayMs, - @Nullable Handler eventHandler, - @Nullable MediaSourceEventListener eventListener) { - this( - new MediaItem.Builder().setUri(manifestUri).setMimeType(MimeTypes.APPLICATION_SS).build(), - /* manifest= */ null, - manifestDataSourceFactory, - manifestParser, - chunkSourceFactory, - new DefaultCompositeSequenceableLoaderFactory(), - DrmSessionManager.getDummyDrmSessionManager(), - new DefaultLoadErrorHandlingPolicy(minLoadableRetryCount), - livePresentationDelayMs); - if (eventHandler != null && eventListener != null) { - addEventListener(eventHandler, eventListener); - } - } - private SsMediaSource( MediaItem mediaItem, @Nullable SsManifest manifest, @@ -783,7 +609,7 @@ private void processManifest() { /* windowDefaultStartPositionUs= */ 0, /* isSeekable= */ true, /* isDynamic= */ manifest.isLive, - /* isLive= */ manifest.isLive, + /* useLiveConfiguration= */ manifest.isLive, manifest, mediaItem); } else if (manifest.isLive) { @@ -806,7 +632,7 @@ private void processManifest() { defaultStartPositionUs, /* isSeekable= */ true, /* isDynamic= */ true, - /* isLive= */ true, + /* useLiveConfiguration= */ true, manifest, mediaItem); } else { @@ -820,7 +646,7 @@ private void processManifest() { /* windowDefaultStartPositionUs= */ 0, /* isSeekable= */ true, /* isDynamic= */ false, - /* isLive= */ false, + /* useLiveConfiguration= */ false, manifest, mediaItem); } diff --git a/library/transformer/README.md b/library/transformer/README.md new file mode 100644 index 00000000000..5de22fa5838 --- /dev/null +++ b/library/transformer/README.md @@ -0,0 +1,10 @@ +# ExoPlayer transformer library module # + +Provides support for transforming media files. + +## Links ## + +* [Javadoc][]: Classes matching `com.google.android.exoplayer2.transformer.*` + belong to this module. + +[Javadoc]: https://exoplayer.dev/doc/reference/index.html diff --git a/library/transformer/build.gradle b/library/transformer/build.gradle new file mode 100644 index 00000000000..6870c9f5774 --- /dev/null +++ b/library/transformer/build.gradle @@ -0,0 +1,47 @@ +// Copyright 2020 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +apply from: "$gradle.ext.exoplayerSettingsDir/common_library_config.gradle" + +android { + buildTypes { + debug { + testCoverageEnabled = true + } + } + + sourceSets.test.assets.srcDir '../../testdata/src/test/assets/' +} + +dependencies { + implementation 'androidx.annotation:annotation:' + androidxAnnotationVersion + implementation project(modulePrefix + 'library-core') + compileOnly 'org.checkerframework:checker-qual:' + checkerframeworkVersion + compileOnly 'org.checkerframework:checker-compat-qual:' + checkerframeworkCompatVersion + compileOnly 'org.jetbrains.kotlin:kotlin-annotations-jvm:' + kotlinAnnotationsVersion + testImplementation project(modulePrefix + 'robolectricutils') + testImplementation project(modulePrefix + 'testutils') + testImplementation project(modulePrefix + 'testdata') + testImplementation 'org.robolectric:robolectric:' + robolectricVersion +} + +ext { + javadocTitle = 'Transformer module' +} +apply from: '../../javadoc_library.gradle' + +ext { + releaseArtifact = 'exoplayer-transformer' + releaseDescription = 'The ExoPlayer library transformer module.' +} +apply from: '../../publish.gradle' diff --git a/library/transformer/src/main/AndroidManifest.xml b/library/transformer/src/main/AndroidManifest.xml new file mode 100644 index 00000000000..3c3792d7a20 --- /dev/null +++ b/library/transformer/src/main/AndroidManifest.xml @@ -0,0 +1,17 @@ + + + + diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/FrameworkMuxer.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/FrameworkMuxer.java new file mode 100644 index 00000000000..5452af42965 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/FrameworkMuxer.java @@ -0,0 +1,194 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Util.SDK_INT; +import static com.google.android.exoplayer2.util.Util.castNonNull; + +import android.media.MediaCodec; +import android.media.MediaFormat; +import android.media.MediaMuxer; +import android.os.ParcelFileDescriptor; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.mediacodec.MediaFormatUtil; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.android.exoplayer2.util.Util; +import java.io.IOException; +import java.lang.reflect.Field; +import java.nio.ByteBuffer; + +/** Muxer implementation that uses a {@link MediaMuxer}. */ +@RequiresApi(18) +/* package */ final class FrameworkMuxer implements Muxer { + + public static final class Factory implements Muxer.Factory { + @Override + public FrameworkMuxer create(String path, String outputMimeType) throws IOException { + MediaMuxer mediaMuxer = new MediaMuxer(path, mimeTypeToMuxerOutputFormat(outputMimeType)); + return new FrameworkMuxer(mediaMuxer, outputMimeType); + } + + @RequiresApi(26) + @Override + public FrameworkMuxer create(ParcelFileDescriptor parcelFileDescriptor, String outputMimeType) + throws IOException { + MediaMuxer mediaMuxer = + new MediaMuxer( + parcelFileDescriptor.getFileDescriptor(), + mimeTypeToMuxerOutputFormat(outputMimeType)); + return new FrameworkMuxer(mediaMuxer, outputMimeType); + } + + @Override + public boolean supportsOutputMimeType(String mimeType) { + try { + mimeTypeToMuxerOutputFormat(mimeType); + } catch (IllegalStateException e) { + return false; + } + return true; + } + } + + private final MediaMuxer mediaMuxer; + private final String outputMimeType; + private final MediaCodec.BufferInfo bufferInfo; + + private boolean isStarted; + + private FrameworkMuxer(MediaMuxer mediaMuxer, String outputMimeType) { + this.mediaMuxer = mediaMuxer; + this.outputMimeType = outputMimeType; + bufferInfo = new MediaCodec.BufferInfo(); + } + + @Override + public boolean supportsSampleMimeType(@Nullable String mimeType) { + // MediaMuxer supported sample formats are documented in MediaMuxer.addTrack(MediaFormat). + boolean isAudio = MimeTypes.isAudio(mimeType); + boolean isVideo = MimeTypes.isVideo(mimeType); + if (outputMimeType.equals(MimeTypes.VIDEO_MP4)) { + if (isVideo) { + return MimeTypes.VIDEO_H263.equals(mimeType) + || MimeTypes.VIDEO_H264.equals(mimeType) + || MimeTypes.VIDEO_MP4V.equals(mimeType) + || (Util.SDK_INT >= 24 && MimeTypes.VIDEO_H265.equals(mimeType)); + } else if (isAudio) { + return MimeTypes.AUDIO_AAC.equals(mimeType) + || MimeTypes.AUDIO_AMR_NB.equals(mimeType) + || MimeTypes.AUDIO_AMR_WB.equals(mimeType); + } + } else if (outputMimeType.equals(MimeTypes.VIDEO_WEBM) && SDK_INT >= 21) { + if (isVideo) { + return MimeTypes.VIDEO_VP8.equals(mimeType) + || (Util.SDK_INT >= 24 && MimeTypes.VIDEO_VP9.equals(mimeType)); + } else if (isAudio) { + return MimeTypes.AUDIO_VORBIS.equals(mimeType); + } + } + return false; + } + + @Override + public int addTrack(Format format) { + String sampleMimeType = checkNotNull(format.sampleMimeType); + MediaFormat mediaFormat; + if (MimeTypes.isAudio(sampleMimeType)) { + mediaFormat = + MediaFormat.createAudioFormat( + castNonNull(sampleMimeType), format.sampleRate, format.channelCount); + } else { + mediaFormat = + MediaFormat.createVideoFormat(castNonNull(sampleMimeType), format.width, format.height); + mediaMuxer.setOrientationHint(format.rotationDegrees); + } + MediaFormatUtil.setCsdBuffers(mediaFormat, format.initializationData); + return mediaMuxer.addTrack(mediaFormat); + } + + @Override + public void writeSampleData( + int trackIndex, ByteBuffer data, boolean isKeyFrame, long presentationTimeUs) { + if (!isStarted) { + isStarted = true; + mediaMuxer.start(); + } + int offset = data.position(); + int size = data.limit() - offset; + int flags = isKeyFrame ? C.BUFFER_FLAG_KEY_FRAME : 0; + bufferInfo.set(offset, size, presentationTimeUs, flags); + mediaMuxer.writeSampleData(trackIndex, data, bufferInfo); + } + + @Override + public void release(boolean forCancellation) { + if (!isStarted) { + mediaMuxer.release(); + return; + } + + isStarted = false; + try { + mediaMuxer.stop(); + } catch (IllegalStateException e) { + if (SDK_INT < 30) { + // Set the muxer state to stopped even if mediaMuxer.stop() failed so that + // mediaMuxer.release() doesn't attempt to stop the muxer and therefore doesn't throw the + // same exception without releasing its resources. This is already implemented in MediaMuxer + // from API level 30. + try { + Field muxerStoppedStateField = MediaMuxer.class.getDeclaredField("MUXER_STATE_STOPPED"); + muxerStoppedStateField.setAccessible(true); + int muxerStoppedState = castNonNull((Integer) muxerStoppedStateField.get(mediaMuxer)); + Field muxerStateField = MediaMuxer.class.getDeclaredField("mState"); + muxerStateField.setAccessible(true); + muxerStateField.set(mediaMuxer, muxerStoppedState); + } catch (Exception reflectionException) { + // Do nothing. + } + } + // It doesn't matter that stopping the muxer throws if the transformation is being cancelled. + if (!forCancellation) { + throw e; + } + } finally { + mediaMuxer.release(); + } + } + + /** + * Converts a {@link MimeTypes MIME type} into a {@link MediaMuxer.OutputFormat MediaMuxer output + * format}. + * + * @param mimeType The {@link MimeTypes MIME type} to convert. + * @return The corresponding {@link MediaMuxer.OutputFormat MediaMuxer output format}. + * @throws IllegalArgumentException If the {@link MimeTypes MIME type} is not supported as output + * format. + */ + private static int mimeTypeToMuxerOutputFormat(String mimeType) { + if (mimeType.equals(MimeTypes.VIDEO_MP4)) { + return MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4; + } else if (SDK_INT >= 21 && mimeType.equals(MimeTypes.VIDEO_WEBM)) { + return MediaMuxer.OutputFormat.MUXER_OUTPUT_WEBM; + } else { + throw new IllegalArgumentException("Unsupported output MIME type: " + mimeType); + } + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/MediaCodecAdapterWrapper.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/MediaCodecAdapterWrapper.java new file mode 100644 index 00000000000..bf8f7f3aae8 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/MediaCodecAdapterWrapper.java @@ -0,0 +1,301 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; + +import android.media.MediaCodec; +import android.media.MediaCodec.BufferInfo; +import android.media.MediaFormat; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.mediacodec.MediaCodecAdapter; +import com.google.android.exoplayer2.mediacodec.MediaFormatUtil; +import com.google.android.exoplayer2.mediacodec.SynchronousMediaCodecAdapter; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.common.collect.ImmutableList; +import java.io.IOException; +import java.nio.ByteBuffer; +import org.checkerframework.checker.nullness.qual.EnsuresNonNullIf; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; + +/** + * A wrapper around {@link MediaCodecAdapter}. + * + *

    Provides a layer of abstraction for callers that need to interact with {@link MediaCodec} + * through {@link MediaCodecAdapter}. This is done by simplifying the calls needed to queue and + * dequeue buffers, removing the need to track buffer indices and codec events. + */ +/* package */ final class MediaCodecAdapterWrapper { + + // MediaCodec decoders always output 16 bit PCM, unless configured to output PCM float. + // https://developer.android.com/reference/android/media/MediaCodec#raw-audio-buffers. + private static final int MEDIA_CODEC_PCM_ENCODING = C.ENCODING_PCM_16BIT; + + private final BufferInfo outputBufferInfo; + private final MediaCodecAdapter codec; + + private @MonotonicNonNull Format outputFormat; + @Nullable private ByteBuffer outputBuffer; + + private int inputBufferIndex; + private int outputBufferIndex; + private boolean inputStreamEnded; + private boolean outputStreamEnded; + + /** + * Returns a {@link MediaCodecAdapterWrapper} for a configured and started {@link + * MediaCodecAdapter} audio decoder. + * + * @param format The {@link Format} (of the input data) used to determine the underlying {@link + * MediaCodec} and its configuration values. + * @return A configured and started decoder wrapper. + * @throws IOException If the underlying codec cannot be created. + */ + public static MediaCodecAdapterWrapper createForAudioDecoding(Format format) throws IOException { + @Nullable MediaCodec decoder = null; + @Nullable MediaCodecAdapter adapter = null; + try { + decoder = MediaCodec.createDecoderByType(checkNotNull(format.sampleMimeType)); + MediaFormat mediaFormat = + MediaFormat.createAudioFormat( + format.sampleMimeType, format.sampleRate, format.channelCount); + MediaFormatUtil.maybeSetInteger( + mediaFormat, MediaFormat.KEY_MAX_INPUT_SIZE, format.maxInputSize); + MediaFormatUtil.setCsdBuffers(mediaFormat, format.initializationData); + adapter = new SynchronousMediaCodecAdapter.Factory().createAdapter(decoder); + adapter.configure(mediaFormat, /* surface= */ null, /* crypto= */ null, /* flags= */ 0); + adapter.start(); + return new MediaCodecAdapterWrapper(adapter); + } catch (Exception e) { + if (adapter != null) { + adapter.release(); + } else if (decoder != null) { + decoder.release(); + } + throw e; + } + } + + /** + * Returns a {@link MediaCodecAdapterWrapper} for a configured and started {@link + * MediaCodecAdapter} audio encoder. + * + * @param format The {@link Format} (of the output data) used to determine the underlying {@link + * MediaCodec} and its configuration values. + * @return A configured and started encoder wrapper. + * @throws IOException If the underlying codec cannot be created. + */ + public static MediaCodecAdapterWrapper createForAudioEncoding(Format format) throws IOException { + @Nullable MediaCodec encoder = null; + @Nullable MediaCodecAdapter adapter = null; + try { + encoder = MediaCodec.createEncoderByType(checkNotNull(format.sampleMimeType)); + MediaFormat mediaFormat = + MediaFormat.createAudioFormat( + format.sampleMimeType, format.sampleRate, format.channelCount); + mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, format.bitrate); + adapter = new SynchronousMediaCodecAdapter.Factory().createAdapter(encoder); + adapter.configure( + mediaFormat, + /* surface= */ null, + /* crypto= */ null, + /* flags= */ MediaCodec.CONFIGURE_FLAG_ENCODE); + adapter.start(); + return new MediaCodecAdapterWrapper(adapter); + } catch (Exception e) { + if (adapter != null) { + adapter.release(); + } else if (encoder != null) { + encoder.release(); + } + throw e; + } + } + + private MediaCodecAdapterWrapper(MediaCodecAdapter codec) { + this.codec = codec; + outputBufferInfo = new BufferInfo(); + inputBufferIndex = C.INDEX_UNSET; + outputBufferIndex = C.INDEX_UNSET; + } + + /** + * Dequeues a writable input buffer, if available. + * + * @param inputBuffer The buffer where the dequeued buffer data is stored. + * @return Whether an input buffer is ready to be used. + */ + @EnsuresNonNullIf(expression = "#1.data", result = true) + public boolean maybeDequeueInputBuffer(DecoderInputBuffer inputBuffer) { + if (inputStreamEnded) { + return false; + } + if (inputBufferIndex < 0) { + inputBufferIndex = codec.dequeueInputBufferIndex(); + if (inputBufferIndex < 0) { + return false; + } + inputBuffer.data = codec.getInputBuffer(inputBufferIndex); + inputBuffer.clear(); + } + checkNotNull(inputBuffer.data); + return true; + } + + /** + * Queues an input buffer to the decoder. No buffers may be queued after an {@link + * DecoderInputBuffer#isEndOfStream() end of stream} buffer has been queued. + */ + public void queueInputBuffer(DecoderInputBuffer inputBuffer) { + checkState( + !inputStreamEnded, "Input buffer can not be queued after the input stream has ended."); + + int offset = 0; + int size = 0; + if (inputBuffer.data != null && inputBuffer.data.hasRemaining()) { + offset = inputBuffer.data.position(); + size = inputBuffer.data.remaining(); + } + int flags = 0; + if (inputBuffer.isEndOfStream()) { + inputStreamEnded = true; + flags = MediaCodec.BUFFER_FLAG_END_OF_STREAM; + } + codec.queueInputBuffer(inputBufferIndex, offset, size, inputBuffer.timeUs, flags); + inputBufferIndex = C.INDEX_UNSET; + inputBuffer.data = null; + } + + /** Returns the current output format, if available. */ + @Nullable + public Format getOutputFormat() { + // The format is updated when dequeueing a 'special' buffer index, so attempt to dequeue now. + maybeDequeueOutputBuffer(); + return outputFormat; + } + + /** Returns the current output {@link ByteBuffer}, if available. */ + @Nullable + public ByteBuffer getOutputBuffer() { + return maybeDequeueOutputBuffer() ? outputBuffer : null; + } + + /** Returns the {@link BufferInfo} associated with the current output buffer, if available. */ + @Nullable + public BufferInfo getOutputBufferInfo() { + return maybeDequeueOutputBuffer() ? outputBufferInfo : null; + } + + /** + * Releases the current output buffer. + * + *

    This should be called after the buffer has been processed. The next output buffer will not + * be available until the previous has been released. + */ + public void releaseOutputBuffer() { + outputBuffer = null; + codec.releaseOutputBuffer(outputBufferIndex, /* render= */ false); + outputBufferIndex = C.INDEX_UNSET; + } + + /** Returns whether the codec output stream has ended, and no more data can be dequeued. */ + public boolean isEnded() { + return outputStreamEnded && outputBufferIndex == C.INDEX_UNSET; + } + + /** Releases the underlying codec. */ + public void release() { + outputBuffer = null; + codec.release(); + } + + /** + * Returns true if there is already an output buffer pending. Otherwise attempts to dequeue an + * output buffer and returns whether there is a new output buffer. + */ + private boolean maybeDequeueOutputBuffer() { + if (outputBufferIndex >= 0) { + return true; + } + if (outputStreamEnded) { + return false; + } + + outputBufferIndex = codec.dequeueOutputBufferIndex(outputBufferInfo); + if (outputBufferIndex < 0) { + if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { + outputFormat = getFormat(codec.getOutputFormat()); + } + return false; + } + if ((outputBufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) { + outputStreamEnded = true; + if (outputBufferInfo.size == 0) { + releaseOutputBuffer(); + return false; + } + } + if ((outputBufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) != 0) { + // Encountered a CSD buffer, skip it. + releaseOutputBuffer(); + return false; + } + + outputBuffer = checkNotNull(codec.getOutputBuffer(outputBufferIndex)); + outputBuffer.position(outputBufferInfo.offset); + outputBuffer.limit(outputBufferInfo.offset + outputBufferInfo.size); + + return true; + } + + private static Format getFormat(MediaFormat mediaFormat) { + ImmutableList.Builder csdBuffers = new ImmutableList.Builder<>(); + int csdIndex = 0; + while (true) { + @Nullable ByteBuffer csdByteBuffer = mediaFormat.getByteBuffer("csd-" + csdIndex); + if (csdByteBuffer == null) { + break; + } + byte[] csdBufferData = new byte[csdByteBuffer.remaining()]; + csdByteBuffer.get(csdBufferData); + csdBuffers.add(csdBufferData); + csdIndex++; + } + String mimeType = mediaFormat.getString(MediaFormat.KEY_MIME); + Format.Builder formatBuilder = + new Format.Builder() + .setSampleMimeType(mediaFormat.getString(MediaFormat.KEY_MIME)) + .setInitializationData(csdBuffers.build()); + if (MimeTypes.isVideo(mimeType)) { + formatBuilder + .setWidth(mediaFormat.getInteger(MediaFormat.KEY_WIDTH)) + .setHeight(mediaFormat.getInteger(MediaFormat.KEY_HEIGHT)); + } else if (MimeTypes.isAudio(mimeType)) { + // TODO(internal b/178685617): Only set the PCM encoding for audio/raw, once we have a way to + // simulate more realistic codec input/output formats in tests. + formatBuilder + .setChannelCount(mediaFormat.getInteger(MediaFormat.KEY_CHANNEL_COUNT)) + .setSampleRate(mediaFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE)) + .setPcmEncoding(MEDIA_CODEC_PCM_ENCODING); + } + return formatBuilder.build(); + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Muxer.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Muxer.java new file mode 100644 index 00000000000..72e5f0f6b83 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Muxer.java @@ -0,0 +1,94 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformer; + +import android.os.ParcelFileDescriptor; +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.util.MimeTypes; +import java.io.IOException; +import java.nio.ByteBuffer; + +/** + * Abstracts media muxing operations. + * + *

    Query whether {@link #supportsSampleMimeType(String) sample MIME types are supported} and + * {@link #addTrack(Format) add all tracks}, then {@link #writeSampleData(int, ByteBuffer, boolean, + * long) write sample data} to mux samples. Once any sample data has been written, it is not + * possible to add tracks. After writing all sample data, {@link #release() release} the instance to + * finish writing to the output and return any resources to the system. + */ +/* package */ interface Muxer { + + /** Factory for muxers. */ + interface Factory { + /** + * Returns a new muxer writing to a file. + * + * @param path The path to the output file. + * @param outputMimeType The container {@link MimeTypes MIME type} of the output file. + * @throws IllegalArgumentException If the path is invalid or the MIME type is not supported. + * @throws IOException If an error occurs opening the output file for writing. + */ + Muxer create(String path, String outputMimeType) throws IOException; + + /** + * Returns a new muxer writing to a file descriptor. + * + * @param parcelFileDescriptor A readable and writable {@link ParcelFileDescriptor} of the + * output. The file referenced by this ParcelFileDescriptor should not be used before the + * muxer is released. It is the responsibility of the caller to close the + * ParcelFileDescriptor. This can be done after this method returns. + * @param outputMimeType The {@link MimeTypes MIME type} of the output. + * @throws IllegalArgumentException If the file descriptor is invalid or the MIME type is not + * supported. + * @throws IOException If an error occurs opening the output file descriptor for writing. + */ + Muxer create(ParcelFileDescriptor parcelFileDescriptor, String outputMimeType) + throws IOException; + + /** Returns whether the {@link MimeTypes MIME type} provided is a supported output format. */ + boolean supportsOutputMimeType(String mimeType); + } + + /** Returns whether the sample {@link MimeTypes MIME type} is supported. */ + boolean supportsSampleMimeType(@Nullable String mimeType); + + /** + * Adds a track with the specified format, and returns its index (to be passed in subsequent calls + * to {@link #writeSampleData(int, ByteBuffer, boolean, long)}). + */ + int addTrack(Format format); + + /** + * Writes the specified sample. + * + * @param trackIndex The index of the track, previously returned by {@link #addTrack(Format)}. + * @param data Buffer containing the sample data to write to the container. + * @param isKeyFrame Whether the sample is a key frame. + * @param presentationTimeUs The presentation time of the sample in microseconds. + */ + void writeSampleData( + int trackIndex, ByteBuffer data, boolean isKeyFrame, long presentationTimeUs); + + /** + * Releases any resources associated with muxing. + * + * @param forCancellation Whether the reason for releasing the resources is the transformation + * cancellation. + */ + void release(boolean forCancellation); +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/MuxerWrapper.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/MuxerWrapper.java new file mode 100644 index 00000000000..2e9710dc15c --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/MuxerWrapper.java @@ -0,0 +1,205 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static com.google.android.exoplayer2.util.Util.minValue; + +import android.util.SparseIntArray; +import android.util.SparseLongArray; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.util.MimeTypes; +import java.nio.ByteBuffer; + +/** + * A wrapper around a media muxer. + * + *

    This wrapper can contain at most one video track and one audio track. + */ +@RequiresApi(18) +/* package */ final class MuxerWrapper { + + /** + * The maximum difference between the track positions, in microseconds. + * + *

    The value of this constant has been chosen based on the interleaving observed in a few media + * files, where continuous chunks of the same track were about 0.5 seconds long. + */ + private static final long MAX_TRACK_WRITE_AHEAD_US = C.msToUs(500); + + private final Muxer muxer; + private final SparseIntArray trackTypeToIndex; + private final SparseLongArray trackTypeToTimeUs; + + private int trackCount; + private int trackFormatCount; + private boolean isReady; + private int previousTrackType; + private long minTrackTimeUs; + + public MuxerWrapper(Muxer muxer) { + this.muxer = muxer; + trackTypeToIndex = new SparseIntArray(); + trackTypeToTimeUs = new SparseLongArray(); + previousTrackType = C.TRACK_TYPE_NONE; + } + + /** + * Registers an output track. + * + *

    All tracks must be registered before any track format is {@link #addTrackFormat(Format) + * added}. + * + * @throws IllegalStateException If a track format was {@link #addTrackFormat(Format) added} + * before calling this method. + */ + public void registerTrack() { + checkState( + trackFormatCount == 0, "Tracks cannot be registered after track formats have been added."); + trackCount++; + } + + /** Returns whether the sample {@link MimeTypes MIME type} is supported. */ + public boolean supportsSampleMimeType(@Nullable String mimeType) { + return muxer.supportsSampleMimeType(mimeType); + } + + /** + * Adds a track format to the muxer. + * + *

    The tracks must all be {@link #registerTrack() registered} before any format is added and + * all the formats must be added before samples are {@link #writeSample(int, ByteBuffer, boolean, + * long) written}. + * + * @param format The {@link Format} to be added. + * @throws IllegalStateException If the format is unsupported or if there is already a track + * format of the same type (audio or video). + */ + public void addTrackFormat(Format format) { + checkState(trackCount > 0, "All tracks should be registered before the formats are added."); + checkState(trackFormatCount < trackCount, "All track formats have already been added."); + @Nullable String sampleMimeType = format.sampleMimeType; + boolean isAudio = MimeTypes.isAudio(sampleMimeType); + boolean isVideo = MimeTypes.isVideo(sampleMimeType); + checkState(isAudio || isVideo, "Unsupported track format: " + sampleMimeType); + int trackType = MimeTypes.getTrackType(sampleMimeType); + checkState( + trackTypeToIndex.get(trackType, /* valueIfKeyNotFound= */ C.INDEX_UNSET) == C.INDEX_UNSET, + "There is already a track of type " + trackType); + + int trackIndex = muxer.addTrack(format); + trackTypeToIndex.put(trackType, trackIndex); + trackTypeToTimeUs.put(trackType, 0L); + trackFormatCount++; + if (trackFormatCount == trackCount) { + isReady = true; + } + } + + /** + * Attempts to write a sample to the muxer. + * + * @param trackType The track type of the sample, defined by the {@code TRACK_TYPE_*} constants in + * {@link C}. + * @param data The sample to write, or {@code null} if the sample is empty. + * @param isKeyFrame Whether the sample is a key frame. + * @param presentationTimeUs The presentation time of the sample in microseconds. + * @return Whether the sample was successfully written. This is {@code false} if the muxer hasn't + * {@link #addTrackFormat(Format) received a format} for every {@link #registerTrack() + * registered track}, or if it should write samples of other track types first to ensure a + * good interleaving. + * @throws IllegalStateException If the muxer doesn't have any {@link #endTrack(int) non-ended} + * track of the given track type. + */ + public boolean writeSample( + int trackType, @Nullable ByteBuffer data, boolean isKeyFrame, long presentationTimeUs) { + int trackIndex = trackTypeToIndex.get(trackType, /* valueIfKeyNotFound= */ C.INDEX_UNSET); + checkState( + trackIndex != C.INDEX_UNSET, + "Could not write sample because there is no track of type " + trackType); + + if (!canWriteSampleOfType(trackType)) { + return false; + } else if (data == null) { + return true; + } + + muxer.writeSampleData(trackIndex, data, isKeyFrame, presentationTimeUs); + trackTypeToTimeUs.put(trackType, presentationTimeUs); + previousTrackType = trackType; + return true; + } + + /** + * Notifies the muxer that all the samples have been {@link #writeSample(int, ByteBuffer, boolean, + * long) written} for a given track. + * + * @param trackType The track type, defined by the {@code TRACK_TYPE_*} constants in {@link C}. + */ + public void endTrack(int trackType) { + trackTypeToIndex.delete(trackType); + trackTypeToTimeUs.delete(trackType); + } + + /** + * Releases any resources associated with muxing. + * + *

    The muxer cannot be used anymore once this method has been called. + * + * @param forCancellation Whether the reason for releasing the resources is the transformation + * cancellation. + */ + public void release(boolean forCancellation) { + isReady = false; + muxer.release(forCancellation); + } + + /** Returns the number of {@link #registerTrack() registered} tracks. */ + public int getTrackCount() { + return trackCount; + } + + /** + * Returns whether the muxer can write a sample of the given track type. + * + * @param trackType The track type, defined by the {@code TRACK_TYPE_*} constants in {@link C}. + * @return Whether the muxer can write a sample of the given track type. This is {@code false} if + * the muxer hasn't {@link #addTrackFormat(Format) received a format} for every {@link + * #registerTrack() registered track}, or if it should write samples of other track types + * first to ensure a good interleaving. + * @throws IllegalStateException If the muxer doesn't have any {@link #endTrack(int) non-ended} + * track of the given track type. + */ + private boolean canWriteSampleOfType(int trackType) { + long trackTimeUs = trackTypeToTimeUs.get(trackType, /* valueIfKeyNotFound= */ C.TIME_UNSET); + checkState(trackTimeUs != C.TIME_UNSET); + if (!isReady) { + return false; + } + if (trackTypeToTimeUs.size() == 1) { + return true; + } + if (trackType != previousTrackType) { + minTrackTimeUs = minValue(trackTypeToTimeUs); + } + return trackTimeUs - minTrackTimeUs <= MAX_TRACK_WRITE_AHEAD_US; + } + +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/ProgressHolder.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/ProgressHolder.java new file mode 100644 index 00000000000..0f34aed8217 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/ProgressHolder.java @@ -0,0 +1,27 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import androidx.annotation.IntRange; + +/** Holds a progress percentage. */ +public final class ProgressHolder { + + /** The held progress, expressed as an integer percentage. */ + @IntRange(from = 0, to = 100) + public int progress; +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SampleTransformer.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SampleTransformer.java new file mode 100644 index 00000000000..266034c905f --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SampleTransformer.java @@ -0,0 +1,31 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import com.google.android.exoplayer2.decoder.DecoderInputBuffer; + +/** A sample transformer for a given track. */ +/* package */ interface SampleTransformer { + + /** + * Transforms the data and metadata of the sample contained in {@code buffer}. + * + * @param buffer The sample to transform. If the sample {@link DecoderInputBuffer#data data} is + * {@code null} after the execution of this method, the sample must be discarded. + */ + void transformSample(DecoderInputBuffer buffer); +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SefSlowMotionVideoSampleTransformer.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SefSlowMotionVideoSampleTransformer.java new file mode 100644 index 00000000000..a232d82a52a --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SefSlowMotionVideoSampleTransformer.java @@ -0,0 +1,397 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.util.Assertions.checkArgument; +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static com.google.android.exoplayer2.util.NalUnitUtil.NAL_START_CODE; +import static com.google.android.exoplayer2.util.Util.castNonNull; +import static java.lang.Math.min; + +import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData; +import com.google.android.exoplayer2.metadata.mp4.SmtaMetadataEntry; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.common.collect.ImmutableList; +import java.nio.ByteBuffer; +import java.util.Arrays; +import java.util.Iterator; +import java.util.List; +import org.checkerframework.checker.nullness.qual.RequiresNonNull; + +/** + * {@link SampleTransformer} that flattens SEF slow motion video samples. + * + *

    Such samples follow the ITU-T Recommendation H.264 with temporal SVC. + * + *

    This transformer leaves the samples received unchanged if the input is not an SEF slow motion + * video. + * + *

    The mathematical formulas used in this class are explained in [Internal ref: + * http://go/exoplayer-sef-slomo-video-flattening]. + */ +/* package */ final class SefSlowMotionVideoSampleTransformer implements SampleTransformer { + + /** + * The frame rate of SEF slow motion videos, in fps. + * + *

    This frame rate is constant and is not equal to the capture frame rate. It is set to a lower + * value so that the video is entirely played in slow motion on players that do not support SEF + * slow motion. + */ + @VisibleForTesting /* package */ static final int INPUT_FRAME_RATE = 30; + + /** + * The target frame rate of the flattened output, in fps. + * + *

    The output frame rate might be slightly different and might not be constant. + */ + private static final int TARGET_OUTPUT_FRAME_RATE = 30; + + private static final int NAL_START_CODE_LENGTH = NAL_START_CODE.length; + /** + * The nal_unit_type corresponding to a prefix NAL unit (see ITU-T Recommendation H.264 (2016) + * table 7-1). + */ + private static final int NAL_UNIT_TYPE_PREFIX = 0x0E; + + private final byte[] scratch; + /** The SEF slow motion configuration of the input. */ + @Nullable private final SlowMotionData slowMotionData; + /** + * An iterator iterating over the slow motion segments, pointing at the segment following {@code + * nextSegmentInfo}, if any. + */ + private final Iterator segmentIterator; + /** The frame rate at which the input has been captured, in fps. */ + private final float captureFrameRate; + /** The maximum SVC temporal layer present in the input. */ + private final int inputMaxLayer; + /** + * The maximum SVC temporal layer value of the frames that should be kept in the input (or a part + * of it) so that it is played at normal speed. + */ + private final int normalSpeedMaxLayer; + + /** + * The {@link SegmentInfo} describing the current slow motion segment, or null if the current + * frame is not in such a segment. + */ + @Nullable private SegmentInfo currentSegmentInfo; + /** + * The {@link SegmentInfo} describing the slow motion segment following (not including) the + * current frame, or null if there is no such segment. + */ + @Nullable private SegmentInfo nextSegmentInfo; + /** + * The time delta to be added to the output timestamps before scaling to take the slow motion + * segments into account, in microseconds. + */ + private long frameTimeDeltaUs; + + public SefSlowMotionVideoSampleTransformer(Format format) { + scratch = new byte[NAL_START_CODE_LENGTH]; + MetadataInfo metadataInfo = getMetadataInfo(format.metadata); + slowMotionData = metadataInfo.slowMotionData; + List segments = + slowMotionData != null ? slowMotionData.segments : ImmutableList.of(); + segmentIterator = segments.iterator(); + captureFrameRate = metadataInfo.captureFrameRate; + inputMaxLayer = metadataInfo.inputMaxLayer; + normalSpeedMaxLayer = metadataInfo.normalSpeedMaxLayer; + nextSegmentInfo = + segmentIterator.hasNext() + ? new SegmentInfo(segmentIterator.next(), inputMaxLayer, normalSpeedMaxLayer) + : null; + if (slowMotionData != null) { + checkArgument( + MimeTypes.VIDEO_H264.equals(format.sampleMimeType), + "Unsupported MIME type for SEF slow motion video track: " + format.sampleMimeType); + } + } + + @Override + public void transformSample(DecoderInputBuffer buffer) { + if (slowMotionData == null) { + // The input is not an SEF slow motion video. + return; + } + + ByteBuffer data = castNonNull(buffer.data); + int originalPosition = data.position(); + data.position(originalPosition + NAL_START_CODE_LENGTH); + data.get(scratch, 0, 4); // Read nal_unit_header_svc_extension. + int nalUnitType = scratch[0] & 0x1F; + boolean svcExtensionFlag = ((scratch[1] & 0xFF) >> 7) == 1; + checkState( + nalUnitType == NAL_UNIT_TYPE_PREFIX && svcExtensionFlag, + "Missing SVC extension prefix NAL unit."); + int layer = (scratch[3] & 0xFF) >> 5; + boolean shouldKeepFrame = processCurrentFrame(layer, buffer.timeUs); + if (shouldKeepFrame) { + buffer.timeUs = getCurrentFrameOutputTimeUs(/* inputTimeUs= */ buffer.timeUs); + skipToNextNalUnit(data); // Skip over prefix_nal_unit_svc. + } else { + buffer.data = null; + } + } + + /** + * Processes the current frame and returns whether it should be kept. + * + * @param layer The frame temporal SVC layer. + * @param timeUs The frame presentation time, in microseconds. + * @return Whether to keep the current frame. + */ + @VisibleForTesting + /* package */ boolean processCurrentFrame(int layer, long timeUs) { + // Skip segments in the unlikely case that they do not contain any frame start time. + while (nextSegmentInfo != null && timeUs >= nextSegmentInfo.endTimeUs) { + enterNextSegment(); + } + + if (nextSegmentInfo != null && timeUs >= nextSegmentInfo.startTimeUs) { + enterNextSegment(); + } else if (currentSegmentInfo != null && timeUs >= currentSegmentInfo.endTimeUs) { + leaveCurrentSegment(); + } + + int maxLayer = currentSegmentInfo != null ? currentSegmentInfo.maxLayer : normalSpeedMaxLayer; + return layer <= maxLayer || shouldKeepFrameForOutputValidity(layer, timeUs); + } + + /** Updates the segments information so that the next segment becomes the current segment. */ + private void enterNextSegment() { + if (currentSegmentInfo != null) { + leaveCurrentSegment(); + } + currentSegmentInfo = nextSegmentInfo; + nextSegmentInfo = + segmentIterator.hasNext() + ? new SegmentInfo(segmentIterator.next(), inputMaxLayer, normalSpeedMaxLayer) + : null; + } + + /** + * Updates the segments information so that there is no current segment. The next segment is + * unchanged. + */ + @RequiresNonNull("currentSegmentInfo") + private void leaveCurrentSegment() { + frameTimeDeltaUs += + (currentSegmentInfo.endTimeUs - currentSegmentInfo.startTimeUs) + * (currentSegmentInfo.speedDivisor - 1); + currentSegmentInfo = null; + } + + /** + * Returns whether the frames of the next segment are based on the current frame. In this case, + * the current frame should be kept in order for the output to be valid. + * + * @param layer The frame temporal SVC layer. + * @param timeUs The frame presentation time, in microseconds. + * @return Whether to keep the current frame. + */ + private boolean shouldKeepFrameForOutputValidity(int layer, long timeUs) { + if (nextSegmentInfo == null || layer >= nextSegmentInfo.maxLayer) { + return false; + } + + long frameOffsetToSegmentEstimate = + (nextSegmentInfo.startTimeUs - timeUs) * INPUT_FRAME_RATE / C.MICROS_PER_SECOND; + float allowedError = 0.45f; + float baseMaxFrameOffsetToSegment = + -(1 << (inputMaxLayer - nextSegmentInfo.maxLayer)) + allowedError; + for (int i = 1; i < nextSegmentInfo.maxLayer; i++) { + if (frameOffsetToSegmentEstimate < (1 << (inputMaxLayer - i)) + baseMaxFrameOffsetToSegment) { + if (layer <= i) { + return true; + } + } else { + return false; + } + } + return false; + } + + /** + * Returns the time of the current frame in the output, in microseconds. + * + *

    This time is computed so that segments start and end at the correct times. As a result, the + * output frame rate might be variable. + * + *

    This method can only be called if all the frames until the current one (included) have been + * {@link #processCurrentFrame(int, long) processed} in order, and if the next frames have not + * been processed yet. + */ + @VisibleForTesting + /* package */ long getCurrentFrameOutputTimeUs(long inputTimeUs) { + long outputTimeUs = inputTimeUs + frameTimeDeltaUs; + if (currentSegmentInfo != null) { + outputTimeUs += + (inputTimeUs - currentSegmentInfo.startTimeUs) * (currentSegmentInfo.speedDivisor - 1); + } + return Math.round(outputTimeUs * INPUT_FRAME_RATE / captureFrameRate); + } + + /** + * Advances the position of {@code data} to the start of the next NAL unit. + * + * @throws IllegalStateException If no NAL unit is found. + */ + private void skipToNextNalUnit(ByteBuffer data) { + int newPosition = data.position(); + while (data.remaining() >= NAL_START_CODE_LENGTH) { + data.get(scratch, 0, NAL_START_CODE_LENGTH); + if (Arrays.equals(scratch, NAL_START_CODE)) { + data.position(newPosition); + return; + } + newPosition++; + data.position(newPosition); + } + throw new IllegalStateException("Could not find NAL unit start code."); + } + + /** Returns the {@link MetadataInfo} derived from the {@link Metadata} provided. */ + private static MetadataInfo getMetadataInfo(@Nullable Metadata metadata) { + MetadataInfo metadataInfo = new MetadataInfo(); + if (metadata == null) { + return metadataInfo; + } + + for (int i = 0; i < metadata.length(); i++) { + Metadata.Entry entry = metadata.get(i); + if (entry instanceof SmtaMetadataEntry) { + SmtaMetadataEntry smtaMetadataEntry = (SmtaMetadataEntry) entry; + metadataInfo.captureFrameRate = smtaMetadataEntry.captureFrameRate; + metadataInfo.inputMaxLayer = smtaMetadataEntry.svcTemporalLayerCount - 1; + } else if (entry instanceof SlowMotionData) { + metadataInfo.slowMotionData = (SlowMotionData) entry; + } + } + + if (metadataInfo.slowMotionData == null) { + return metadataInfo; + } + + checkState(metadataInfo.inputMaxLayer != C.INDEX_UNSET, "SVC temporal layer count not found."); + checkState(metadataInfo.captureFrameRate != C.RATE_UNSET, "Capture frame rate not found."); + checkState( + metadataInfo.captureFrameRate % 1 == 0 + && metadataInfo.captureFrameRate % TARGET_OUTPUT_FRAME_RATE == 0, + "Invalid capture frame rate: " + metadataInfo.captureFrameRate); + + int frameCountDivisor = (int) metadataInfo.captureFrameRate / TARGET_OUTPUT_FRAME_RATE; + int normalSpeedMaxLayer = metadataInfo.inputMaxLayer; + while (normalSpeedMaxLayer >= 0) { + if ((frameCountDivisor & 1) == 1) { + // Set normalSpeedMaxLayer only if captureFrameRate / TARGET_OUTPUT_FRAME_RATE is a power of + // 2. Otherwise, the target output frame rate cannot be reached because removing a layer + // divides the number of frames by 2. + checkState( + frameCountDivisor >> 1 == 0, + "Could not compute normal speed max SVC layer for capture frame rate " + + metadataInfo.captureFrameRate); + metadataInfo.normalSpeedMaxLayer = normalSpeedMaxLayer; + break; + } + frameCountDivisor >>= 1; + normalSpeedMaxLayer--; + } + return metadataInfo; + } + + /** Metadata of an SEF slow motion input. */ + private static final class MetadataInfo { + /** + * The frame rate at which the slow motion video has been captured in fps, or {@link + * C#RATE_UNSET} if it is unknown or invalid. + */ + public float captureFrameRate; + /** + * The maximum SVC layer value of the input frames, or {@link C#INDEX_UNSET} if it is unknown. + */ + public int inputMaxLayer; + /** + * The maximum SVC layer value of the frames to keep in order to play the video at normal speed + * at {@link #TARGET_OUTPUT_FRAME_RATE}, or {@link C#INDEX_UNSET} if it is unknown. + */ + public int normalSpeedMaxLayer; + /** The input {@link SlowMotionData}. */ + @Nullable public SlowMotionData slowMotionData; + + public MetadataInfo() { + captureFrameRate = C.RATE_UNSET; + inputMaxLayer = C.INDEX_UNSET; + normalSpeedMaxLayer = C.INDEX_UNSET; + } + } + + /** Information about a slow motion segment. */ + private static final class SegmentInfo { + /** The segment start time, in microseconds. */ + public final long startTimeUs; + /** The segment end time, in microseconds. */ + public final long endTimeUs; + /** + * The segment speedDivisor. + * + * @see SlowMotionData.Segment#speedDivisor + */ + public final int speedDivisor; + /** + * The maximum SVC layer value of the frames to keep in the segment in order to slow down the + * segment by {@code speedDivisor}. + */ + public final int maxLayer; + + public SegmentInfo(SlowMotionData.Segment segment, int inputMaxLayer, int normalSpeedLayer) { + this.startTimeUs = C.msToUs(segment.startTimeMs); + this.endTimeUs = C.msToUs(segment.endTimeMs); + this.speedDivisor = segment.speedDivisor; + this.maxLayer = getSlowMotionMaxLayer(speedDivisor, inputMaxLayer, normalSpeedLayer); + } + + private static int getSlowMotionMaxLayer( + int speedDivisor, int inputMaxLayer, int normalSpeedMaxLayer) { + int maxLayer = normalSpeedMaxLayer; + // Increase the maximum layer to increase the number of frames in the segment. For every layer + // increment, the number of frames is doubled. + int shiftedSpeedDivisor = speedDivisor; + while (shiftedSpeedDivisor > 0) { + if ((shiftedSpeedDivisor & 1) == 1) { + checkState(shiftedSpeedDivisor >> 1 == 0, "Invalid speed divisor: " + speedDivisor); + break; + } + maxLayer++; + shiftedSpeedDivisor >>= 1; + } + + // The optimal segment max layer can be larger than the input max layer. In this case, it is + // not possible to have speedDivisor times more frames in the segment than outside the + // segments. The desired speed must therefore be reached by keeping all the frames and by + // decreasing the frame rate in the segment. + return min(maxLayer, inputMaxLayer); + } + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SegmentSpeedProvider.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SegmentSpeedProvider.java new file mode 100644 index 00000000000..2320367076c --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SegmentSpeedProvider.java @@ -0,0 +1,119 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.metadata.mp4.SlowMotionData.Segment.BY_START_THEN_END_THEN_DIVISOR; +import static com.google.android.exoplayer2.util.Assertions.checkArgument; + +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData.Segment; +import com.google.android.exoplayer2.metadata.mp4.SmtaMetadataEntry; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSortedMap; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.TreeMap; + +/** A {@link SpeedProvider} for slow motion segments. */ +/* package */ class SegmentSpeedProvider implements SpeedProvider { + + /** + * Input frame rate of Samsung Slow motion videos is always 30. See + * go/exoplayer-sef-slomo-video-flattening. + */ + private static final int INPUT_FRAME_RATE = 30; + + private final ImmutableSortedMap speedsByStartTimeUs; + private final float baseSpeedMultiplier; + + public SegmentSpeedProvider(Format format) { + float captureFrameRate = getCaptureFrameRate(format); + this.baseSpeedMultiplier = + captureFrameRate == C.RATE_UNSET ? 1 : captureFrameRate / INPUT_FRAME_RATE; + this.speedsByStartTimeUs = buildSpeedByStartTimeUsMap(format, baseSpeedMultiplier); + } + + @Override + public float getSpeed(long timeUs) { + checkArgument(timeUs >= 0); + @Nullable Map.Entry entry = speedsByStartTimeUs.floorEntry(timeUs); + return entry != null ? entry.getValue() : baseSpeedMultiplier; + } + + private static ImmutableSortedMap buildSpeedByStartTimeUsMap( + Format format, float baseSpeed) { + List segments = extractSlowMotionSegments(format); + + if (segments.isEmpty()) { + return ImmutableSortedMap.of(); + } + + TreeMap speedsByStartTimeUs = new TreeMap<>(); + + // Start time maps to the segment speed. + for (int i = 0; i < segments.size(); i++) { + Segment currentSegment = segments.get(i); + speedsByStartTimeUs.put( + C.msToUs(currentSegment.startTimeMs), baseSpeed / currentSegment.speedDivisor); + } + + // If the map has an entry at endTime, this is the next segments start time. If no such entry + // exists, map the endTime to base speed because the times after the end time are not in a + // segment. + for (int i = 0; i < segments.size(); i++) { + Segment currentSegment = segments.get(i); + if (!speedsByStartTimeUs.containsKey(C.msToUs(currentSegment.endTimeMs))) { + speedsByStartTimeUs.put(C.msToUs(currentSegment.endTimeMs), baseSpeed); + } + } + + return ImmutableSortedMap.copyOf(speedsByStartTimeUs); + } + + private static float getCaptureFrameRate(Format format) { + @Nullable Metadata metadata = format.metadata; + if (metadata == null) { + return C.RATE_UNSET; + } + for (int i = 0; i < metadata.length(); i++) { + Metadata.Entry entry = metadata.get(i); + if (entry instanceof SmtaMetadataEntry) { + return ((SmtaMetadataEntry) entry).captureFrameRate; + } + } + + return C.RATE_UNSET; + } + + private static ImmutableList extractSlowMotionSegments(Format format) { + List segments = new ArrayList<>(); + @Nullable Metadata metadata = format.metadata; + if (metadata != null) { + for (int i = 0; i < metadata.length(); i++) { + Metadata.Entry entry = metadata.get(i); + if (entry instanceof SlowMotionData) { + segments.addAll(((SlowMotionData) entry).segments); + } + } + } + return ImmutableList.sortedCopyOf(BY_START_THEN_END_THEN_DIVISOR, segments); + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SpeedProvider.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SpeedProvider.java new file mode 100644 index 00000000000..f8109e031c7 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/SpeedProvider.java @@ -0,0 +1,28 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformer; + +/** A custom interface that determines the speed for media at specific timestamps. */ +public interface SpeedProvider { + + /** + * Provides the speed that the media should be played at, based on the timeUs. + * + * @param timeUs The timestamp of the media. + * @return The speed that the media should be played at, based on the timeUs. + */ + float getSpeed(long timeUs); +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Transformation.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Transformation.java new file mode 100644 index 00000000000..b0c9e8d2ccf --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Transformation.java @@ -0,0 +1,37 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +/** A media transformation configuration. */ +/* package */ final class Transformation { + + public final boolean removeAudio; + public final boolean removeVideo; + public final boolean flattenForSlowMotion; + public final String outputMimeType; + + public Transformation( + boolean removeAudio, + boolean removeVideo, + boolean flattenForSlowMotion, + String outputMimeType) { + this.removeAudio = removeAudio; + this.removeVideo = removeVideo; + this.flattenForSlowMotion = flattenForSlowMotion; + this.outputMimeType = outputMimeType; + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Transformer.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Transformer.java new file mode 100644 index 00000000000..1ca5be3570e --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/Transformer.java @@ -0,0 +1,662 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.DefaultLoadControl.DEFAULT_BUFFER_FOR_PLAYBACK_AFTER_REBUFFER_MS; +import static com.google.android.exoplayer2.DefaultLoadControl.DEFAULT_BUFFER_FOR_PLAYBACK_MS; +import static com.google.android.exoplayer2.DefaultLoadControl.DEFAULT_MAX_BUFFER_MS; +import static com.google.android.exoplayer2.DefaultLoadControl.DEFAULT_MIN_BUFFER_MS; +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static com.google.android.exoplayer2.util.Assertions.checkStateNotNull; +import static java.lang.Math.min; + +import android.content.Context; +import android.media.MediaFormat; +import android.media.MediaMuxer; +import android.os.Handler; +import android.os.Looper; +import android.os.ParcelFileDescriptor; +import androidx.annotation.IntDef; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.DefaultLoadControl; +import com.google.android.exoplayer2.ExoPlaybackException; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.Renderer; +import com.google.android.exoplayer2.RenderersFactory; +import com.google.android.exoplayer2.SimpleExoPlayer; +import com.google.android.exoplayer2.Timeline; +import com.google.android.exoplayer2.analytics.AnalyticsListener; +import com.google.android.exoplayer2.audio.AudioRendererEventListener; +import com.google.android.exoplayer2.extractor.DefaultExtractorsFactory; +import com.google.android.exoplayer2.extractor.mp4.Mp4Extractor; +import com.google.android.exoplayer2.metadata.MetadataOutput; +import com.google.android.exoplayer2.source.DefaultMediaSourceFactory; +import com.google.android.exoplayer2.source.MediaSourceFactory; +import com.google.android.exoplayer2.source.TrackGroupArray; +import com.google.android.exoplayer2.text.TextOutput; +import com.google.android.exoplayer2.trackselection.DefaultTrackSelector; +import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.util.Clock; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.android.exoplayer2.util.Util; +import com.google.android.exoplayer2.video.VideoRendererEventListener; +import java.io.IOException; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; + +/** + * A transformer to transform media inputs. + * + *

    The same Transformer instance can be used to transform multiple inputs (sequentially, not + * concurrently). + * + *

    Transformer instances must be accessed from a single application thread. For the vast majority + * of cases this should be the application's main thread. The thread on which a Transformer instance + * must be accessed can be explicitly specified by passing a {@link Looper} when creating the + * transformer. If no Looper is specified, then the Looper of the thread that the {@link + * Transformer.Builder} is created on is used, or if that thread does not have a Looper, the Looper + * of the application's main thread is used. In all cases the Looper of the thread from which the + * transformer must be accessed can be queried using {@link #getApplicationLooper()}. + */ +@RequiresApi(18) +public final class Transformer { + + /** A builder for {@link Transformer} instances. */ + public static final class Builder { + + private @MonotonicNonNull Context context; + private @MonotonicNonNull MediaSourceFactory mediaSourceFactory; + private Muxer.Factory muxerFactory; + private boolean removeAudio; + private boolean removeVideo; + private boolean flattenForSlowMotion; + private String outputMimeType; + private Transformer.Listener listener; + private Looper looper; + private Clock clock; + + /** Creates a builder with default values. */ + public Builder() { + muxerFactory = new FrameworkMuxer.Factory(); + outputMimeType = MimeTypes.VIDEO_MP4; + listener = new Listener() {}; + looper = Util.getCurrentOrMainLooper(); + clock = Clock.DEFAULT; + } + + /** Creates a builder with the values of the provided {@link Transformer}. */ + private Builder(Transformer transformer) { + this.context = transformer.context; + this.mediaSourceFactory = transformer.mediaSourceFactory; + this.muxerFactory = transformer.muxerFactory; + this.removeAudio = transformer.transformation.removeAudio; + this.removeVideo = transformer.transformation.removeVideo; + this.flattenForSlowMotion = transformer.transformation.flattenForSlowMotion; + this.outputMimeType = transformer.transformation.outputMimeType; + this.listener = transformer.listener; + this.looper = transformer.looper; + this.clock = transformer.clock; + } + + /** + * Sets the {@link Context}. + * + *

    This parameter is mandatory. + * + * @param context The {@link Context}. + * @return This builder. + */ + public Builder setContext(Context context) { + this.context = context.getApplicationContext(); + return this; + } + + /** + * Sets the {@link MediaSourceFactory} to be used to retrieve the inputs to transform. The + * default value is a {@link DefaultMediaSourceFactory} built with the context provided in + * {@link #setContext(Context)}. + * + * @param mediaSourceFactory A {@link MediaSourceFactory}. + * @return This builder. + */ + public Builder setMediaSourceFactory(MediaSourceFactory mediaSourceFactory) { + this.mediaSourceFactory = mediaSourceFactory; + return this; + } + + /** + * Sets whether to remove the audio from the output. The default value is {@code false}. + * + *

    The audio and video cannot both be removed because the output would not contain any + * samples. + * + * @param removeAudio Whether to remove the audio. + * @return This builder. + */ + public Builder setRemoveAudio(boolean removeAudio) { + this.removeAudio = removeAudio; + return this; + } + + /** + * Sets whether to remove the video from the output. The default value is {@code false}. + * + *

    The audio and video cannot both be removed because the output would not contain any + * samples. + * + * @param removeVideo Whether to remove the video. + * @return This builder. + */ + public Builder setRemoveVideo(boolean removeVideo) { + this.removeVideo = removeVideo; + return this; + } + + /** + * Sets whether the input should be flattened for media containing slow motion markers. The + * transformed output is obtained by removing the slow motion metadata and by actually slowing + * down the parts of the video and audio streams defined in this metadata. The default value for + * {@code flattenForSlowMotion} is {@code false}. + * + *

    Only Samsung Extension Format (SEF) slow motion metadata type is supported. The + * transformation has no effect if the input does not contain this metadata type. + * + *

    For SEF slow motion media, the following assumptions are made on the input: + * + *

      + *
    • The input container format is (unfragmented) MP4. + *
    • The input contains an AVC video elementary stream with temporal SVC. + *
    • The recording frame rate of the video is 120 or 240 fps. + *
    + * + *

    If specifying a {@link MediaSourceFactory} using {@link + * #setMediaSourceFactory(MediaSourceFactory)}, make sure that {@link + * Mp4Extractor#FLAG_READ_SEF_DATA} is set on the {@link Mp4Extractor} used. Otherwise, the slow + * motion metadata will be ignored and the input won't be flattened. + * + * @param flattenForSlowMotion Whether to flatten for slow motion. + * @return This builder. + */ + public Builder setFlattenForSlowMotion(boolean flattenForSlowMotion) { + this.flattenForSlowMotion = flattenForSlowMotion; + return this; + } + + /** + * Sets the MIME type of the output. The default value is {@link MimeTypes#VIDEO_MP4}. Supported + * values are: + * + *

      + *
    • {@link MimeTypes#VIDEO_MP4} + *
    • {@link MimeTypes#VIDEO_WEBM} from API level 21 + *
    + * + * @param outputMimeType The MIME type of the output. + * @return This builder. + */ + public Builder setOutputMimeType(String outputMimeType) { + this.outputMimeType = outputMimeType; + return this; + } + + /** + * Sets the {@link Transformer.Listener} to listen to the transformation events. + * + *

    This is equivalent to {@link Transformer#setListener(Listener)}. + * + * @param listener A {@link Transformer.Listener}. + * @return This builder. + */ + public Builder setListener(Transformer.Listener listener) { + this.listener = listener; + return this; + } + + /** + * Sets the {@link Looper} that must be used for all calls to the transformer and that is used + * to call listeners on. The default value is the Looper of the thread that this builder was + * created on, or if that thread does not have a Looper, the Looper of the application's main + * thread. + * + * @param looper A {@link Looper}. + * @return This builder. + */ + public Builder setLooper(Looper looper) { + this.looper = looper; + return this; + } + + /** + * Sets the {@link Clock} that will be used by the transformer. The default value is {@link + * Clock#DEFAULT}. + * + * @param clock The {@link Clock} instance. + * @return This builder. + */ + @VisibleForTesting + /* package */ Builder setClock(Clock clock) { + this.clock = clock; + return this; + } + + /** + * Sets the factory for muxers that write the media container. + * + * @param muxerFactory A {@link Muxer.Factory}. + * @return This builder. + */ + @VisibleForTesting + /* package */ Builder setMuxerFactory(Muxer.Factory muxerFactory) { + this.muxerFactory = muxerFactory; + return this; + } + + /** + * Builds a {@link Transformer} instance. + * + * @throws IllegalStateException If the {@link Context} has not been provided. + * @throws IllegalStateException If both audio and video have been removed (otherwise the output + * would not contain any samples). + * @throws IllegalStateException If the muxer doesn't support the requested output MIME type. + */ + public Transformer build() { + checkStateNotNull(context); + if (mediaSourceFactory == null) { + DefaultExtractorsFactory defaultExtractorsFactory = new DefaultExtractorsFactory(); + if (flattenForSlowMotion) { + defaultExtractorsFactory.setMp4ExtractorFlags(Mp4Extractor.FLAG_READ_SEF_DATA); + } + mediaSourceFactory = new DefaultMediaSourceFactory(context, defaultExtractorsFactory); + } + checkState( + muxerFactory.supportsOutputMimeType(outputMimeType), + "Unsupported output MIME type: " + outputMimeType); + Transformation transformation = + new Transformation(removeAudio, removeVideo, flattenForSlowMotion, outputMimeType); + return new Transformer( + context, mediaSourceFactory, muxerFactory, transformation, listener, looper, clock); + } + } + + /** A listener for the transformation events. */ + public interface Listener { + + /** + * Called when the transformation is completed. + * + * @param inputMediaItem The {@link MediaItem} for which the transformation is completed. + */ + default void onTransformationCompleted(MediaItem inputMediaItem) {} + + /** + * Called if an error occurs during the transformation. + * + * @param inputMediaItem The {@link MediaItem} for which the error occurs. + * @param exception The exception describing the error. + */ + default void onTransformationError(MediaItem inputMediaItem, Exception exception) {} + } + + /** + * Progress state. One of {@link #PROGRESS_STATE_WAITING_FOR_AVAILABILITY}, {@link + * #PROGRESS_STATE_AVAILABLE}, {@link #PROGRESS_STATE_UNAVAILABLE}, {@link + * #PROGRESS_STATE_NO_TRANSFORMATION} + */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + PROGRESS_STATE_WAITING_FOR_AVAILABILITY, + PROGRESS_STATE_AVAILABLE, + PROGRESS_STATE_UNAVAILABLE, + PROGRESS_STATE_NO_TRANSFORMATION + }) + public @interface ProgressState {} + + /** + * Indicates that the progress is unavailable for the current transformation, but might become + * available. + */ + public static final int PROGRESS_STATE_WAITING_FOR_AVAILABILITY = 0; + /** Indicates that the progress is available. */ + public static final int PROGRESS_STATE_AVAILABLE = 1; + /** Indicates that the progress is permanently unavailable for the current transformation. */ + public static final int PROGRESS_STATE_UNAVAILABLE = 2; + /** Indicates that there is no current transformation. */ + public static final int PROGRESS_STATE_NO_TRANSFORMATION = 4; + + private final Context context; + private final MediaSourceFactory mediaSourceFactory; + private final Muxer.Factory muxerFactory; + private final Transformation transformation; + private final Looper looper; + private final Clock clock; + + private Transformer.Listener listener; + @Nullable private MuxerWrapper muxerWrapper; + @Nullable private SimpleExoPlayer player; + @ProgressState private int progressState; + + private Transformer( + Context context, + MediaSourceFactory mediaSourceFactory, + Muxer.Factory muxerFactory, + Transformation transformation, + Transformer.Listener listener, + Looper looper, + Clock clock) { + checkState( + !transformation.removeAudio || !transformation.removeVideo, + "Audio and video cannot both be removed."); + this.context = context; + this.mediaSourceFactory = mediaSourceFactory; + this.muxerFactory = muxerFactory; + this.transformation = transformation; + this.listener = listener; + this.looper = looper; + this.clock = clock; + progressState = PROGRESS_STATE_NO_TRANSFORMATION; + } + + /** Returns a {@link Transformer.Builder} initialized with the values of this instance. */ + public Builder buildUpon() { + return new Builder(this); + } + + /** + * Sets the {@link Transformer.Listener} to listen to the transformation events. + * + * @param listener A {@link Transformer.Listener}. + * @throws IllegalStateException If this method is called from the wrong thread. + */ + public void setListener(Transformer.Listener listener) { + verifyApplicationThread(); + this.listener = listener; + } + + /** + * Starts an asynchronous operation to transform the given {@link MediaItem}. + * + *

    The transformation state is notified through the {@link Builder#setListener(Listener) + * listener}. + * + *

    Concurrent transformations on the same Transformer object are not allowed. + * + *

    The output can contain at most one video track and one audio track. Other track types are + * ignored. For adaptive bitrate {@link com.google.android.exoplayer2.source.MediaSource media + * sources}, the highest bitrate video and audio streams are selected. + * + * @param mediaItem The {@link MediaItem} to transform. The supported sample formats depend on the + * output container format and are described in {@link MediaMuxer#addTrack(MediaFormat)}. + * @param path The path to the output file. + * @throws IllegalArgumentException If the path is invalid. + * @throws IllegalStateException If this method is called from the wrong thread. + * @throws IllegalStateException If a transformation is already in progress. + * @throws IOException If an error occurs opening the output file for writing. + */ + public void startTransformation(MediaItem mediaItem, String path) throws IOException { + startTransformation(mediaItem, muxerFactory.create(path, transformation.outputMimeType)); + } + + /** + * Starts an asynchronous operation to transform the given {@link MediaItem}. + * + *

    The transformation state is notified through the {@link Builder#setListener(Listener) + * listener}. + * + *

    Concurrent transformations on the same Transformer object are not allowed. + * + *

    The output can contain at most one video track and one audio track. Other track types are + * ignored. For adaptive bitrate {@link com.google.android.exoplayer2.source.MediaSource media + * sources}, the highest bitrate video and audio streams are selected. + * + * @param mediaItem The {@link MediaItem} to transform. The supported sample formats depend on the + * output container format and are described in {@link MediaMuxer#addTrack(MediaFormat)}. + * @param parcelFileDescriptor A readable and writable {@link ParcelFileDescriptor} of the output. + * The file referenced by this ParcelFileDescriptor should not be used before the + * transformation is completed. It is the responsibility of the caller to close the + * ParcelFileDescriptor. This can be done after this method returns. + * @throws IllegalArgumentException If the file descriptor is invalid. + * @throws IllegalStateException If this method is called from the wrong thread. + * @throws IllegalStateException If a transformation is already in progress. + * @throws IOException If an error occurs opening the output file for writing. + */ + @RequiresApi(26) + public void startTransformation(MediaItem mediaItem, ParcelFileDescriptor parcelFileDescriptor) + throws IOException { + startTransformation( + mediaItem, muxerFactory.create(parcelFileDescriptor, transformation.outputMimeType)); + } + + private void startTransformation(MediaItem mediaItem, Muxer muxer) { + verifyApplicationThread(); + if (player != null) { + throw new IllegalStateException("There is already a transformation in progress."); + } + + MuxerWrapper muxerWrapper = new MuxerWrapper(muxer); + this.muxerWrapper = muxerWrapper; + DefaultTrackSelector trackSelector = new DefaultTrackSelector(context); + trackSelector.setParameters( + new DefaultTrackSelector.ParametersBuilder(context) + .setForceHighestSupportedBitrate(true) + .build()); + // Arbitrarily decrease buffers for playback so that samples start being sent earlier to the + // muxer (rebuffers are less problematic for the transformation use case). + DefaultLoadControl loadControl = + new DefaultLoadControl.Builder() + .setBufferDurationsMs( + DEFAULT_MIN_BUFFER_MS, + DEFAULT_MAX_BUFFER_MS, + DEFAULT_BUFFER_FOR_PLAYBACK_MS / 10, + DEFAULT_BUFFER_FOR_PLAYBACK_AFTER_REBUFFER_MS / 10) + .build(); + player = + new SimpleExoPlayer.Builder( + context, new TransformerRenderersFactory(muxerWrapper, transformation)) + .setMediaSourceFactory(mediaSourceFactory) + .setTrackSelector(trackSelector) + .setLoadControl(loadControl) + .setLooper(looper) + .setClock(clock) + .build(); + player.setMediaItem(mediaItem); + player.addAnalyticsListener(new TransformerAnalyticsListener(mediaItem, muxerWrapper)); + player.prepare(); + + progressState = PROGRESS_STATE_WAITING_FOR_AVAILABILITY; + } + + /** + * Returns the {@link Looper} associated with the application thread that's used to access the + * transformer and on which transformer events are received. + */ + public Looper getApplicationLooper() { + return looper; + } + + /** + * Returns the current {@link ProgressState} and updates {@code progressHolder} with the current + * progress if it is {@link #PROGRESS_STATE_AVAILABLE available}. + * + *

    After a transformation {@link Listener#onTransformationCompleted(MediaItem) completes}, this + * method returns {@link #PROGRESS_STATE_NO_TRANSFORMATION}. + * + * @param progressHolder A {@link ProgressHolder}, updated to hold the percentage progress if + * {@link #PROGRESS_STATE_AVAILABLE available}. + * @return The {@link ProgressState}. + * @throws IllegalStateException If this method is called from the wrong thread. + */ + @ProgressState + public int getProgress(ProgressHolder progressHolder) { + verifyApplicationThread(); + if (progressState == PROGRESS_STATE_AVAILABLE) { + Player player = checkNotNull(this.player); + long durationMs = player.getDuration(); + long positionMs = player.getCurrentPosition(); + progressHolder.progress = min((int) (positionMs * 100 / durationMs), 99); + } + return progressState; + } + + /** + * Cancels the transformation that is currently in progress, if any. + * + * @throws IllegalStateException If this method is called from the wrong thread. + */ + public void cancel() { + releaseResources(/* forCancellation= */ true); + } + + /** + * Releases the resources. + * + * @param forCancellation Whether the reason for releasing the resources is the transformation + * cancellation. + * @throws IllegalStateException If this method is called from the wrong thread. + * @throws IllegalStateException If the muxer is in the wrong state and {@code forCancellation} is + * false. + */ + private void releaseResources(boolean forCancellation) { + verifyApplicationThread(); + if (player != null) { + player.release(); + player = null; + } + if (muxerWrapper != null) { + muxerWrapper.release(forCancellation); + muxerWrapper = null; + } + progressState = PROGRESS_STATE_NO_TRANSFORMATION; + } + + private void verifyApplicationThread() { + if (Looper.myLooper() != looper) { + throw new IllegalStateException("Transformer is accessed on the wrong thread."); + } + } + + private static final class TransformerRenderersFactory implements RenderersFactory { + + private final MuxerWrapper muxerWrapper; + private final TransformerMediaClock mediaClock; + private final Transformation transformation; + + public TransformerRenderersFactory(MuxerWrapper muxerWrapper, Transformation transformation) { + this.muxerWrapper = muxerWrapper; + this.transformation = transformation; + mediaClock = new TransformerMediaClock(); + } + + @Override + public Renderer[] createRenderers( + Handler eventHandler, + VideoRendererEventListener videoRendererEventListener, + AudioRendererEventListener audioRendererEventListener, + TextOutput textRendererOutput, + MetadataOutput metadataRendererOutput) { + int rendererCount = transformation.removeAudio || transformation.removeVideo ? 1 : 2; + Renderer[] renderers = new Renderer[rendererCount]; + int index = 0; + if (!transformation.removeAudio) { + renderers[index] = new TransformerAudioRenderer(muxerWrapper, mediaClock, transformation); + index++; + } + if (!transformation.removeVideo) { + renderers[index] = new TransformerVideoRenderer(muxerWrapper, mediaClock, transformation); + index++; + } + return renderers; + } + } + + private final class TransformerAnalyticsListener implements AnalyticsListener { + + private final MediaItem mediaItem; + private final MuxerWrapper muxerWrapper; + + public TransformerAnalyticsListener(MediaItem mediaItem, MuxerWrapper muxerWrapper) { + this.mediaItem = mediaItem; + this.muxerWrapper = muxerWrapper; + } + + @Override + public void onPlaybackStateChanged(EventTime eventTime, int state) { + if (state == Player.STATE_ENDED) { + handleTransformationEnded(/* exception= */ null); + } + } + + @Override + public void onTimelineChanged(EventTime eventTime, int reason) { + if (progressState != PROGRESS_STATE_WAITING_FOR_AVAILABILITY) { + return; + } + Timeline.Window window = new Timeline.Window(); + eventTime.timeline.getWindow(/* windowIndex= */ 0, window); + if (!window.isPlaceholder) { + long durationUs = window.durationUs; + // Make progress permanently unavailable if the duration is unknown, so that it doesn't jump + // to a high value at the end of the transformation if the duration is set once the media is + // entirely loaded. + progressState = + durationUs <= 0 || durationUs == C.TIME_UNSET + ? PROGRESS_STATE_UNAVAILABLE + : PROGRESS_STATE_AVAILABLE; + checkNotNull(player).play(); + } + } + + @Override + public void onTracksChanged( + EventTime eventTime, TrackGroupArray trackGroups, TrackSelectionArray trackSelections) { + if (muxerWrapper.getTrackCount() == 0) { + handleTransformationEnded( + new IllegalStateException( + "The output does not contain any tracks. Check that at least one of the input" + + " sample formats is supported.")); + } + } + + @Override + public void onPlayerError(EventTime eventTime, ExoPlaybackException error) { + handleTransformationEnded(error); + } + + private void handleTransformationEnded(@Nullable Exception exception) { + try { + releaseResources(/* forCancellation= */ false); + } catch (IllegalStateException e) { + if (exception == null) { + exception = e; + } + } + + if (exception == null) { + listener.onTransformationCompleted(mediaItem); + } else { + listener.onTransformationError(mediaItem, exception); + } + } + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerAudioRenderer.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerAudioRenderer.java new file mode 100644 index 00000000000..c0fc23bb78b --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerAudioRenderer.java @@ -0,0 +1,417 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; +import static com.google.android.exoplayer2.util.Assertions.checkState; +import static java.lang.Math.min; + +import android.media.MediaCodec.BufferInfo; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.ExoPlaybackException; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.FormatHolder; +import com.google.android.exoplayer2.audio.AudioProcessor; +import com.google.android.exoplayer2.audio.AudioProcessor.AudioFormat; +import com.google.android.exoplayer2.audio.SonicAudioProcessor; +import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.source.SampleStream; +import java.io.IOException; +import java.nio.ByteBuffer; + +@RequiresApi(18) +/* package */ final class TransformerAudioRenderer extends TransformerBaseRenderer { + + private static final String TAG = "TransformerAudioRenderer"; + private static final int DEFAULT_ENCODER_BITRATE = 128 * 1024; + private static final float SPEED_UNSET = -1f; + + private final DecoderInputBuffer decoderInputBuffer; + private final DecoderInputBuffer encoderInputBuffer; + private final SonicAudioProcessor sonicAudioProcessor; + + @Nullable private MediaCodecAdapterWrapper decoder; + @Nullable private MediaCodecAdapterWrapper encoder; + @Nullable private SpeedProvider speedProvider; + @Nullable private Format inputFormat; + @Nullable private AudioFormat encoderInputAudioFormat; + + private ByteBuffer sonicOutputBuffer; + private long nextEncoderInputBufferTimeUs; + private float currentSpeed; + private boolean muxerWrapperTrackEnded; + private boolean hasEncoderOutputFormat; + private boolean drainingSonicForSpeedChange; + + public TransformerAudioRenderer( + MuxerWrapper muxerWrapper, TransformerMediaClock mediaClock, Transformation transformation) { + super(C.TRACK_TYPE_AUDIO, muxerWrapper, mediaClock, transformation); + decoderInputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DISABLED); + encoderInputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DISABLED); + sonicAudioProcessor = new SonicAudioProcessor(); + sonicOutputBuffer = AudioProcessor.EMPTY_BUFFER; + nextEncoderInputBufferTimeUs = 0; + currentSpeed = SPEED_UNSET; + } + + @Override + public String getName() { + return TAG; + } + + @Override + public boolean isEnded() { + return muxerWrapperTrackEnded; + } + + @Override + protected void onReset() { + decoderInputBuffer.clear(); + decoderInputBuffer.data = null; + encoderInputBuffer.clear(); + encoderInputBuffer.data = null; + sonicAudioProcessor.reset(); + if (decoder != null) { + decoder.release(); + decoder = null; + } + if (encoder != null) { + encoder.release(); + encoder = null; + } + speedProvider = null; + inputFormat = null; + encoderInputAudioFormat = null; + sonicOutputBuffer = AudioProcessor.EMPTY_BUFFER; + nextEncoderInputBufferTimeUs = 0; + currentSpeed = SPEED_UNSET; + muxerWrapperTrackEnded = false; + hasEncoderOutputFormat = false; + drainingSonicForSpeedChange = false; + } + + @Override + public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackException { + if (!isRendererStarted || isEnded()) { + return; + } + + if (ensureDecoderConfigured()) { + if (ensureEncoderAndAudioProcessingConfigured()) { + while (drainEncoderToFeedMuxer()) {} + if (sonicAudioProcessor.isActive()) { + while (drainSonicToFeedEncoder()) {} + while (drainDecoderToFeedSonic()) {} + } else { + while (drainDecoderToFeedEncoder()) {} + } + } + while (feedDecoderInputFromSource()) {} + } + } + + /** + * Attempts to write encoder output data to the muxer, and returns whether it may be possible to + * write more data immediately by calling this method again. + */ + private boolean drainEncoderToFeedMuxer() { + MediaCodecAdapterWrapper encoder = checkNotNull(this.encoder); + if (!hasEncoderOutputFormat) { + @Nullable Format encoderOutputFormat = encoder.getOutputFormat(); + if (encoderOutputFormat == null) { + return false; + } + hasEncoderOutputFormat = true; + muxerWrapper.addTrackFormat(encoderOutputFormat); + } + + if (encoder.isEnded()) { + muxerWrapper.endTrack(getTrackType()); + muxerWrapperTrackEnded = true; + return false; + } + @Nullable ByteBuffer encoderOutputBuffer = encoder.getOutputBuffer(); + if (encoderOutputBuffer == null) { + return false; + } + BufferInfo encoderOutputBufferInfo = checkNotNull(encoder.getOutputBufferInfo()); + if (!muxerWrapper.writeSample( + getTrackType(), + encoderOutputBuffer, + /* isKeyFrame= */ true, + encoderOutputBufferInfo.presentationTimeUs)) { + return false; + } + encoder.releaseOutputBuffer(); + return true; + } + + /** + * Attempts to pass decoder output data to the encoder, and returns whether it may be possible to + * pass more data immediately by calling this method again. + */ + private boolean drainDecoderToFeedEncoder() { + MediaCodecAdapterWrapper decoder = checkNotNull(this.decoder); + MediaCodecAdapterWrapper encoder = checkNotNull(this.encoder); + if (!encoder.maybeDequeueInputBuffer(encoderInputBuffer)) { + return false; + } + + if (decoder.isEnded()) { + queueEndOfStreamToEncoder(); + return false; + } + + @Nullable ByteBuffer decoderOutputBuffer = decoder.getOutputBuffer(); + if (decoderOutputBuffer == null) { + return false; + } + if (isSpeedChanging(checkNotNull(decoder.getOutputBufferInfo()))) { + flushSonicAndSetSpeed(currentSpeed); + return false; + } + feedEncoder(decoderOutputBuffer); + if (!decoderOutputBuffer.hasRemaining()) { + decoder.releaseOutputBuffer(); + } + return true; + } + + /** + * Attempts to pass audio processor output data to the encoder, and returns whether it may be + * possible to pass more data immediately by calling this method again. + */ + private boolean drainSonicToFeedEncoder() { + MediaCodecAdapterWrapper encoder = checkNotNull(this.encoder); + if (!encoder.maybeDequeueInputBuffer(encoderInputBuffer)) { + return false; + } + + if (!sonicOutputBuffer.hasRemaining()) { + sonicOutputBuffer = sonicAudioProcessor.getOutput(); + if (!sonicOutputBuffer.hasRemaining()) { + if (checkNotNull(decoder).isEnded() && sonicAudioProcessor.isEnded()) { + queueEndOfStreamToEncoder(); + } + return false; + } + } + + feedEncoder(sonicOutputBuffer); + return true; + } + + /** + * Attempts to process decoder output audio, and returns whether it may be possible to process + * more data immediately by calling this method again. + */ + private boolean drainDecoderToFeedSonic() { + MediaCodecAdapterWrapper decoder = checkNotNull(this.decoder); + + if (drainingSonicForSpeedChange) { + if (sonicAudioProcessor.isEnded() && !sonicOutputBuffer.hasRemaining()) { + flushSonicAndSetSpeed(currentSpeed); + drainingSonicForSpeedChange = false; + } + return false; + } + + // Sonic invalidates any previous output buffer when more input is queued, so we don't queue if + // there is output still to be processed. + if (sonicOutputBuffer.hasRemaining()) { + return false; + } + + if (decoder.isEnded()) { + sonicAudioProcessor.queueEndOfStream(); + return false; + } + checkState(!sonicAudioProcessor.isEnded()); + + @Nullable ByteBuffer decoderOutputBuffer = decoder.getOutputBuffer(); + if (decoderOutputBuffer == null) { + return false; + } + if (isSpeedChanging(checkNotNull(decoder.getOutputBufferInfo()))) { + sonicAudioProcessor.queueEndOfStream(); + drainingSonicForSpeedChange = true; + return false; + } + sonicAudioProcessor.queueInput(decoderOutputBuffer); + if (!decoderOutputBuffer.hasRemaining()) { + decoder.releaseOutputBuffer(); + } + return true; + } + + /** + * Attempts to pass input data to the decoder, and returns whether it may be possible to pass more + * data immediately by calling this method again. + */ + private boolean feedDecoderInputFromSource() { + MediaCodecAdapterWrapper decoder = checkNotNull(this.decoder); + if (!decoder.maybeDequeueInputBuffer(decoderInputBuffer)) { + return false; + } + + decoderInputBuffer.clear(); + @SampleStream.ReadDataResult + int result = readSource(getFormatHolder(), decoderInputBuffer, /* formatRequired= */ false); + switch (result) { + case C.RESULT_BUFFER_READ: + mediaClock.updateTimeForTrackType(getTrackType(), decoderInputBuffer.timeUs); + decoderInputBuffer.flip(); + decoder.queueInputBuffer(decoderInputBuffer); + return !decoderInputBuffer.isEndOfStream(); + case C.RESULT_FORMAT_READ: + throw new IllegalStateException("Format changes are not supported."); + case C.RESULT_NOTHING_READ: + default: + return false; + } + } + + /** + * Feeds as much data as possible between the current position and limit of the specified {@link + * ByteBuffer} to the encoder, and advances its position by the number of bytes fed. + */ + private void feedEncoder(ByteBuffer inputBuffer) { + AudioFormat encoderInputAudioFormat = checkNotNull(this.encoderInputAudioFormat); + MediaCodecAdapterWrapper encoder = checkNotNull(this.encoder); + ByteBuffer encoderInputBufferData = checkNotNull(encoderInputBuffer.data); + int bufferLimit = inputBuffer.limit(); + inputBuffer.limit(min(bufferLimit, inputBuffer.position() + encoderInputBufferData.capacity())); + encoderInputBufferData.put(inputBuffer); + encoderInputBuffer.timeUs = nextEncoderInputBufferTimeUs; + nextEncoderInputBufferTimeUs += + getBufferDurationUs( + /* bytesWritten= */ encoderInputBufferData.position(), + encoderInputAudioFormat.bytesPerFrame, + encoderInputAudioFormat.sampleRate); + encoderInputBuffer.setFlags(0); + encoderInputBuffer.flip(); + inputBuffer.limit(bufferLimit); + encoder.queueInputBuffer(encoderInputBuffer); + } + + private void queueEndOfStreamToEncoder() { + MediaCodecAdapterWrapper encoder = checkNotNull(this.encoder); + checkState(checkNotNull(encoderInputBuffer.data).position() == 0); + encoderInputBuffer.addFlag(C.BUFFER_FLAG_END_OF_STREAM); + encoderInputBuffer.flip(); + // Queuing EOS should only occur with an empty buffer. + encoder.queueInputBuffer(encoderInputBuffer); + } + + /** + * Attempts to configure the {@link #encoder} and Sonic (if applicable), if they have not been + * configured yet, and returns whether they have been configured. + */ + private boolean ensureEncoderAndAudioProcessingConfigured() throws ExoPlaybackException { + if (encoder != null) { + return true; + } + MediaCodecAdapterWrapper decoder = checkNotNull(this.decoder); + @Nullable Format decoderOutputFormat = decoder.getOutputFormat(); + if (decoderOutputFormat == null) { + return false; + } + AudioFormat outputAudioFormat = + new AudioFormat( + decoderOutputFormat.sampleRate, + decoderOutputFormat.channelCount, + decoderOutputFormat.pcmEncoding); + if (transformation.flattenForSlowMotion) { + try { + outputAudioFormat = sonicAudioProcessor.configure(outputAudioFormat); + flushSonicAndSetSpeed(currentSpeed); + } catch (AudioProcessor.UnhandledAudioFormatException e) { + throw createRendererException(e); + } + } + try { + encoder = + MediaCodecAdapterWrapper.createForAudioEncoding( + new Format.Builder() + .setSampleMimeType(checkNotNull(inputFormat).sampleMimeType) + .setSampleRate(outputAudioFormat.sampleRate) + .setChannelCount(outputAudioFormat.channelCount) + .setAverageBitrate(DEFAULT_ENCODER_BITRATE) + .build()); + } catch (IOException e) { + throw createRendererException(e); + } + encoderInputAudioFormat = outputAudioFormat; + return true; + } + + /** + * Attempts to configure the {@link #decoder} if it has not been configured yet, and returns + * whether the decoder has been configured. + */ + private boolean ensureDecoderConfigured() throws ExoPlaybackException { + if (decoder != null) { + return true; + } + + FormatHolder formatHolder = getFormatHolder(); + @SampleStream.ReadDataResult + int result = readSource(formatHolder, decoderInputBuffer, /* formatRequired= */ true); + if (result != C.RESULT_FORMAT_READ) { + return false; + } + inputFormat = checkNotNull(formatHolder.format); + try { + decoder = MediaCodecAdapterWrapper.createForAudioDecoding(inputFormat); + } catch (IOException e) { + throw createRendererException(e); + } + speedProvider = new SegmentSpeedProvider(inputFormat); + currentSpeed = speedProvider.getSpeed(0); + return true; + } + + private boolean isSpeedChanging(BufferInfo bufferInfo) { + if (!transformation.flattenForSlowMotion) { + return false; + } + float newSpeed = checkNotNull(speedProvider).getSpeed(bufferInfo.presentationTimeUs); + boolean speedChanging = newSpeed != currentSpeed; + currentSpeed = newSpeed; + return speedChanging; + } + + private void flushSonicAndSetSpeed(float speed) { + sonicAudioProcessor.setSpeed(speed); + sonicAudioProcessor.setPitch(speed); + sonicAudioProcessor.flush(); + } + + private ExoPlaybackException createRendererException(Throwable cause) { + return ExoPlaybackException.createForRenderer( + cause, TAG, getIndex(), inputFormat, /* rendererFormatSupport= */ C.FORMAT_HANDLED); + } + + private static long getBufferDurationUs(long bytesWritten, int bytesPerFrame, int sampleRate) { + long framesWritten = bytesWritten / bytesPerFrame; + return framesWritten * C.MICROS_PER_SECOND / sampleRate; + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerBaseRenderer.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerBaseRenderer.java new file mode 100644 index 00000000000..33888226b88 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerBaseRenderer.java @@ -0,0 +1,87 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + + +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.BaseRenderer; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.RendererCapabilities; +import com.google.android.exoplayer2.util.MediaClock; +import com.google.android.exoplayer2.util.MimeTypes; + +@RequiresApi(18) +/* package */ abstract class TransformerBaseRenderer extends BaseRenderer { + + protected final MuxerWrapper muxerWrapper; + protected final TransformerMediaClock mediaClock; + protected final Transformation transformation; + + protected boolean isRendererStarted; + + public TransformerBaseRenderer( + int trackType, + MuxerWrapper muxerWrapper, + TransformerMediaClock mediaClock, + Transformation transformation) { + super(trackType); + this.muxerWrapper = muxerWrapper; + this.mediaClock = mediaClock; + this.transformation = transformation; + } + + @Override + @C.FormatSupport + public final int supportsFormat(Format format) { + @Nullable String sampleMimeType = format.sampleMimeType; + if (MimeTypes.getTrackType(sampleMimeType) != getTrackType()) { + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_TYPE); + } else if (muxerWrapper.supportsSampleMimeType(sampleMimeType)) { + return RendererCapabilities.create(C.FORMAT_HANDLED); + } else { + return RendererCapabilities.create(C.FORMAT_UNSUPPORTED_SUBTYPE); + } + } + + @Override + public final boolean isReady() { + return isSourceReady(); + } + + @Override + public final MediaClock getMediaClock() { + return mediaClock; + } + + @Override + protected final void onEnabled(boolean joining, boolean mayRenderStartOfStream) { + muxerWrapper.registerTrack(); + mediaClock.updateTimeForTrackType(getTrackType(), 0L); + } + + @Override + protected final void onStarted() { + isRendererStarted = true; + } + + @Override + protected final void onStopped() { + isRendererStarted = false; + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerMediaClock.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerMediaClock.java new file mode 100644 index 00000000000..210eaf0ecd8 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerMediaClock.java @@ -0,0 +1,68 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.util.Util.minValue; + +import android.util.SparseLongArray; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.PlaybackParameters; +import com.google.android.exoplayer2.util.MediaClock; + +@RequiresApi(18) +/* package */ final class TransformerMediaClock implements MediaClock { + + private final SparseLongArray trackTypeToTimeUs; + private long minTrackTimeUs; + + public TransformerMediaClock() { + trackTypeToTimeUs = new SparseLongArray(); + } + + /** + * Updates the time for a given track type. The clock time is computed based on the different + * track times. + */ + public void updateTimeForTrackType(int trackType, long timeUs) { + long previousTimeUs = trackTypeToTimeUs.get(trackType, /* valueIfKeyNotFound= */ C.TIME_UNSET); + if (previousTimeUs != C.TIME_UNSET && timeUs <= previousTimeUs) { + // Make sure that the track times are increasing and therefore that the clock time is + // increasing. This is necessary for progress updates. + return; + } + trackTypeToTimeUs.put(trackType, timeUs); + if (previousTimeUs == C.TIME_UNSET || previousTimeUs == minTrackTimeUs) { + minTrackTimeUs = minValue(trackTypeToTimeUs); + } + } + + @Override + public long getPositionUs() { + // Use minimum position among tracks as position to ensure that the buffered duration is + // positive. This is also useful for controlling samples interleaving. + return minTrackTimeUs; + } + + @Override + public void setPlaybackParameters(PlaybackParameters playbackParameters) {} + + @Override + public PlaybackParameters getPlaybackParameters() { + // Playback parameters are unknown. Set default value. + return PlaybackParameters.DEFAULT; + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerVideoRenderer.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerVideoRenderer.java new file mode 100644 index 00000000000..621dab6f5c1 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/TransformerVideoRenderer.java @@ -0,0 +1,128 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.util.Assertions.checkNotNull; + +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.FormatHolder; +import com.google.android.exoplayer2.decoder.DecoderInputBuffer; +import com.google.android.exoplayer2.source.SampleStream; +import java.nio.ByteBuffer; + +@RequiresApi(18) +/* package */ final class TransformerVideoRenderer extends TransformerBaseRenderer { + + private static final String TAG = "TransformerVideoRenderer"; + + private final DecoderInputBuffer buffer; + + @Nullable private SampleTransformer sampleTransformer; + + private boolean formatRead; + private boolean isBufferPending; + private boolean isInputStreamEnded; + + public TransformerVideoRenderer( + MuxerWrapper muxerWrapper, TransformerMediaClock mediaClock, Transformation transformation) { + super(C.TRACK_TYPE_VIDEO, muxerWrapper, mediaClock, transformation); + buffer = new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + } + + @Override + public String getName() { + return TAG; + } + + @Override + public void render(long positionUs, long elapsedRealtimeUs) { + if (!isRendererStarted || isEnded()) { + return; + } + + if (!formatRead) { + FormatHolder formatHolder = getFormatHolder(); + @SampleStream.ReadDataResult + int result = readSource(formatHolder, buffer, /* formatRequired= */ true); + if (result != C.RESULT_FORMAT_READ) { + return; + } + Format format = checkNotNull(formatHolder.format); + formatRead = true; + if (transformation.flattenForSlowMotion) { + sampleTransformer = new SefSlowMotionVideoSampleTransformer(format); + } + muxerWrapper.addTrackFormat(format); + } + + while (true) { + // Read sample. + if (!isBufferPending && !readAndTransformBuffer()) { + return; + } + // Write sample. + isBufferPending = + !muxerWrapper.writeSample( + getTrackType(), buffer.data, buffer.isKeyFrame(), buffer.timeUs); + if (isBufferPending) { + return; + } + } + } + + @Override + public boolean isEnded() { + return isInputStreamEnded; + } + + /** + * Checks whether a sample can be read and, if so, reads it, transforms it and writes the + * resulting sample to the {@link #buffer}. + * + *

    The buffer data can be set to null if the transformation applied discards the sample. + * + * @return Whether a sample has been read and transformed. + */ + private boolean readAndTransformBuffer() { + buffer.clear(); + @SampleStream.ReadDataResult + int result = readSource(getFormatHolder(), buffer, /* formatRequired= */ false); + if (result == C.RESULT_FORMAT_READ) { + throw new IllegalStateException("Format changes are not supported."); + } else if (result == C.RESULT_NOTHING_READ) { + return false; + } + + // Buffer read. + + if (buffer.isEndOfStream()) { + isInputStreamEnded = true; + muxerWrapper.endTrack(getTrackType()); + return false; + } + mediaClock.updateTimeForTrackType(getTrackType(), buffer.timeUs); + ByteBuffer data = checkNotNull(buffer.data); + data.flip(); + if (sampleTransformer != null) { + sampleTransformer.transformSample(buffer); + } + return true; + } +} diff --git a/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/package-info.java b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/package-info.java new file mode 100644 index 00000000000..1093e108822 --- /dev/null +++ b/library/transformer/src/main/java/com/google/android/exoplayer2/transformer/package-info.java @@ -0,0 +1,19 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +@NonNullApi +package com.google.android.exoplayer2.transformer; + +import com.google.android.exoplayer2.util.NonNullApi; diff --git a/library/transformer/src/test/AndroidManifest.xml b/library/transformer/src/test/AndroidManifest.xml new file mode 100644 index 00000000000..0ef3273ee03 --- /dev/null +++ b/library/transformer/src/test/AndroidManifest.xml @@ -0,0 +1,19 @@ + + + + + + diff --git a/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/SefSlowMotionVideoSampleTransformerTest.java b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/SefSlowMotionVideoSampleTransformerTest.java new file mode 100644 index 00000000000..a63db831fc8 --- /dev/null +++ b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/SefSlowMotionVideoSampleTransformerTest.java @@ -0,0 +1,301 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.transformer.SefSlowMotionVideoSampleTransformer.INPUT_FRAME_RATE; +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData; +import com.google.android.exoplayer2.metadata.mp4.SmtaMetadataEntry; +import com.google.android.exoplayer2.util.MimeTypes; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link SefSlowMotionVideoSampleTransformer}. */ +@RunWith(AndroidJUnit4.class) +public class SefSlowMotionVideoSampleTransformerTest { + + /** + * Sequence of temporal SVC layers in an SEF slow motion video track with a maximum layer of 3. + * + *

    Each value is attached to a frame and the sequence is repeated until there is no frame left. + */ + private static final int[] LAYER_SEQUENCE_MAX_LAYER_THREE = new int[] {0, 3, 2, 3, 1, 3, 2, 3}; + + @Test + public void processCurrentFrame_240fps_keepsExpectedFrames() { + int captureFrameRate = 240; + int inputMaxLayer = 3; + int frameCount = 46; + SlowMotionData.Segment segment1 = + createSegment(/* startFrameIndex= */ 11, /* endFrameIndex= */ 17, /* speedDivisor= */ 2); + SlowMotionData.Segment segment2 = + createSegment(/* startFrameIndex= */ 31, /* endFrameIndex= */ 38, /* speedDivisor= */ 8); + Format format = + createSefSlowMotionFormat( + captureFrameRate, inputMaxLayer, Arrays.asList(segment1, segment2)); + + SefSlowMotionVideoSampleTransformer sampleTransformer = + new SefSlowMotionVideoSampleTransformer(format); + List outputLayers = + getKeptOutputLayers(sampleTransformer, LAYER_SEQUENCE_MAX_LAYER_THREE, frameCount); + + List expectedLayers = Arrays.asList(0, 0, 1, 0, 0, 1, 2, 3, 0, 3, 2, 3, 1, 3, 0); + assertThat(outputLayers).isEqualTo(expectedLayers); + } + + @Test + public void processCurrentFrame_120fps_keepsExpectedFrames() { + int captureFrameRate = 120; + int inputMaxLayer = 3; + int frameCount = 46; + SlowMotionData.Segment segment1 = + createSegment(/* startFrameIndex= */ 9, /* endFrameIndex= */ 17, /* speedDivisor= */ 4); + SlowMotionData.Segment segment2 = + createSegment(/* startFrameIndex= */ 31, /* endFrameIndex= */ 38, /* speedDivisor= */ 8); + Format format = + createSefSlowMotionFormat( + captureFrameRate, inputMaxLayer, Arrays.asList(segment1, segment2)); + + SefSlowMotionVideoSampleTransformer sampleTransformer = + new SefSlowMotionVideoSampleTransformer(format); + List outputLayers = + getKeptOutputLayers(sampleTransformer, LAYER_SEQUENCE_MAX_LAYER_THREE, frameCount); + + List expectedLayers = + Arrays.asList(0, 1, 0, 3, 2, 3, 1, 3, 2, 3, 0, 1, 0, 1, 2, 3, 0, 3, 2, 3, 1, 3, 0, 1); + assertThat(outputLayers).isEqualTo(expectedLayers); + } + + @Test + public void processCurrentFrame_contiguousSegments_keepsExpectedFrames() { + int captureFrameRate = 240; + int inputMaxLayer = 3; + int frameCount = 26; + SlowMotionData.Segment segment1 = + createSegment(/* startFrameIndex= */ 11, /* endFrameIndex= */ 19, /* speedDivisor= */ 2); + SlowMotionData.Segment segment2 = + createSegment(/* startFrameIndex= */ 19, /* endFrameIndex= */ 22, /* speedDivisor= */ 8); + Format format = + createSefSlowMotionFormat( + captureFrameRate, inputMaxLayer, Arrays.asList(segment1, segment2)); + + SefSlowMotionVideoSampleTransformer sampleTransformer = + new SefSlowMotionVideoSampleTransformer(format); + List outputLayers = + getKeptOutputLayers(sampleTransformer, LAYER_SEQUENCE_MAX_LAYER_THREE, frameCount); + + List expectedLayers = Arrays.asList(0, 0, 1, 0, 2, 3, 1, 3, 0); + assertThat(outputLayers).isEqualTo(expectedLayers); + } + + @Test + public void processCurrentFrame_skipsSegmentsWithNoFrame() { + int captureFrameRate = 240; + int inputMaxLayer = 3; + int frameCount = 16; + SlowMotionData.Segment segmentWithNoFrame1 = + new SlowMotionData.Segment( + /* startTimeMs= */ 120, /* endTimeMs= */ 130, /* speedDivisor= */ 2); + SlowMotionData.Segment segmentWithNoFrame2 = + new SlowMotionData.Segment( + /* startTimeMs= */ 270, /* endTimeMs= */ 280, /* speedDivisor= */ 2); + SlowMotionData.Segment segmentWithFrame = + createSegment(/* startFrameIndex= */ 11, /* endFrameIndex= */ 16, /* speedDivisor= */ 2); + Format format = + createSefSlowMotionFormat( + captureFrameRate, + inputMaxLayer, + Arrays.asList(segmentWithNoFrame1, segmentWithNoFrame2, segmentWithFrame)); + + SefSlowMotionVideoSampleTransformer sampleTransformer = + new SefSlowMotionVideoSampleTransformer(format); + List outputLayers = + getKeptOutputLayers(sampleTransformer, LAYER_SEQUENCE_MAX_LAYER_THREE, frameCount); + + List expectedLayers = Arrays.asList(0, 0, 1); + assertThat(outputLayers).isEqualTo(expectedLayers); + } + + @Test + public void getCurrentFrameOutputTimeUs_240fps_outputsExpectedTimes() { + int captureFrameRate = 240; + int inputMaxLayer = 3; + int frameCount = 16; + SlowMotionData.Segment segment1 = + new SlowMotionData.Segment( + /* startTimeMs= */ 50, /* endTimeMs= */ 150, /* speedDivisor= */ 2); + SlowMotionData.Segment segment2 = + new SlowMotionData.Segment( + /* startTimeMs= */ 210, /* endTimeMs= */ 360, /* speedDivisor= */ 8); + Format format = + createSefSlowMotionFormat( + captureFrameRate, inputMaxLayer, Arrays.asList(segment1, segment2)); + + SefSlowMotionVideoSampleTransformer sampleTransformer = + new SefSlowMotionVideoSampleTransformer(format); + List outputTimesUs = + getOutputTimesUs(sampleTransformer, LAYER_SEQUENCE_MAX_LAYER_THREE, frameCount); + + // Test frame inside segment. + assertThat(outputTimesUs.get(9)) + .isEqualTo(Math.round((300.0 + 100 + (300 - 210) * 7) * 1000 * 30 / 240)); + // Test frame outside segment. + assertThat(outputTimesUs.get(13)) + .isEqualTo(Math.round((433 + 1 / 3.0 + 100 + 150 * 7) * 1000 * 30 / 240)); + } + + @Test + public void getCurrentFrameOutputTimeUs_120fps_outputsExpectedTimes() { + int captureFrameRate = 120; + int inputMaxLayer = 3; + int frameCount = 16; + SlowMotionData.Segment segment1 = + new SlowMotionData.Segment( + /* startTimeMs= */ 50, /* endTimeMs= */ 150, /* speedDivisor= */ 2); + SlowMotionData.Segment segment2 = + new SlowMotionData.Segment( + /* startTimeMs= */ 210, /* endTimeMs= */ 360, /* speedDivisor= */ 8); + Format format = + createSefSlowMotionFormat( + captureFrameRate, inputMaxLayer, Arrays.asList(segment1, segment2)); + + SefSlowMotionVideoSampleTransformer sampleTransformer = + new SefSlowMotionVideoSampleTransformer(format); + List outputTimesUs = + getOutputTimesUs(sampleTransformer, LAYER_SEQUENCE_MAX_LAYER_THREE, frameCount); + + // Test frame inside segment. + assertThat(outputTimesUs.get(9)) + .isEqualTo(Math.round((300.0 + 100 + (300 - 210) * 7) * 1000 * 30 / 120)); + // Test frame outside segment. + assertThat(outputTimesUs.get(13)) + .isEqualTo(Math.round((433 + 1 / 3.0 + 100 + 150 * 7) * 1000 * 30 / 120)); + } + + @Test + public void getCurrentFrameOutputTimeUs_contiguousSegments_outputsExpectedTimes() { + int captureFrameRate = 240; + int inputMaxLayer = 3; + int frameCount = 16; + SlowMotionData.Segment segment1 = + new SlowMotionData.Segment( + /* startTimeMs= */ 50, /* endTimeMs= */ 210, /* speedDivisor= */ 2); + SlowMotionData.Segment segment2 = + new SlowMotionData.Segment( + /* startTimeMs= */ 210, /* endTimeMs= */ 360, /* speedDivisor= */ 8); + Format format = + createSefSlowMotionFormat( + captureFrameRate, inputMaxLayer, Arrays.asList(segment1, segment2)); + + SefSlowMotionVideoSampleTransformer sampleTransformer = + new SefSlowMotionVideoSampleTransformer(format); + List outputTimesUs = + getOutputTimesUs(sampleTransformer, LAYER_SEQUENCE_MAX_LAYER_THREE, frameCount); + + // Test frame inside second segment. + assertThat(outputTimesUs.get(9)).isEqualTo(136_250); + } + + /** + * Creates a {@link SlowMotionData.Segment}. + * + * @param startFrameIndex The index of the first frame in the segment. + * @param endFrameIndex The index of the first frame following the segment. + * @param speedDivisor The factor by which the input is slowed down in the segment. + * @return A {@link SlowMotionData.Segment}. + */ + private static SlowMotionData.Segment createSegment( + int startFrameIndex, int endFrameIndex, int speedDivisor) { + return new SlowMotionData.Segment( + /* startTimeMs= */ (int) (startFrameIndex * C.MILLIS_PER_SECOND / INPUT_FRAME_RATE), + /* endTimeMs= */ (int) (endFrameIndex * C.MILLIS_PER_SECOND / INPUT_FRAME_RATE) - 1, + speedDivisor); + } + + /** Creates a {@link Format} for an SEF slow motion video track. */ + private static Format createSefSlowMotionFormat( + int captureFrameRate, + int inputMaxLayer, + List segments) { + SmtaMetadataEntry smtaMetadataEntry = + new SmtaMetadataEntry(captureFrameRate, /* svcTemporalLayerCount= */ inputMaxLayer + 1); + SlowMotionData slowMotionData = new SlowMotionData(segments); + Metadata metadata = new Metadata(smtaMetadataEntry, slowMotionData); + return new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setMetadata(metadata) + .build(); + } + + /** + * Returns a list containing the temporal SVC layers of the frames that should be kept according + * to {@link SefSlowMotionVideoSampleTransformer#processCurrentFrame(int, long)}. + * + * @param sampleTransformer The {@link SefSlowMotionVideoSampleTransformer}. + * @param layerSequence The sequence of layer values in the input. + * @param frameCount The number of video frames in the input. + * @return The output layers. + */ + private static List getKeptOutputLayers( + SefSlowMotionVideoSampleTransformer sampleTransformer, + int[] layerSequence, + int frameCount) { + List outputLayers = new ArrayList<>(); + for (int i = 0; i < frameCount; i++) { + int layer = layerSequence[i % layerSequence.length]; + long timeUs = i * C.MICROS_PER_SECOND / INPUT_FRAME_RATE; + if (sampleTransformer.processCurrentFrame(layer, timeUs)) { + outputLayers.add(layer); + } + } + return outputLayers; + } + + /** + * Returns a list containing the frame output times obtained using {@link + * SefSlowMotionVideoSampleTransformer#getCurrentFrameOutputTimeUs(long)}. + * + *

    The output contains the output times for all the input frames, regardless of whether they + * should be kept or not. + * + * @param sampleTransformer The {@link SefSlowMotionVideoSampleTransformer}. + * @param layerSequence The sequence of layer values in the input. + * @param frameCount The number of video frames in the input. + * @return The frame output times, in microseconds. + */ + private static List getOutputTimesUs( + SefSlowMotionVideoSampleTransformer sampleTransformer, + int[] layerSequence, + int frameCount) { + List outputTimesUs = new ArrayList<>(); + for (int i = 0; i < frameCount; i++) { + int layer = layerSequence[i % layerSequence.length]; + long inputTimeUs = i * C.MICROS_PER_SECOND / INPUT_FRAME_RATE; + sampleTransformer.processCurrentFrame(layer, inputTimeUs); + outputTimesUs.add(sampleTransformer.getCurrentFrameOutputTimeUs(inputTimeUs)); + } + return outputTimesUs; + } +} diff --git a/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/SegmentSpeedProviderTest.java b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/SegmentSpeedProviderTest.java new file mode 100644 index 00000000000..616443e9632 --- /dev/null +++ b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/SegmentSpeedProviderTest.java @@ -0,0 +1,85 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformer; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.C; +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.metadata.Metadata; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData; +import com.google.android.exoplayer2.metadata.mp4.SlowMotionData.Segment; +import com.google.android.exoplayer2.metadata.mp4.SmtaMetadataEntry; +import com.google.common.collect.ImmutableList; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link SegmentSpeedProvider}. */ +@RunWith(AndroidJUnit4.class) +public class SegmentSpeedProviderTest { + + private static final SmtaMetadataEntry SMTA_SPEED_8 = + new SmtaMetadataEntry(/* captureFrameRate= */ 240, /* svcTemporalLayerCount= */ 4); + + @Test + public void getSpeed_noSegments_returnsBaseSpeed() { + SegmentSpeedProvider provider = + new SegmentSpeedProvider( + new Format.Builder().setMetadata(new Metadata(SMTA_SPEED_8)).build()); + assertThat(provider.getSpeed(0)).isEqualTo(8); + assertThat(provider.getSpeed(1_000_000)).isEqualTo(8); + } + + @Test + public void getSpeed_returnsCorrectSpeed() { + List segments = + ImmutableList.of( + new Segment(/* startTimeMs= */ 500, /* endTimeMs= */ 1000, /* speedDivisor= */ 8), + new Segment(/* startTimeMs= */ 1500, /* endTimeMs= */ 2000, /* speedDivisor= */ 4), + new Segment(/* startTimeMs= */ 2000, /* endTimeMs= */ 2500, /* speedDivisor= */ 2)); + + SegmentSpeedProvider provider = + new SegmentSpeedProvider( + new Format.Builder() + .setMetadata(new Metadata(new SlowMotionData(segments), SMTA_SPEED_8)) + .build()); + + assertThat(provider.getSpeed(C.msToUs(0))).isEqualTo(8); + assertThat(provider.getSpeed(C.msToUs(500))).isEqualTo(1); + assertThat(provider.getSpeed(C.msToUs(800))).isEqualTo(1); + assertThat(provider.getSpeed(C.msToUs(1000))).isEqualTo(8); + assertThat(provider.getSpeed(C.msToUs(1250))).isEqualTo(8); + assertThat(provider.getSpeed(C.msToUs(1500))).isEqualTo(2); + assertThat(provider.getSpeed(C.msToUs(1650))).isEqualTo(2); + assertThat(provider.getSpeed(C.msToUs(2000))).isEqualTo(4); + assertThat(provider.getSpeed(C.msToUs(2400))).isEqualTo(4); + assertThat(provider.getSpeed(C.msToUs(2500))).isEqualTo(8); + assertThat(provider.getSpeed(C.msToUs(3000))).isEqualTo(8); + } + + @Test + public void getSpeed_withNegativeTimestamp_throwsException() { + assertThrows( + IllegalArgumentException.class, + () -> + new SegmentSpeedProvider( + new Format.Builder().setMetadata(new Metadata(SMTA_SPEED_8)).build()) + .getSpeed(-1)); + } +} diff --git a/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TestMuxer.java b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TestMuxer.java new file mode 100644 index 00000000000..e4835274c23 --- /dev/null +++ b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TestMuxer.java @@ -0,0 +1,109 @@ +/* + * Copyright 2021 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package com.google.android.exoplayer2.transformer; + +import com.google.android.exoplayer2.Format; +import com.google.android.exoplayer2.testutil.DumpableFormat; +import com.google.android.exoplayer2.testutil.Dumper; +import java.io.IOException; +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +/** + * An implementation of {@link Muxer} that supports dumping information about all interactions (for + * testing purposes) and delegates the actual muxing operations to a {@link FrameworkMuxer}. + */ +public final class TestMuxer implements Muxer, Dumper.Dumpable { + + private final Muxer frameworkMuxer; + private final List dumpables; + + /** Creates a new test muxer. */ + public TestMuxer(String path, String outputMimeType) throws IOException { + frameworkMuxer = new FrameworkMuxer.Factory().create(path, outputMimeType); + dumpables = new ArrayList<>(); + dumpables.add(dumper -> dumper.add("containerMimeType", outputMimeType)); + } + + // Muxer implementation. + + @Override + public boolean supportsSampleMimeType(String mimeType) { + return frameworkMuxer.supportsSampleMimeType(mimeType); + } + + @Override + public int addTrack(Format format) { + int trackIndex = frameworkMuxer.addTrack(format); + dumpables.add(new DumpableFormat(format, trackIndex)); + return trackIndex; + } + + @Override + public void writeSampleData( + int trackIndex, ByteBuffer data, boolean isKeyFrame, long presentationTimeUs) { + dumpables.add(new DumpableSample(trackIndex, data, isKeyFrame, presentationTimeUs)); + frameworkMuxer.writeSampleData(trackIndex, data, isKeyFrame, presentationTimeUs); + } + + @Override + public void release(boolean forCancellation) { + dumpables.add(dumper -> dumper.add("released", true)); + frameworkMuxer.release(forCancellation); + } + + // Dumper.Dumpable implementation. + + @Override + public void dump(Dumper dumper) { + for (Dumper.Dumpable dumpable : dumpables) { + dumpable.dump(dumper); + } + } + + private static final class DumpableSample implements Dumper.Dumpable { + + private final int trackIndex; + private final long presentationTimeUs; + private final boolean isKeyFrame; + private final int sampleDataHashCode; + + public DumpableSample( + int trackIndex, ByteBuffer sample, boolean isKeyFrame, long presentationTimeUs) { + this.trackIndex = trackIndex; + this.presentationTimeUs = presentationTimeUs; + this.isKeyFrame = isKeyFrame; + int initialPosition = sample.position(); + byte[] data = new byte[sample.remaining()]; + sample.get(data); + sample.position(initialPosition); + sampleDataHashCode = Arrays.hashCode(data); + } + + @Override + public void dump(Dumper dumper) { + dumper + .startBlock("sample") + .add("trackIndex", trackIndex) + .add("dataHashCode", sampleDataHashCode) + .add("isKeyFrame", isKeyFrame) + .add("presentationTimeUs", presentationTimeUs) + .endBlock(); + } + } +} diff --git a/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerBuilderTest.java b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerBuilderTest.java new file mode 100644 index 00000000000..8cfba3156db --- /dev/null +++ b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerBuilderTest.java @@ -0,0 +1,57 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static org.junit.Assert.assertThrows; + +import android.content.Context; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.util.MimeTypes; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link Transformer.Builder}. */ +@RunWith(AndroidJUnit4.class) +public class TransformerBuilderTest { + + @Test + public void setOutputMimeType_unsupportedMimeType_throws() { + assertThrows( + IllegalStateException.class, + () -> new Transformer.Builder().setOutputMimeType(MimeTypes.VIDEO_FLV).build()); + } + + @Test + public void build_withoutContext_throws() { + assertThrows(IllegalStateException.class, () -> new Transformer.Builder().build()); + } + + @Test + public void build_removeAudioAndVideo_throws() { + Context context = ApplicationProvider.getApplicationContext(); + + assertThrows( + IllegalStateException.class, + () -> + new Transformer.Builder() + .setContext(context) + .setRemoveAudio(true) + .setRemoveVideo(true) + .build()); + } +} diff --git a/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerTest.java b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerTest.java new file mode 100644 index 00000000000..757f9587f64 --- /dev/null +++ b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerTest.java @@ -0,0 +1,588 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.transformer.Transformer.PROGRESS_STATE_AVAILABLE; +import static com.google.android.exoplayer2.transformer.Transformer.PROGRESS_STATE_NO_TRANSFORMATION; +import static com.google.android.exoplayer2.transformer.Transformer.PROGRESS_STATE_UNAVAILABLE; +import static com.google.android.exoplayer2.transformer.Transformer.PROGRESS_STATE_WAITING_FOR_AVAILABILITY; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import android.content.Context; +import android.os.Handler; +import android.os.HandlerThread; +import android.os.Looper; +import android.os.Message; +import android.os.ParcelFileDescriptor; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.exoplayer2.ExoPlaybackException; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.testutil.AutoAdvancingFakeClock; +import com.google.android.exoplayer2.testutil.DumpFileAsserts; +import com.google.android.exoplayer2.util.MimeTypes; +import com.google.android.exoplayer2.util.Util; +import com.google.common.collect.Iterables; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.shadows.ShadowMediaCodec; + +/** Unit test for {@link Transformer}. */ +@RunWith(AndroidJUnit4.class) +public final class TransformerTest { + + private static final String URI_PREFIX = "asset:///media/"; + private static final String FILE_VIDEO_ONLY = "mkv/sample.mkv"; + private static final String FILE_AUDIO_ONLY = "amr/sample_nb.amr"; + private static final String FILE_AUDIO_VIDEO = "mp4/sample.mp4"; + private static final String FILE_WITH_SUBTITLES = "mkv/sample_with_srt.mkv"; + private static final String FILE_WITH_SEF_SLOW_MOTION = "mp4/sample_sef_slow_motion.mp4"; + private static final String FILE_WITH_ALL_SAMPLE_FORMATS_UNSUPPORTED = "mp4/sample_ac3.mp4"; + private static final String FILE_UNKNOWN_DURATION = "mp4/sample_fragmented.mp4"; + public static final String DUMP_FILE_OUTPUT_DIRECTORY = "transformerdumps"; + public static final String DUMP_FILE_EXTENSION = "dump"; + + private Context context; + private String outputPath; + private TestMuxer testMuxer; + private AutoAdvancingFakeClock clock; + private ProgressHolder progressHolder; + + @Before + public void setUp() throws Exception { + context = ApplicationProvider.getApplicationContext(); + outputPath = Util.createTempFile(context, "TransformerTest").getPath(); + clock = new AutoAdvancingFakeClock(); + progressHolder = new ProgressHolder(); + createEncodersAndDecoders(); + } + + @After + public void tearDown() throws Exception { + Files.delete(Paths.get(outputPath)); + removeEncodersAndDecoders(); + } + + @Test + public void startTransformation_videoOnly_completesSuccessfully() throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_VIDEO_ONLY); + + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput(context, testMuxer, getDumpFileName(FILE_VIDEO_ONLY)); + } + + @Test + public void startTransformation_audioOnly_completesSuccessfully() throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_AUDIO_ONLY); + + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput(context, testMuxer, getDumpFileName(FILE_AUDIO_ONLY)); + } + + @Test + public void startTransformation_audioAndVideo_completesSuccessfully() throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_AUDIO_VIDEO); + + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput(context, testMuxer, getDumpFileName(FILE_AUDIO_VIDEO)); + } + + @Test + public void startTransformation_withSubtitles_completesSuccessfully() throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_WITH_SUBTITLES); + + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput(context, testMuxer, getDumpFileName(FILE_WITH_SUBTITLES)); + } + + @Test + public void startTransformation_successiveTransformations_completesSuccessfully() + throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_VIDEO_ONLY); + + // Transform first media item. + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + Files.delete(Paths.get(outputPath)); + + // Transformer.startTransformation() will create a new SimpleExoPlayer instance. Reset the + // clock's handler so that the clock advances with the new SimpleExoPlayer instance. + clock.resetHandler(); + // Transform second media item. + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput(context, testMuxer, getDumpFileName(FILE_VIDEO_ONLY)); + } + + @Test + public void startTransformation_concurrentTransformations_throwsError() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_VIDEO_ONLY); + + transformer.startTransformation(mediaItem, outputPath); + + assertThrows( + IllegalStateException.class, () -> transformer.startTransformation(mediaItem, outputPath)); + } + + @Test + public void startTransformation_removeAudio_completesSuccessfully() throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setRemoveAudio(true) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_AUDIO_VIDEO); + + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput( + context, testMuxer, getDumpFileName(FILE_AUDIO_VIDEO + ".noaudio")); + } + + @Test + public void startTransformation_removeVideo_completesSuccessfully() throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setRemoveVideo(true) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_AUDIO_VIDEO); + + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput( + context, testMuxer, getDumpFileName(FILE_AUDIO_VIDEO + ".novideo")); + } + + @Test + public void startTransformation_flattenForSlowMotion_completesSuccessfully() throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setFlattenForSlowMotion(true) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_WITH_SEF_SLOW_MOTION); + + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput(context, testMuxer, getDumpFileName(FILE_WITH_SEF_SLOW_MOTION)); + } + + @Test + public void startTransformation_withPlayerError_completesWithError() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri("asset:///non-existing-path.mp4"); + + transformer.startTransformation(mediaItem, outputPath); + Exception exception = TransformerTestRunner.runUntilError(transformer); + + assertThat(exception).isInstanceOf(ExoPlaybackException.class); + assertThat(exception).hasCauseThat().isInstanceOf(IOException.class); + } + + @Test + public void startTransformation_withAllSampleFormatsUnsupported_completesWithError() + throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_WITH_ALL_SAMPLE_FORMATS_UNSUPPORTED); + + transformer.startTransformation(mediaItem, outputPath); + Exception exception = TransformerTestRunner.runUntilError(transformer); + + assertThat(exception).isInstanceOf(IllegalStateException.class); + } + + @Test + public void startTransformation_afterCancellation_completesSuccessfully() throws Exception { + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_VIDEO_ONLY); + + transformer.startTransformation(mediaItem, outputPath); + transformer.cancel(); + Files.delete(Paths.get(outputPath)); + // Transformer.startTransformation() will create a new SimpleExoPlayer instance. Reset the + // clock's handler so that the clock advances with the new SimpleExoPlayer instance. + clock.resetHandler(); + // This would throw if the previous transformation had not been cancelled. + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + + DumpFileAsserts.assertOutput(context, testMuxer, getDumpFileName(FILE_VIDEO_ONLY)); + } + + @Test + public void startTransformation_fromSpecifiedThread_completesSuccessfully() throws Exception { + HandlerThread anotherThread = new HandlerThread("AnotherThread"); + anotherThread.start(); + Looper looper = anotherThread.getLooper(); + Transformer transformer = + new Transformer.Builder() + .setContext(context) + .setLooper(looper) + .setClock(clock) + .setMuxerFactory(new TestMuxerFactory()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_AUDIO_ONLY); + AtomicReference exception = new AtomicReference<>(); + CountDownLatch countDownLatch = new CountDownLatch(1); + + new Handler(looper) + .post( + () -> { + try { + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + } catch (Exception e) { + exception.set(e); + } finally { + countDownLatch.countDown(); + } + }); + countDownLatch.await(); + + assertThat(exception.get()).isNull(); + DumpFileAsserts.assertOutput(context, testMuxer, getDumpFileName(FILE_AUDIO_ONLY)); + } + + @Test + public void startTransformation_fromWrongThread_throwsError() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_AUDIO_ONLY); + HandlerThread anotherThread = new HandlerThread("AnotherThread"); + AtomicReference illegalStateException = new AtomicReference<>(); + CountDownLatch countDownLatch = new CountDownLatch(1); + + anotherThread.start(); + new Handler(anotherThread.getLooper()) + .post( + () -> { + try { + transformer.startTransformation(mediaItem, outputPath); + } catch (IOException e) { + // Do nothing. + } catch (IllegalStateException e) { + illegalStateException.set(e); + } finally { + countDownLatch.countDown(); + } + }); + countDownLatch.await(); + + assertThat(illegalStateException.get()).isNotNull(); + } + + @Test + public void getProgress_knownDuration_returnsConsistentStates() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_VIDEO_ONLY); + AtomicInteger previousProgressState = + new AtomicInteger(PROGRESS_STATE_WAITING_FOR_AVAILABILITY); + AtomicBoolean foundInconsistentState = new AtomicBoolean(); + Handler progressHandler = + new Handler(Looper.myLooper()) { + @Override + public void handleMessage(Message msg) { + @Transformer.ProgressState int progressState = transformer.getProgress(progressHolder); + if (progressState == PROGRESS_STATE_UNAVAILABLE) { + foundInconsistentState.set(true); + return; + } + switch (previousProgressState.get()) { + case PROGRESS_STATE_WAITING_FOR_AVAILABILITY: + break; + case PROGRESS_STATE_AVAILABLE: + if (progressState == PROGRESS_STATE_WAITING_FOR_AVAILABILITY) { + foundInconsistentState.set(true); + return; + } + break; + case PROGRESS_STATE_NO_TRANSFORMATION: + if (progressState != PROGRESS_STATE_NO_TRANSFORMATION) { + foundInconsistentState.set(true); + return; + } + break; + default: + throw new IllegalStateException(); + } + previousProgressState.set(progressState); + sendEmptyMessage(0); + } + }; + + transformer.startTransformation(mediaItem, outputPath); + progressHandler.sendEmptyMessage(0); + TransformerTestRunner.runUntilCompleted(transformer); + + assertThat(foundInconsistentState.get()).isFalse(); + } + + @Test + public void getProgress_knownDuration_givesIncreasingPercentages() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_VIDEO_ONLY); + List progresses = new ArrayList<>(); + Handler progressHandler = + new Handler(Looper.myLooper()) { + @Override + public void handleMessage(Message msg) { + @Transformer.ProgressState int progressState = transformer.getProgress(progressHolder); + if (progressState == PROGRESS_STATE_NO_TRANSFORMATION) { + return; + } + if (progressState != PROGRESS_STATE_WAITING_FOR_AVAILABILITY + && (progresses.isEmpty() + || Iterables.getLast(progresses) != progressHolder.progress)) { + progresses.add(progressHolder.progress); + } + sendEmptyMessage(0); + } + }; + + transformer.startTransformation(mediaItem, outputPath); + progressHandler.sendEmptyMessage(0); + TransformerTestRunner.runUntilCompleted(transformer); + + assertThat(progresses).isInOrder(); + if (!progresses.isEmpty()) { + // The progress list could be empty if the transformation ends before any progress can be + // retrieved. + assertThat(progresses.get(0)).isAtLeast(0); + assertThat(Iterables.getLast(progresses)).isLessThan(100); + } + } + + @Test + public void getProgress_noCurrentTransformation_returnsNoTransformation() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_VIDEO_ONLY); + + @Transformer.ProgressState int stateBeforeTransform = transformer.getProgress(progressHolder); + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + @Transformer.ProgressState int stateAfterTransform = transformer.getProgress(progressHolder); + + assertThat(stateBeforeTransform).isEqualTo(Transformer.PROGRESS_STATE_NO_TRANSFORMATION); + assertThat(stateAfterTransform).isEqualTo(Transformer.PROGRESS_STATE_NO_TRANSFORMATION); + } + + @Test + public void getProgress_unknownDuration_returnsConsistentStates() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_UNKNOWN_DURATION); + AtomicInteger previousProgressState = + new AtomicInteger(PROGRESS_STATE_WAITING_FOR_AVAILABILITY); + AtomicBoolean foundInconsistentState = new AtomicBoolean(); + Handler progressHandler = + new Handler(Looper.myLooper()) { + @Override + public void handleMessage(Message msg) { + @Transformer.ProgressState int progressState = transformer.getProgress(progressHolder); + switch (previousProgressState.get()) { + case PROGRESS_STATE_WAITING_FOR_AVAILABILITY: + break; + case PROGRESS_STATE_UNAVAILABLE: + case PROGRESS_STATE_AVAILABLE: // See [Internal: b/176145097]. + if (progressState == PROGRESS_STATE_WAITING_FOR_AVAILABILITY) { + foundInconsistentState.set(true); + return; + } + break; + case PROGRESS_STATE_NO_TRANSFORMATION: + if (progressState != PROGRESS_STATE_NO_TRANSFORMATION) { + foundInconsistentState.set(true); + return; + } + break; + default: + throw new IllegalStateException(); + } + previousProgressState.set(progressState); + sendEmptyMessage(0); + } + }; + + transformer.startTransformation(mediaItem, outputPath); + progressHandler.sendEmptyMessage(0); + TransformerTestRunner.runUntilCompleted(transformer); + + assertThat(foundInconsistentState.get()).isFalse(); + } + + @Test + public void getProgress_fromWrongThread_throwsError() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + HandlerThread anotherThread = new HandlerThread("AnotherThread"); + AtomicReference illegalStateException = new AtomicReference<>(); + CountDownLatch countDownLatch = new CountDownLatch(1); + + anotherThread.start(); + new Handler(anotherThread.getLooper()) + .post( + () -> { + try { + transformer.getProgress(progressHolder); + } catch (IllegalStateException e) { + illegalStateException.set(e); + } finally { + countDownLatch.countDown(); + } + }); + countDownLatch.await(); + + assertThat(illegalStateException.get()).isNotNull(); + } + + @Test + public void cancel_afterCompletion_doesNotThrow() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + MediaItem mediaItem = MediaItem.fromUri(URI_PREFIX + FILE_VIDEO_ONLY); + + transformer.startTransformation(mediaItem, outputPath); + TransformerTestRunner.runUntilCompleted(transformer); + transformer.cancel(); + } + + @Test + public void cancel_fromWrongThread_throwsError() throws Exception { + Transformer transformer = new Transformer.Builder().setContext(context).setClock(clock).build(); + HandlerThread anotherThread = new HandlerThread("AnotherThread"); + AtomicReference illegalStateException = new AtomicReference<>(); + CountDownLatch countDownLatch = new CountDownLatch(1); + + anotherThread.start(); + new Handler(anotherThread.getLooper()) + .post( + () -> { + try { + transformer.cancel(); + } catch (IllegalStateException e) { + illegalStateException.set(e); + } finally { + countDownLatch.countDown(); + } + }); + countDownLatch.await(); + + assertThat(illegalStateException.get()).isNotNull(); + } + + private static void createEncodersAndDecoders() { + ShadowMediaCodec.CodecConfig codecConfig = + new ShadowMediaCodec.CodecConfig( + /* inputBufferSize= */ 10_000, + /* outputBufferSize= */ 10_000, + /* codec= */ (in, out) -> out.put(in)); + ShadowMediaCodec.addDecoder(MimeTypes.AUDIO_AAC, codecConfig); + ShadowMediaCodec.addDecoder(MimeTypes.AUDIO_AMR_NB, codecConfig); + ShadowMediaCodec.addEncoder(MimeTypes.AUDIO_AAC, codecConfig); + } + + private static void removeEncodersAndDecoders() { + ShadowMediaCodec.clearCodecs(); + } + + private static String getDumpFileName(String originalFileName) { + return DUMP_FILE_OUTPUT_DIRECTORY + '/' + originalFileName + '.' + DUMP_FILE_EXTENSION; + } + + private final class TestMuxerFactory implements Muxer.Factory { + @Override + public Muxer create(String path, String outputMimeType) throws IOException { + testMuxer = new TestMuxer(path, outputMimeType); + return testMuxer; + } + + @Override + public Muxer create(ParcelFileDescriptor parcelFileDescriptor, String outputMimeType) + throws IOException { + testMuxer = new TestMuxer("FD:" + parcelFileDescriptor.getFd(), outputMimeType); + return testMuxer; + } + + @Override + public boolean supportsOutputMimeType(String mimeType) { + return true; + } + } +} diff --git a/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerTestRunner.java b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerTestRunner.java new file mode 100644 index 00000000000..1eacbc46e01 --- /dev/null +++ b/library/transformer/src/test/java/com/google/android/exoplayer2/transformer/TransformerTestRunner.java @@ -0,0 +1,93 @@ +/* + * Copyright 2020 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.google.android.exoplayer2.transformer; + +import static com.google.android.exoplayer2.robolectric.RobolectricUtil.runLooperUntil; + +import androidx.annotation.Nullable; +import com.google.android.exoplayer2.MediaItem; +import com.google.android.exoplayer2.robolectric.RobolectricUtil; +import java.util.concurrent.TimeoutException; + +/** Helper class to run a {@link Transformer} test. */ +public final class TransformerTestRunner { + + private TransformerTestRunner() {} + + /** + * Runs tasks of the {@link Transformer#getApplicationLooper() transformer Looper} until the + * current {@link Transformer transformation} completes. + * + * @param transformer The {@link Transformer}. + * @throws TimeoutException If the {@link RobolectricUtil#DEFAULT_TIMEOUT_MS default timeout} is + * exceeded. + * @throws IllegalStateException If the method is not called from the main thread, or if the + * transformation completes with error. + */ + public static void runUntilCompleted(Transformer transformer) throws TimeoutException { + @Nullable Exception exception = runUntilListenerCalled(transformer); + if (exception != null) { + throw new IllegalStateException(exception); + } + } + + /** + * Runs tasks of the {@link Transformer#getApplicationLooper() transformer Looper} until a {@link + * Transformer} error occurs. + * + * @param transformer The {@link Transformer}. + * @return The raised exception. + * @throws TimeoutException If the {@link RobolectricUtil#DEFAULT_TIMEOUT_MS default timeout} is + * exceeded. + * @throws IllegalStateException If the method is not called from the main thread, or if the + * transformation completes without error. + */ + public static Exception runUntilError(Transformer transformer) throws TimeoutException { + @Nullable Exception exception = runUntilListenerCalled(transformer); + if (exception == null) { + throw new IllegalStateException("The transformation completed without error."); + } + return exception; + } + + @Nullable + private static Exception runUntilListenerCalled(Transformer transformer) throws TimeoutException { + TransformationResult transformationResult = new TransformationResult(); + Transformer.Listener listener = + new Transformer.Listener() { + @Override + public void onTransformationCompleted(MediaItem inputMediaItem) { + transformationResult.isCompleted = true; + } + + @Override + public void onTransformationError(MediaItem inputMediaItem, Exception exception) { + transformationResult.exception = exception; + } + }; + transformer.setListener(listener); + runLooperUntil( + transformer.getApplicationLooper(), + () -> transformationResult.isCompleted || transformationResult.exception != null); + return transformationResult.exception; + } + + private static class TransformationResult { + public boolean isCompleted; + @Nullable public Exception exception; + } +} diff --git a/library/ui/proguard-rules.txt b/library/ui/proguard-rules.txt index ad7c139ea8e..f18183637b0 100644 --- a/library/ui/proguard-rules.txt +++ b/library/ui/proguard-rules.txt @@ -11,6 +11,17 @@ public androidx.appcompat.app.AlertDialog$Builder setNegativeButton(int, android.content.DialogInterface$OnClickListener); public androidx.appcompat.app.AlertDialog create(); } +# Equivalent methods needed when the library is de-jetified. +-dontnote android.support.v7.app.AlertDialog.Builder +-keepclassmembers class android.support.v7.app.AlertDialog$Builder { + (android.content.Context, int); + public android.content.Context getContext(); + public android.support.v7.app.AlertDialog$Builder setTitle(java.lang.CharSequence); + public android.support.v7.app.AlertDialog$Builder setView(android.view.View); + public android.support.v7.app.AlertDialog$Builder setPositiveButton(int, android.content.DialogInterface$OnClickListener); + public android.support.v7.app.AlertDialog$Builder setNegativeButton(int, android.content.DialogInterface$OnClickListener); + public android.support.v7.app.AlertDialog create(); +} # Don't warn about checkerframework and Kotlin annotations -dontwarn org.checkerframework.** diff --git a/library/ui/src/main/java/com/google/android/exoplayer2/ui/DefaultTimeBar.java b/library/ui/src/main/java/com/google/android/exoplayer2/ui/DefaultTimeBar.java index 4e96d39e7c1..ce041cba4eb 100644 --- a/library/ui/src/main/java/com/google/android/exoplayer2/ui/DefaultTimeBar.java +++ b/library/ui/src/main/java/com/google/android/exoplayer2/ui/DefaultTimeBar.java @@ -157,8 +157,6 @@ public class DefaultTimeBar extends View implements TimeBar { public static final int BAR_GRAVITY_CENTER = 0; /** Vertical gravity for progress bar to be located at the bottom in the view. */ public static final int BAR_GRAVITY_BOTTOM = 1; - /** Vertical gravity for progress bar to be located at the top in the view. */ - public static final int BAR_GRAVITY_TOP = 2; // LINT.ThenChange(../../../../../../../../../ui/src/main/res/values/attrs.xml) /** The threshold in dps above the bar at which touch events trigger fine scrub mode. */ @@ -216,6 +214,7 @@ public class DefaultTimeBar extends View implements TimeBar { private ValueAnimator scrubberScalingAnimator; private float scrubberScale; + private boolean scrubberPaddingDisabled; private boolean scrubbing; private long scrubPosition; private long duration; @@ -370,7 +369,12 @@ public DefaultTimeBar( /** Shows the scrubber handle. */ public void showScrubber() { - showScrubber(/* showAnimationDurationMs= */ 0); + if (scrubberScalingAnimator.isStarted()) { + scrubberScalingAnimator.cancel(); + } + scrubberPaddingDisabled = false; + scrubberScale = 1; + invalidate(seekBounds); } /** @@ -382,14 +386,20 @@ public void showScrubber(long showAnimationDurationMs) { if (scrubberScalingAnimator.isStarted()) { scrubberScalingAnimator.cancel(); } + scrubberPaddingDisabled = false; scrubberScalingAnimator.setFloatValues(scrubberScale, SHOWN_SCRUBBER_SCALE); scrubberScalingAnimator.setDuration(showAnimationDurationMs); scrubberScalingAnimator.start(); } /** Hides the scrubber handle. */ - public void hideScrubber() { - hideScrubber(/* hideAnimationDurationMs= */ 0); + public void hideScrubber(boolean disableScrubberPadding) { + if (scrubberScalingAnimator.isStarted()) { + scrubberScalingAnimator.cancel(); + } + scrubberPaddingDisabled = disableScrubberPadding; + scrubberScale = 0; + invalidate(seekBounds); } /** @@ -668,20 +678,25 @@ protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) { protected void onLayout(boolean changed, int left, int top, int right, int bottom) { int width = right - left; int height = bottom - top; - int barY = (height - touchTargetHeight) / 2; int seekLeft = getPaddingLeft(); int seekRight = width - getPaddingRight(); - int progressY; + int seekBoundsY; + int progressBarY; + int scrubberPadding = scrubberPaddingDisabled ? 0 : this.scrubberPadding; if (barGravity == BAR_GRAVITY_BOTTOM) { - progressY = barY + touchTargetHeight - (getPaddingBottom() + scrubberPadding + barHeight / 2); - } else if (barGravity == BAR_GRAVITY_TOP) { - progressY = barY + getPaddingTop() + scrubberPadding - barHeight / 2; + seekBoundsY = height - getPaddingBottom() - touchTargetHeight; + progressBarY = + height - getPaddingBottom() - barHeight - Math.max(scrubberPadding - (barHeight / 2), 0); } else { - progressY = barY + (touchTargetHeight - barHeight) / 2; - } - seekBounds.set(seekLeft, barY, seekRight, barY + touchTargetHeight); - progressBar.set(seekBounds.left + scrubberPadding, progressY, - seekBounds.right - scrubberPadding, progressY + barHeight); + seekBoundsY = (height - touchTargetHeight) / 2; + progressBarY = (height - barHeight) / 2; + } + seekBounds.set(seekLeft, seekBoundsY, seekRight, seekBoundsY + touchTargetHeight); + progressBar.set( + seekBounds.left + scrubberPadding, + progressBarY, + seekBounds.right - scrubberPadding, + progressBarY + barHeight); if (Util.SDK_INT >= 29) { setSystemGestureExclusionRectsV29(width, height); } diff --git a/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerControlView.java b/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerControlView.java index 1ae7812bd46..7f9259d6780 100644 --- a/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerControlView.java +++ b/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerControlView.java @@ -15,6 +15,14 @@ */ package com.google.android.exoplayer2.ui; +import static com.google.android.exoplayer2.Player.EVENT_IS_PLAYING_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAYBACK_STATE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAY_WHEN_READY_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_POSITION_DISCONTINUITY; +import static com.google.android.exoplayer2.Player.EVENT_REPEAT_MODE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_TIMELINE_CHANGED; + import android.annotation.SuppressLint; import android.content.Context; import android.content.res.Resources; @@ -38,6 +46,7 @@ import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.PlaybackPreparer; import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.Player.Events; import com.google.android.exoplayer2.Player.State; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.util.Assertions; @@ -506,12 +515,6 @@ public PlayerControlView( resources.getString(R.string.exo_controls_shuffle_off_description); } - @SuppressWarnings("ResourceType") - private static @RepeatModeUtil.RepeatToggleModes int getRepeatToggleModes( - TypedArray a, @RepeatModeUtil.RepeatToggleModes int repeatToggleModes) { - return a.getInt(R.styleable.PlayerControlView_repeat_toggle_modes, repeatToggleModes); - } - /** * Returns the {@link Player} currently being controlled by this view, or null if no player is * set. @@ -1309,6 +1312,12 @@ private static boolean canShowMultiWindowTimeBar(Timeline timeline, Timeline.Win return true; } + @SuppressWarnings("ResourceType") + private static @RepeatModeUtil.RepeatToggleModes int getRepeatToggleModes( + TypedArray a, @RepeatModeUtil.RepeatToggleModes int defaultValue) { + return a.getInt(R.styleable.PlayerControlView_repeat_toggle_modes, defaultValue); + } + private final class ComponentListener implements Player.EventListener, TimeBar.OnScrubListener, OnClickListener { @@ -1336,45 +1345,30 @@ public void onScrubStop(TimeBar timeBar, long position, boolean canceled) { } @Override - public void onPlaybackStateChanged(@Player.State int playbackState) { - updatePlayPauseButton(); - updateProgress(); - } - - @Override - public void onPlayWhenReadyChanged( - boolean playWhenReady, @Player.PlayWhenReadyChangeReason int reason) { - updatePlayPauseButton(); - updateProgress(); - } - - @Override - public void onIsPlayingChanged(boolean isPlaying) { - updateProgress(); - } - - @Override - public void onRepeatModeChanged(int repeatMode) { - updateRepeatModeButton(); - updateNavigation(); - } - - @Override - public void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) { - updateShuffleButton(); - updateNavigation(); - } - - @Override - public void onPositionDiscontinuity(@Player.DiscontinuityReason int reason) { - updateNavigation(); - updateTimeline(); - } - - @Override - public void onTimelineChanged(Timeline timeline, @Player.TimelineChangeReason int reason) { - updateNavigation(); - updateTimeline(); + public void onEvents(Player player, Events events) { + if (events.containsAny(EVENT_PLAYBACK_STATE_CHANGED, EVENT_PLAY_WHEN_READY_CHANGED)) { + updatePlayPauseButton(); + } + if (events.containsAny( + EVENT_PLAYBACK_STATE_CHANGED, EVENT_PLAY_WHEN_READY_CHANGED, EVENT_IS_PLAYING_CHANGED)) { + updateProgress(); + } + if (events.contains(EVENT_REPEAT_MODE_CHANGED)) { + updateRepeatModeButton(); + } + if (events.contains(EVENT_SHUFFLE_MODE_ENABLED_CHANGED)) { + updateShuffleButton(); + } + if (events.containsAny( + EVENT_REPEAT_MODE_CHANGED, + EVENT_SHUFFLE_MODE_ENABLED_CHANGED, + EVENT_POSITION_DISCONTINUITY, + EVENT_TIMELINE_CHANGED)) { + updateNavigation(); + } + if (events.containsAny(EVENT_POSITION_DISCONTINUITY, EVENT_TIMELINE_CHANGED)) { + updateTimeline(); + } } @Override diff --git a/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerNotificationManager.java b/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerNotificationManager.java index 18862c4103f..e5b29b6a858 100644 --- a/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerNotificationManager.java +++ b/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerNotificationManager.java @@ -15,6 +15,15 @@ */ package com.google.android.exoplayer2.ui; +import static com.google.android.exoplayer2.Player.EVENT_IS_PLAYING_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAYBACK_PARAMETERS_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAYBACK_STATE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAY_WHEN_READY_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_POSITION_DISCONTINUITY; +import static com.google.android.exoplayer2.Player.EVENT_REPEAT_MODE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_TIMELINE_CHANGED; + import android.app.Notification; import android.app.NotificationChannel; import android.app.PendingIntent; @@ -38,7 +47,6 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ControlDispatcher; import com.google.android.exoplayer2.DefaultControlDispatcher; -import com.google.android.exoplayer2.PlaybackParameters; import com.google.android.exoplayer2.PlaybackPreparer; import com.google.android.exoplayer2.Player; import com.google.android.exoplayer2.Timeline; @@ -563,7 +571,7 @@ public PlayerNotificationManager( notificationId, mediaDescriptionAdapter, notificationListener, - /* customActionReceiver*/ null); + /* customActionReceiver= */ null); } /** @@ -591,7 +599,7 @@ public PlayerNotificationManager( channelId, notificationId, mediaDescriptionAdapter, - /* notificationListener */ null, + /* notificationListener= */ null, customActionReceiver); } @@ -1396,44 +1404,18 @@ private static void setLargeIcon(NotificationCompat.Builder builder, @Nullable B private class PlayerListener implements Player.EventListener { @Override - public void onPlaybackStateChanged(@Player.State int playbackState) { - postStartOrUpdateNotification(); - } - - @Override - public void onPlayWhenReadyChanged( - boolean playWhenReady, @Player.PlayWhenReadyChangeReason int reason) { - postStartOrUpdateNotification(); - } - - @Override - public void onIsPlayingChanged(boolean isPlaying) { - postStartOrUpdateNotification(); - } - - @Override - public void onTimelineChanged(Timeline timeline, int reason) { - postStartOrUpdateNotification(); - } - - @Override - public void onPlaybackParametersChanged(PlaybackParameters playbackParameters) { - postStartOrUpdateNotification(); - } - - @Override - public void onPositionDiscontinuity(int reason) { - postStartOrUpdateNotification(); - } - - @Override - public void onRepeatModeChanged(@Player.RepeatMode int repeatMode) { - postStartOrUpdateNotification(); - } - - @Override - public void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) { - postStartOrUpdateNotification(); + public void onEvents(Player player, Player.Events events) { + if (events.containsAny( + EVENT_PLAYBACK_STATE_CHANGED, + EVENT_PLAY_WHEN_READY_CHANGED, + EVENT_IS_PLAYING_CHANGED, + EVENT_TIMELINE_CHANGED, + EVENT_PLAYBACK_PARAMETERS_CHANGED, + EVENT_POSITION_DISCONTINUITY, + EVENT_REPEAT_MODE_CHANGED, + EVENT_SHUFFLE_MODE_ENABLED_CHANGED)) { + postStartOrUpdateNotification(); + } } } diff --git a/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerView.java b/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerView.java index 726d6cf6f01..b913d322110 100644 --- a/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerView.java +++ b/library/ui/src/main/java/com/google/android/exoplayer2/ui/PlayerView.java @@ -57,7 +57,6 @@ import com.google.android.exoplayer2.source.ads.AdsLoader; import com.google.android.exoplayer2.text.Cue; import com.google.android.exoplayer2.text.TextOutput; -import com.google.android.exoplayer2.trackselection.TrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; import com.google.android.exoplayer2.ui.AspectRatioFrameLayout.ResizeMode; import com.google.android.exoplayer2.ui.spherical.SingleTapListener; @@ -573,8 +572,6 @@ public void setPlayer(@Nullable Player player) { oldVideoComponent.clearVideoTextureView((TextureView) surfaceView); } else if (surfaceView instanceof SphericalGLSurfaceView) { ((SphericalGLSurfaceView) surfaceView).setVideoComponent(null); - } else if (surfaceView instanceof VideoDecoderGLSurfaceView) { - oldVideoComponent.setVideoDecoderOutputBufferRenderer(null); } else if (surfaceView instanceof SurfaceView) { oldVideoComponent.clearVideoSurfaceView((SurfaceView) surfaceView); } @@ -601,9 +598,6 @@ public void setPlayer(@Nullable Player player) { newVideoComponent.setVideoTextureView((TextureView) surfaceView); } else if (surfaceView instanceof SphericalGLSurfaceView) { ((SphericalGLSurfaceView) surfaceView).setVideoComponent(newVideoComponent); - } else if (surfaceView instanceof VideoDecoderGLSurfaceView) { - newVideoComponent.setVideoDecoderOutputBufferRenderer( - ((VideoDecoderGLSurfaceView) surfaceView).getVideoDecoderOutputBufferRenderer()); } else if (surfaceView instanceof SurfaceView) { newVideoComponent.setVideoSurfaceView((SurfaceView) surfaceView); } @@ -672,19 +666,6 @@ public Drawable getDefaultArtwork() { return defaultArtwork; } - /** - * Sets the default artwork to display if {@code useArtwork} is {@code true} and no artwork is - * present in the media. - * - * @param defaultArtwork the default artwork to display. - * @deprecated use (@link {@link #setDefaultArtwork(Drawable)} instead. - */ - @Deprecated - public void setDefaultArtwork(@Nullable Bitmap defaultArtwork) { - setDefaultArtwork( - defaultArtwork == null ? null : new BitmapDrawable(getResources(), defaultArtwork)); - } - /** * Sets the default artwork to display if {@code useArtwork} is {@code true} and no artwork is * present in the media. @@ -738,9 +719,8 @@ public void setShutterBackgroundColor(int color) { /** * Sets whether the currently displayed video frame or media artwork is kept visible when the * player is reset. A player reset is defined to mean the player being re-prepared with different - * media, the player transitioning to unprepared media, {@link Player#stop(boolean)} being called - * with {@code reset=true}, or the player being replaced or cleared by calling {@link - * #setPlayer(Player)}. + * media, the player transitioning to unprepared media or an empty list of media items, or the + * player being replaced or cleared by calling {@link #setPlayer(Player)}. * *

    If enabled, the currently displayed video frame or media artwork will be kept visible until * the player set on the view has been successfully prepared with new media and loaded enough of @@ -778,18 +758,6 @@ public void setUseSensorRotation(boolean useSensorRotation) { } } - /** - * Sets whether a buffering spinner is displayed when the player is in the buffering state. The - * buffering spinner is not displayed by default. - * - * @deprecated Use {@link #setShowBuffering(int)} - * @param showBuffering Whether the buffering icon is displayed - */ - @Deprecated - public void setShowBuffering(boolean showBuffering) { - setShowBuffering(showBuffering ? SHOW_BUFFERING_WHEN_PLAYING : SHOW_BUFFERING_NEVER); - } - /** * Sets whether a buffering spinner is displayed when the player is in the buffering state. The * buffering spinner is not displayed by default. @@ -1378,15 +1346,9 @@ private void updateForCurrentTrackSelections(boolean isNewPlayer) { closeShutter(); // Display artwork if enabled and available, else hide it. if (useArtwork()) { - for (int i = 0; i < selections.length; i++) { - @Nullable TrackSelection selection = selections.get(i); - if (selection != null) { - for (int j = 0; j < selection.length(); j++) { - @Nullable Metadata metadata = selection.getFormat(j).metadata; - if (metadata != null && setArtworkFromMetadata(metadata)) { - return; - } - } + for (Metadata metadata : player.getCurrentStaticMetadata()) { + if (setArtworkFromMetadata(metadata)) { + return; } } if (setDrawableArtwork(defaultArtwork)) { diff --git a/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerControlView.java b/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerControlView.java index ab09454c95c..464bb5a4778 100644 --- a/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerControlView.java +++ b/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerControlView.java @@ -15,6 +15,15 @@ */ package com.google.android.exoplayer2.ui; +import static com.google.android.exoplayer2.Player.EVENT_IS_PLAYING_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAYBACK_PARAMETERS_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAYBACK_STATE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_PLAY_WHEN_READY_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_POSITION_DISCONTINUITY; +import static com.google.android.exoplayer2.Player.EVENT_REPEAT_MODE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_TIMELINE_CHANGED; +import static com.google.android.exoplayer2.Player.EVENT_TRACKS_CHANGED; import static com.google.android.exoplayer2.util.Assertions.checkNotNull; import android.annotation.SuppressLint; @@ -42,13 +51,13 @@ import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.ControlDispatcher; import com.google.android.exoplayer2.DefaultControlDispatcher; +import com.google.android.exoplayer2.ExoPlayer; import com.google.android.exoplayer2.ExoPlayerLibraryInfo; import com.google.android.exoplayer2.Format; -import com.google.android.exoplayer2.PlaybackParameters; import com.google.android.exoplayer2.PlaybackPreparer; import com.google.android.exoplayer2.Player; +import com.google.android.exoplayer2.Player.Events; import com.google.android.exoplayer2.Player.State; -import com.google.android.exoplayer2.RendererCapabilities; import com.google.android.exoplayer2.Timeline; import com.google.android.exoplayer2.source.TrackGroup; import com.google.android.exoplayer2.source.TrackGroupArray; @@ -58,6 +67,7 @@ import com.google.android.exoplayer2.trackselection.MappingTrackSelector.MappedTrackInfo; import com.google.android.exoplayer2.trackselection.TrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; +import com.google.android.exoplayer2.trackselection.TrackSelector; import com.google.android.exoplayer2.util.Assertions; import com.google.android.exoplayer2.util.RepeatModeUtil; import com.google.android.exoplayer2.util.Util; @@ -260,6 +270,10 @@ *

      *
    • Type: {@link ImageView} *
    + *
  • {@code exo_minimal_fullscreen} - The fullscreen button in minimal mode. + *
      + *
    • Type: {@link ImageView} + *
    *
  • {@code exo_position} - Text view displaying the current playback position. *
      *
    • Type: {@link TextView} @@ -354,7 +368,6 @@ public interface OnFullScreenModeChangedListener { private static final int SETTINGS_PLAYBACK_SPEED_POSITION = 0; private static final int SETTINGS_AUDIO_TRACK_SELECTION_POSITION = 1; - private static final int UNDEFINED_POSITION = -1; private final ComponentListener componentListener; private final CopyOnWriteArrayList visibilityListeners; @@ -423,15 +436,13 @@ public interface OnFullScreenModeChangedListener { private StyledPlayerControlViewLayoutManager controlViewLayoutManager; private Resources resources; - // Relating to Settings List View private int selectedMainSettingsPosition; private RecyclerView settingsView; private SettingsAdapter settingsAdapter; private SubSettingsAdapter subSettingsAdapter; private PopupWindow settingsWindow; - private List playbackSpeedTextList; - private List playbackSpeedMultBy100List; - private int customPlaybackSpeedIndex; + private String[] playbackSpeedTexts; + private int[] playbackSpeedsMultBy100; private int selectedPlaybackSpeedIndex; private boolean needToHideBars; private int settingsWindowMargin; @@ -442,9 +453,9 @@ public interface OnFullScreenModeChangedListener { // TODO(insun): Add setTrackNameProvider to use customized track name provider. private TrackNameProvider trackNameProvider; - // Relating to Bottom Bar Right View @Nullable private ImageView subtitleButton; @Nullable private ImageView fullScreenButton; + @Nullable private ImageView minimalFullScreenButton; @Nullable private View settingsButton; public StyledPlayerControlView(Context context) { @@ -546,20 +557,19 @@ public StyledPlayerControlView( controlDispatcher = new DefaultControlDispatcher(fastForwardMs, rewindMs); updateProgressAction = this::updateProgress; - // Relating to Bottom Bar Left View durationView = findViewById(R.id.exo_duration); positionView = findViewById(R.id.exo_position); - // Relating to Bottom Bar Right View subtitleButton = findViewById(R.id.exo_subtitle); if (subtitleButton != null) { subtitleButton.setOnClickListener(componentListener); } + fullScreenButton = findViewById(R.id.exo_fullscreen); - if (fullScreenButton != null) { - fullScreenButton.setVisibility(GONE); - fullScreenButton.setOnClickListener(this::onFullScreenButtonClicked); - } + initializeFullScreenButton(fullScreenButton, this::onFullScreenButtonClicked); + minimalFullScreenButton = findViewById(R.id.exo_minimal_fullscreen); + initializeFullScreenButton(minimalFullScreenButton, this::onFullScreenButtonClicked); + settingsButton = findViewById(R.id.exo_settings); if (settingsButton != null) { settingsButton.setOnClickListener(componentListener); @@ -642,7 +652,6 @@ public StyledPlayerControlView( controlViewLayoutManager = new StyledPlayerControlViewLayoutManager(this); controlViewLayoutManager.setAnimationEnabled(animationEnabled); - // Related to Settings List View String[] settingTexts = new String[2]; Drawable[] settingIcons = new Drawable[2]; settingTexts[SETTINGS_PLAYBACK_SPEED_POSITION] = @@ -655,19 +664,11 @@ public StyledPlayerControlView( resources.getDrawable(R.drawable.exo_styled_controls_audiotrack); settingsAdapter = new SettingsAdapter(settingTexts, settingIcons); - playbackSpeedTextList = - new ArrayList<>(Arrays.asList(resources.getStringArray(R.array.exo_playback_speeds))); - playbackSpeedMultBy100List = new ArrayList<>(); - int[] speeds = resources.getIntArray(R.array.exo_speed_multiplied_by_100); - for (int speed : speeds) { - playbackSpeedMultBy100List.add(speed); - } - selectedPlaybackSpeedIndex = playbackSpeedMultBy100List.indexOf(100); - customPlaybackSpeedIndex = UNDEFINED_POSITION; + playbackSpeedTexts = resources.getStringArray(R.array.exo_playback_speeds); + playbackSpeedsMultBy100 = resources.getIntArray(R.array.exo_speed_multiplied_by_100); settingsWindowMargin = resources.getDimensionPixelSize(R.dimen.exo_settings_offset); subSettingsAdapter = new SubSettingsAdapter(); - subSettingsAdapter.setCheckPosition(UNDEFINED_POSITION); settingsView = (RecyclerView) LayoutInflater.from(context).inflate(R.layout.exo_styled_settings_list, null); @@ -730,12 +731,6 @@ public StyledPlayerControlView( addOnLayoutChangeListener(this::onLayoutChange); } - @SuppressWarnings("ResourceType") - private static @RepeatModeUtil.RepeatToggleModes int getRepeatToggleModes( - TypedArray a, @RepeatModeUtil.RepeatToggleModes int repeatToggleModes) { - return a.getInt(R.styleable.StyledPlayerControlView_repeat_toggle_modes, repeatToggleModes); - } - /** * Returns the {@link Player} currently being controlled by this view, or null if no player is * set. @@ -766,8 +761,11 @@ public void setPlayer(@Nullable Player player) { if (player != null) { player.addListener(componentListener); } - if (player != null && player.getTrackSelector() instanceof DefaultTrackSelector) { - this.trackSelector = (DefaultTrackSelector) player.getTrackSelector(); + if (player instanceof ExoPlayer) { + TrackSelector trackSelector = ((ExoPlayer) player).getTrackSelector(); + if (trackSelector instanceof DefaultTrackSelector) { + this.trackSelector = (DefaultTrackSelector) trackSelector; + } } else { this.trackSelector = null; } @@ -1059,16 +1057,9 @@ public void setTimeBarMinUpdateInterval(int minUpdateIntervalMs) { */ public void setOnFullScreenModeChangedListener( @Nullable OnFullScreenModeChangedListener listener) { - if (fullScreenButton == null) { - return; - } - onFullScreenModeChangedListener = listener; - if (onFullScreenModeChangedListener == null) { - fullScreenButton.setVisibility(GONE); - } else { - fullScreenButton.setVisibility(VISIBLE); - } + updateFullScreenButtonVisibility(fullScreenButton, listener != null); + updateFullScreenButtonVisibility(minimalFullScreenButton, listener != null); } /** @@ -1314,7 +1305,7 @@ private void gatherTrackInfosForAdapter( for (int trackIndex = 0; trackIndex < trackGroup.length; trackIndex++) { Format format = trackGroup.getFormat(trackIndex); if (mappedTrackInfo.getTrackSupport(rendererIndex, groupIndex, trackIndex) - == RendererCapabilities.FORMAT_HANDLED) { + == C.FORMAT_HANDLED) { boolean trackIsSelected = trackSelection != null && trackSelection.indexOf(format) != C.INDEX_UNSET; tracks.add( @@ -1452,25 +1443,18 @@ private void updateSettingsPlaybackSpeedLists() { } float speed = player.getPlaybackParameters().speed; int currentSpeedMultBy100 = Math.round(speed * 100); - int indexForCurrentSpeed = playbackSpeedMultBy100List.indexOf(currentSpeedMultBy100); - if (indexForCurrentSpeed == UNDEFINED_POSITION) { - if (customPlaybackSpeedIndex != UNDEFINED_POSITION) { - playbackSpeedMultBy100List.remove(customPlaybackSpeedIndex); - playbackSpeedTextList.remove(customPlaybackSpeedIndex); - customPlaybackSpeedIndex = UNDEFINED_POSITION; + int closestMatchIndex = 0; + int closestMatchDifference = Integer.MAX_VALUE; + for (int i = 0; i < playbackSpeedsMultBy100.length; i++) { + int difference = Math.abs(currentSpeedMultBy100 - playbackSpeedsMultBy100[i]); + if (difference < closestMatchDifference) { + closestMatchIndex = i; + closestMatchDifference = difference; } - indexForCurrentSpeed = - -Collections.binarySearch(playbackSpeedMultBy100List, currentSpeedMultBy100) - 1; - String customSpeedText = - resources.getString(R.string.exo_controls_custom_playback_speed, speed); - playbackSpeedMultBy100List.add(indexForCurrentSpeed, currentSpeedMultBy100); - playbackSpeedTextList.add(indexForCurrentSpeed, customSpeedText); - customPlaybackSpeedIndex = indexForCurrentSpeed; } - - selectedPlaybackSpeedIndex = indexForCurrentSpeed; + selectedPlaybackSpeedIndex = closestMatchIndex; settingsAdapter.setSubTextAtPosition( - SETTINGS_PLAYBACK_SPEED_POSITION, playbackSpeedTextList.get(indexForCurrentSpeed)); + SETTINGS_PLAYBACK_SPEED_POSITION, playbackSpeedTexts[closestMatchIndex]); } private void updateSettingsWindowSize() { @@ -1506,7 +1490,8 @@ private void setPlaybackSpeed(float speed) { if (player == null) { return; } - player.setPlaybackParameters(new PlaybackParameters(speed)); + controlDispatcher.dispatchSetPlaybackParameters( + player, player.getPlaybackParameters().withSpeed(speed)); } /* package */ void requestPlayPauseFocus() { @@ -1557,11 +1542,23 @@ private boolean seekTo(Player player, int windowIndex, long positionMs) { } private void onFullScreenButtonClicked(View v) { - if (onFullScreenModeChangedListener == null || fullScreenButton == null) { + if (onFullScreenModeChangedListener == null) { return; } isFullScreen = !isFullScreen; + updateFullScreenButtonForState(fullScreenButton, isFullScreen); + updateFullScreenButtonForState(minimalFullScreenButton, isFullScreen); + if (onFullScreenModeChangedListener != null) { + onFullScreenModeChangedListener.onFullScreenModeChanged(isFullScreen); + } + } + + private void updateFullScreenButtonForState( + @Nullable ImageView fullScreenButton, boolean isFullScreen) { + if (fullScreenButton == null) { + return; + } if (isFullScreen) { fullScreenButton.setImageDrawable(fullScreenExitDrawable); fullScreenButton.setContentDescription(fullScreenExitContentDescription); @@ -1569,16 +1566,11 @@ private void onFullScreenButtonClicked(View v) { fullScreenButton.setImageDrawable(fullScreenEnterDrawable); fullScreenButton.setContentDescription(fullScreenEnterContentDescription); } - - if (onFullScreenModeChangedListener != null) { - onFullScreenModeChangedListener.onFullScreenModeChanged(isFullScreen); - } } private void onSettingViewClicked(int position) { if (position == SETTINGS_PLAYBACK_SPEED_POSITION) { - subSettingsAdapter.setTexts(playbackSpeedTextList); - subSettingsAdapter.setCheckPosition(selectedPlaybackSpeedIndex); + subSettingsAdapter.init(playbackSpeedTexts, selectedPlaybackSpeedIndex); selectedMainSettingsPosition = SETTINGS_PLAYBACK_SPEED_POSITION; displaySettingsWindow(subSettingsAdapter); } else if (position == SETTINGS_AUDIO_TRACK_SELECTION_POSITION) { @@ -1592,7 +1584,7 @@ private void onSettingViewClicked(int position) { private void onSubSettingViewClicked(int position) { if (selectedMainSettingsPosition == SETTINGS_PLAYBACK_SPEED_POSITION) { if (position != selectedPlaybackSpeedIndex) { - float speed = playbackSpeedMultBy100List.get(position) / 100.0f; + float speed = playbackSpeedsMultBy100[position] / 100.0f; setPlaybackSpeed(speed); } } @@ -1760,6 +1752,32 @@ private static boolean canShowMultiWindowTimeBar(Timeline timeline, Timeline.Win return true; } + private static void initializeFullScreenButton(View fullScreenButton, OnClickListener listener) { + if (fullScreenButton == null) { + return; + } + fullScreenButton.setVisibility(GONE); + fullScreenButton.setOnClickListener(listener); + } + + private static void updateFullScreenButtonVisibility( + @Nullable View fullScreenButton, boolean visible) { + if (fullScreenButton == null) { + return; + } + if (visible) { + fullScreenButton.setVisibility(VISIBLE); + } else { + fullScreenButton.setVisibility(GONE); + } + } + + @SuppressWarnings("ResourceType") + private static @RepeatModeUtil.RepeatToggleModes int getRepeatToggleModes( + TypedArray a, @RepeatModeUtil.RepeatToggleModes int defaultValue) { + return a.getInt(R.styleable.StyledPlayerControlView_repeat_toggle_modes, defaultValue); + } + private final class ComponentListener implements Player.EventListener, TimeBar.OnScrubListener, @@ -1792,55 +1810,36 @@ public void onScrubStop(TimeBar timeBar, long position, boolean canceled) { } @Override - public void onPlaybackStateChanged(@Player.State int playbackState) { - updatePlayPauseButton(); - updateProgress(); - } - - @Override - public void onPlayWhenReadyChanged( - boolean playWhenReady, @Player.PlayWhenReadyChangeReason int state) { - updatePlayPauseButton(); - updateProgress(); - } - - @Override - public void onIsPlayingChanged(boolean isPlaying) { - updateProgress(); - } - - @Override - public void onRepeatModeChanged(int repeatMode) { - updateRepeatModeButton(); - updateNavigation(); - } - - @Override - public void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) { - updateShuffleButton(); - updateNavigation(); - } - - @Override - public void onPositionDiscontinuity(@Player.DiscontinuityReason int reason) { - updateNavigation(); - updateTimeline(); - } - - @Override - public void onPlaybackParametersChanged(PlaybackParameters playbackParameters) { - updateSettingsPlaybackSpeedLists(); - } - - @Override - public void onTracksChanged(TrackGroupArray trackGroups, TrackSelectionArray trackSelections) { - updateTrackLists(); - } - - @Override - public void onTimelineChanged(Timeline timeline, @Player.TimelineChangeReason int reason) { - updateNavigation(); - updateTimeline(); + public void onEvents(Player player, Events events) { + if (events.containsAny(EVENT_PLAYBACK_STATE_CHANGED, EVENT_PLAY_WHEN_READY_CHANGED)) { + updatePlayPauseButton(); + } + if (events.containsAny( + EVENT_PLAYBACK_STATE_CHANGED, EVENT_PLAY_WHEN_READY_CHANGED, EVENT_IS_PLAYING_CHANGED)) { + updateProgress(); + } + if (events.contains(EVENT_REPEAT_MODE_CHANGED)) { + updateRepeatModeButton(); + } + if (events.contains(EVENT_SHUFFLE_MODE_ENABLED_CHANGED)) { + updateShuffleButton(); + } + if (events.containsAny( + EVENT_REPEAT_MODE_CHANGED, + EVENT_SHUFFLE_MODE_ENABLED_CHANGED, + EVENT_POSITION_DISCONTINUITY, + EVENT_TIMELINE_CHANGED)) { + updateNavigation(); + } + if (events.containsAny(EVENT_POSITION_DISCONTINUITY, EVENT_TIMELINE_CHANGED)) { + updateTimeline(); + } + if (events.contains(EVENT_PLAYBACK_PARAMETERS_CHANGED)) { + updateSettingsPlaybackSpeedLists(); + } + if (events.contains(EVENT_TRACKS_CHANGED)) { + updateTrackLists(); + } } @Override @@ -1885,6 +1884,7 @@ public void onClick(View view) { } private class SettingsAdapter extends RecyclerView.Adapter { + private final String[] mainTexts; private final String[] subTexts; private final Drawable[] iconIds; @@ -1935,6 +1935,7 @@ public void setSubTextAtPosition(int position, String subText) { } private final class SettingViewHolder extends RecyclerView.ViewHolder { + private final TextView mainTextView; private final TextView subTextView; private final ImageView iconView; @@ -1949,8 +1950,18 @@ public SettingViewHolder(View itemView) { } private class SubSettingsAdapter extends RecyclerView.Adapter { - @Nullable private List texts; - private int checkPosition; + + private String[] texts; + private int selectedIndex; + + public SubSettingsAdapter() { + texts = new String[0]; + } + + public void init(String[] texts, int selectedIndex) { + this.texts = texts; + this.selectedIndex = selectedIndex; + } @Override public SubSettingViewHolder onCreateViewHolder(ViewGroup parent, int viewType) { @@ -1962,23 +1973,15 @@ public SubSettingViewHolder onCreateViewHolder(ViewGroup parent, int viewType) { @Override public void onBindViewHolder(SubSettingViewHolder holder, int position) { - if (texts != null) { - holder.textView.setText(texts.get(position)); + if (position < texts.length) { + holder.textView.setText(texts[position]); } - holder.checkView.setVisibility(position == checkPosition ? VISIBLE : INVISIBLE); + holder.checkView.setVisibility(position == selectedIndex ? VISIBLE : INVISIBLE); } @Override public int getItemCount() { - return texts != null ? texts.size() : 0; - } - - public void setTexts(@Nullable List texts) { - this.texts = texts; - } - - public void setCheckPosition(int checkPosition) { - this.checkPosition = checkPosition; + return texts.length; } } @@ -1995,6 +1998,7 @@ public SubSettingViewHolder(View itemView) { } private static final class TrackInfo { + public final int rendererIndex; public final int groupIndex; public final int trackIndex; @@ -2164,6 +2168,7 @@ public void init( private abstract class TrackSelectionAdapter extends RecyclerView.Adapter { + protected List rendererIndices; protected List tracks; protected @Nullable MappedTrackInfo mappedTrackInfo; @@ -2247,6 +2252,7 @@ public void clear() { } private static class TrackSelectionViewHolder extends RecyclerView.ViewHolder { + public final TextView textView; public final View checkView; diff --git a/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerControlViewLayoutManager.java b/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerControlViewLayoutManager.java index 9f035c62412..4045559ccb0 100644 --- a/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerControlViewLayoutManager.java +++ b/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerControlViewLayoutManager.java @@ -24,6 +24,7 @@ import android.view.View; import android.view.View.OnLayoutChangeListener; import android.view.ViewGroup; +import android.view.ViewGroup.LayoutParams; import android.view.ViewGroup.MarginLayoutParams; import android.view.animation.LinearInterpolator; import androidx.annotation.Nullable; @@ -49,7 +50,7 @@ private final StyledPlayerControlView styledPlayerControlView; - @Nullable private final ViewGroup embeddedTransportControls; + @Nullable private final ViewGroup centerControls; @Nullable private final ViewGroup bottomBar; @Nullable private final ViewGroup minimalControls; @Nullable private final ViewGroup basicControls; @@ -59,10 +60,10 @@ @Nullable private final View timeBar; @Nullable private final View overflowShowButton; - private final AnimatorSet hideMainBarsAnimator; + private final AnimatorSet hideMainBarAnimator; private final AnimatorSet hideProgressBarAnimator; private final AnimatorSet hideAllBarsAnimator; - private final AnimatorSet showMainBarsAnimator; + private final AnimatorSet showMainBarAnimator; private final AnimatorSet showAllBarsAnimator; private final ValueAnimator overflowShowAnimator; private final ValueAnimator overflowHideAnimator; @@ -70,7 +71,7 @@ private final Runnable showAllBarsRunnable; private final Runnable hideAllBarsRunnable; private final Runnable hideProgressBarRunnable; - private final Runnable hideMainBarsRunnable; + private final Runnable hideMainBarRunnable; private final Runnable hideControllerRunnable; private final OnLayoutChangeListener onLayoutChangeListener; @@ -90,7 +91,7 @@ public StyledPlayerControlViewLayoutManager(StyledPlayerControlView styledPlayer showAllBarsRunnable = this::showAllBars; hideAllBarsRunnable = this::hideAllBars; hideProgressBarRunnable = this::hideProgressBar; - hideMainBarsRunnable = this::hideMainBars; + hideMainBarRunnable = this::hideMainBar; hideControllerRunnable = this::hideController; onLayoutChangeListener = this::onLayoutChange; animationEnabled = true; @@ -98,9 +99,8 @@ public StyledPlayerControlViewLayoutManager(StyledPlayerControlView styledPlayer shownButtons = new ArrayList<>(); // Relating to Center View - ViewGroup centerView = styledPlayerControlView.findViewById(R.id.exo_center_view); - embeddedTransportControls = - styledPlayerControlView.findViewById(R.id.exo_embedded_transport_controls); + View controlsBackground = styledPlayerControlView.findViewById(R.id.exo_controls_background); + centerControls = styledPlayerControlView.findViewById(R.id.exo_center_controls); // Relating to Minimal Layout minimalControls = styledPlayerControlView.findViewById(R.id.exo_minimal_controls); @@ -124,22 +124,16 @@ public StyledPlayerControlViewLayoutManager(StyledPlayerControlView styledPlayer overflowHideButton.setOnClickListener(this::onOverflowButtonClick); } - Resources resources = styledPlayerControlView.getResources(); - float bottomBarHeight = - resources.getDimension(R.dimen.exo_bottom_bar_height) - - resources.getDimension(R.dimen.exo_styled_progress_bar_height); - float progressBarHeight = - resources.getDimension(R.dimen.exo_styled_progress_margin_bottom) - + resources.getDimension(R.dimen.exo_styled_progress_layout_height) - - bottomBarHeight; - ValueAnimator fadeOutAnimator = ValueAnimator.ofFloat(1.0f, 0.0f); fadeOutAnimator.setInterpolator(new LinearInterpolator()); fadeOutAnimator.addUpdateListener( animation -> { float animatedValue = (float) animation.getAnimatedValue(); - if (centerView != null) { - centerView.setAlpha(animatedValue); + if (controlsBackground != null) { + controlsBackground.setAlpha(animatedValue); + } + if (centerControls != null) { + centerControls.setAlpha(animatedValue); } if (minimalControls != null) { minimalControls.setAlpha(animatedValue); @@ -156,8 +150,11 @@ public void onAnimationStart(Animator animation) { @Override public void onAnimationEnd(Animator animation) { - if (centerView != null) { - centerView.setVisibility(View.INVISIBLE); + if (controlsBackground != null) { + controlsBackground.setVisibility(View.INVISIBLE); + } + if (centerControls != null) { + centerControls.setVisibility(View.INVISIBLE); } if (minimalControls != null) { minimalControls.setVisibility(View.INVISIBLE); @@ -170,8 +167,11 @@ public void onAnimationEnd(Animator animation) { fadeInAnimator.addUpdateListener( animation -> { float animatedValue = (float) animation.getAnimatedValue(); - if (centerView != null) { - centerView.setAlpha(animatedValue); + if (controlsBackground != null) { + controlsBackground.setAlpha(animatedValue); + } + if (centerControls != null) { + centerControls.setAlpha(animatedValue); } if (minimalControls != null) { minimalControls.setAlpha(animatedValue); @@ -181,8 +181,11 @@ public void onAnimationEnd(Animator animation) { new AnimatorListenerAdapter() { @Override public void onAnimationStart(Animator animation) { - if (centerView != null) { - centerView.setVisibility(View.VISIBLE); + if (controlsBackground != null) { + controlsBackground.setVisibility(View.VISIBLE); + } + if (centerControls != null) { + centerControls.setVisibility(View.VISIBLE); } if (minimalControls != null) { minimalControls.setVisibility(isMinimalMode ? View.VISIBLE : View.INVISIBLE); @@ -193,9 +196,15 @@ public void onAnimationStart(Animator animation) { } }); - hideMainBarsAnimator = new AnimatorSet(); - hideMainBarsAnimator.setDuration(DURATION_FOR_HIDING_ANIMATION_MS); - hideMainBarsAnimator.addListener( + Resources resources = styledPlayerControlView.getResources(); + float translationYForProgressBar = + resources.getDimension(R.dimen.exo_styled_bottom_bar_height) + - resources.getDimension(R.dimen.exo_styled_progress_bar_height); + float translationYForNoBars = resources.getDimension(R.dimen.exo_styled_bottom_bar_height); + + hideMainBarAnimator = new AnimatorSet(); + hideMainBarAnimator.setDuration(DURATION_FOR_HIDING_ANIMATION_MS); + hideMainBarAnimator.addListener( new AnimatorListenerAdapter() { @Override public void onAnimationStart(Animator animation) { @@ -211,10 +220,10 @@ public void onAnimationEnd(Animator animation) { } } }); - hideMainBarsAnimator + hideMainBarAnimator .play(fadeOutAnimator) - .with(ofTranslationY(0, bottomBarHeight, timeBar)) - .with(ofTranslationY(0, bottomBarHeight, bottomBar)); + .with(ofTranslationY(0, translationYForProgressBar, timeBar)) + .with(ofTranslationY(0, translationYForProgressBar, bottomBar)); hideProgressBarAnimator = new AnimatorSet(); hideProgressBarAnimator.setDuration(DURATION_FOR_HIDING_ANIMATION_MS); @@ -235,8 +244,8 @@ public void onAnimationEnd(Animator animation) { } }); hideProgressBarAnimator - .play(ofTranslationY(bottomBarHeight, bottomBarHeight + progressBarHeight, timeBar)) - .with(ofTranslationY(bottomBarHeight, bottomBarHeight + progressBarHeight, bottomBar)); + .play(ofTranslationY(translationYForProgressBar, translationYForNoBars, timeBar)) + .with(ofTranslationY(translationYForProgressBar, translationYForNoBars, bottomBar)); hideAllBarsAnimator = new AnimatorSet(); hideAllBarsAnimator.setDuration(DURATION_FOR_HIDING_ANIMATION_MS); @@ -258,12 +267,12 @@ public void onAnimationEnd(Animator animation) { }); hideAllBarsAnimator .play(fadeOutAnimator) - .with(ofTranslationY(0, bottomBarHeight + progressBarHeight, timeBar)) - .with(ofTranslationY(0, bottomBarHeight + progressBarHeight, bottomBar)); + .with(ofTranslationY(0, translationYForNoBars, timeBar)) + .with(ofTranslationY(0, translationYForNoBars, bottomBar)); - showMainBarsAnimator = new AnimatorSet(); - showMainBarsAnimator.setDuration(DURATION_FOR_SHOWING_ANIMATION_MS); - showMainBarsAnimator.addListener( + showMainBarAnimator = new AnimatorSet(); + showMainBarAnimator.setDuration(DURATION_FOR_SHOWING_ANIMATION_MS); + showMainBarAnimator.addListener( new AnimatorListenerAdapter() { @Override public void onAnimationStart(Animator animation) { @@ -275,10 +284,10 @@ public void onAnimationEnd(Animator animation) { setUxState(UX_STATE_ALL_VISIBLE); } }); - showMainBarsAnimator + showMainBarAnimator .play(fadeInAnimator) - .with(ofTranslationY(bottomBarHeight, 0, timeBar)) - .with(ofTranslationY(bottomBarHeight, 0, bottomBar)); + .with(ofTranslationY(translationYForProgressBar, 0, timeBar)) + .with(ofTranslationY(translationYForProgressBar, 0, bottomBar)); showAllBarsAnimator = new AnimatorSet(); showAllBarsAnimator.setDuration(DURATION_FOR_SHOWING_ANIMATION_MS); @@ -296,8 +305,8 @@ public void onAnimationEnd(Animator animation) { }); showAllBarsAnimator .play(fadeInAnimator) - .with(ofTranslationY(bottomBarHeight + progressBarHeight, 0, timeBar)) - .with(ofTranslationY(bottomBarHeight + progressBarHeight, 0, bottomBar)); + .with(ofTranslationY(translationYForNoBars, 0, timeBar)) + .with(ofTranslationY(translationYForNoBars, 0, bottomBar)); overflowShowAnimator = ValueAnimator.ofFloat(0.0f, 1.0f); overflowShowAnimator.setDuration(DURATION_FOR_SHOWING_ANIMATION_MS); @@ -395,7 +404,7 @@ public void resetHideCallbacks() { } else if (uxState == UX_STATE_ONLY_PROGRESS_VISIBLE) { postDelayedRunnable(hideProgressBarRunnable, ANIMATION_INTERVAL_MS); } else { - postDelayedRunnable(hideMainBarsRunnable, showTimeoutMs); + postDelayedRunnable(hideMainBarRunnable, showTimeoutMs); } } } @@ -403,7 +412,7 @@ public void resetHideCallbacks() { public void removeHideCallbacks() { styledPlayerControlView.removeCallbacks(hideControllerRunnable); styledPlayerControlView.removeCallbacks(hideAllBarsRunnable); - styledPlayerControlView.removeCallbacks(hideMainBarsRunnable); + styledPlayerControlView.removeCallbacks(hideMainBarRunnable); styledPlayerControlView.removeCallbacks(hideProgressBarRunnable); } @@ -466,9 +475,9 @@ private void onLayoutChange( int oldRight, int oldBottom) { - boolean shouldBeMinimalMode = shouldBeMinimalMode(); - if (isMinimalMode != shouldBeMinimalMode) { - isMinimalMode = shouldBeMinimalMode; + boolean useMinimalMode = useMinimalMode(); + if (isMinimalMode != useMinimalMode) { + isMinimalMode = useMinimalMode; v.post(this::updateLayoutForSizeChange); } boolean widthChanged = (right - left) != (oldRight - oldLeft); @@ -498,7 +507,7 @@ private void showAllBars() { showAllBarsAnimator.start(); break; case UX_STATE_ONLY_PROGRESS_VISIBLE: - showMainBarsAnimator.start(); + showMainBarAnimator.start(); break; case UX_STATE_ANIMATING_HIDE: needToShowBars = true; @@ -519,8 +528,8 @@ private void hideProgressBar() { hideProgressBarAnimator.start(); } - private void hideMainBars() { - hideMainBarsAnimator.start(); + private void hideMainBar() { + hideMainBarAnimator.start(); postDelayedRunnable(hideProgressBarRunnable, ANIMATION_INTERVAL_MS); } @@ -553,7 +562,7 @@ private void animateOverflow(float animatedValue) { } } - private boolean shouldBeMinimalMode() { + private boolean useMinimalMode() { int width = styledPlayerControlView.getWidth() - styledPlayerControlView.getPaddingLeft() @@ -562,13 +571,21 @@ private boolean shouldBeMinimalMode() { styledPlayerControlView.getHeight() - styledPlayerControlView.getPaddingBottom() - styledPlayerControlView.getPaddingTop(); - int defaultModeWidth = + + int centerControlWidth = + getWidthWithMargins(centerControls) + - (centerControls != null + ? (centerControls.getPaddingLeft() + centerControls.getPaddingRight()) + : 0); + + int defaultModeMinimumWidth = Math.max( - getWidth(embeddedTransportControls), getWidth(timeView) + getWidth(overflowShowButton)); - int defaultModeHeight = - getHeight(embeddedTransportControls) + getHeight(timeBar) + getHeight(bottomBar); + centerControlWidth, + getWidthWithMargins(timeView) + getWidthWithMargins(overflowShowButton)); + int defaultModeMinimumHeight = + getHeightWithMargins(centerControls) + 2 * getHeightWithMargins(bottomBar); - return (width <= defaultModeWidth || height <= defaultModeHeight); + return width <= defaultModeMinimumWidth || height <= defaultModeMinimumHeight; } private void updateLayoutForSizeChange() { @@ -576,20 +593,6 @@ private void updateLayoutForSizeChange() { minimalControls.setVisibility(isMinimalMode ? View.VISIBLE : View.INVISIBLE); } - View fullScreenButton = styledPlayerControlView.findViewById(R.id.exo_fullscreen); - if (fullScreenButton != null) { - ViewGroup parent = (ViewGroup) fullScreenButton.getParent(); - parent.removeView(fullScreenButton); - - if (isMinimalMode && minimalControls != null) { - minimalControls.addView(fullScreenButton); - } else if (!isMinimalMode && basicControls != null) { - int index = Math.max(0, basicControls.getChildCount() - 1); - basicControls.addView(fullScreenButton, index); - } else { - parent.addView(fullScreenButton); - } - } if (timeBar != null) { MarginLayoutParams timeBarParams = (MarginLayoutParams) timeBar.getLayoutParams(); int timeBarMarginBottom = @@ -598,13 +601,14 @@ private void updateLayoutForSizeChange() { .getDimensionPixelSize(R.dimen.exo_styled_progress_margin_bottom); timeBarParams.bottomMargin = (isMinimalMode ? 0 : timeBarMarginBottom); timeBar.setLayoutParams(timeBarParams); - if (timeBar instanceof DefaultTimeBar - && uxState != UX_STATE_ANIMATING_HIDE - && uxState != UX_STATE_ANIMATING_SHOW) { - if (isMinimalMode || uxState != UX_STATE_ALL_VISIBLE) { - ((DefaultTimeBar) timeBar).hideScrubber(); - } else { - ((DefaultTimeBar) timeBar).showScrubber(); + if (timeBar instanceof DefaultTimeBar) { + DefaultTimeBar defaultTimeBar = (DefaultTimeBar) timeBar; + if (isMinimalMode) { + defaultTimeBar.hideScrubber(/* disableScrubberPadding= */ true); + } else if (uxState == UX_STATE_ONLY_PROGRESS_VISIBLE) { + defaultTimeBar.hideScrubber(/* disableScrubberPadding= */ false); + } else if (uxState != UX_STATE_ANIMATING_HIDE && uxState != UX_STATE_ANIMATING_SHOW) { + defaultTimeBar.showScrubber(); } } } @@ -634,68 +638,86 @@ private void onLayoutWidthChanged() { styledPlayerControlView.getWidth() - styledPlayerControlView.getPaddingLeft() - styledPlayerControlView.getPaddingRight(); - int bottomBarWidth = getWidth(timeView); - for (int i = 0; i < basicControls.getChildCount(); ++i) { - bottomBarWidth += basicControls.getChildAt(i).getWidth(); - } - - // BasicControls keeps overflow button at least. - int minBasicControlsChildCount = 1; - // ExtraControls keeps overflow button and settings button at least. - int minExtraControlsChildCount = 2; - - if (bottomBarWidth > width) { - // move control views from basicControls to extraControls - ArrayList movingChildren = new ArrayList<>(); - int movingWidth = 0; - int endIndex = basicControls.getChildCount() - minBasicControlsChildCount; - for (int index = 0; index < endIndex; index++) { - View child = basicControls.getChildAt(index); - movingWidth += child.getWidth(); - movingChildren.add(child); - if (bottomBarWidth - movingWidth <= width) { - break; - } - } - if (!movingChildren.isEmpty()) { - basicControls.removeViews(0, movingChildren.size()); + // Reset back to all controls being basic controls and the overflow not being needed. The last + // child of extraControls is the overflow hide button, which shouldn't be moved back. + while (extraControls.getChildCount() > 1) { + int controlViewIndex = extraControls.getChildCount() - 2; + View controlView = extraControls.getChildAt(controlViewIndex); + extraControls.removeViewAt(controlViewIndex); + basicControls.addView(controlView, /* index= */ 0); + } + if (overflowShowButton != null) { + overflowShowButton.setVisibility(View.GONE); + } - for (View child : movingChildren) { - int index = extraControls.getChildCount() - minExtraControlsChildCount; - extraControls.addView(child, index); - } - } + // Calculate how much of the available width is occupied. The last child of basicControls is the + // overflow show button, which we're currently assuming will not be visible. + int occupiedWidth = getWidthWithMargins(timeView); + int endIndex = basicControls.getChildCount() - 1; + for (int i = 0; i < endIndex; i++) { + View controlView = basicControls.getChildAt(i); + occupiedWidth += getWidthWithMargins(controlView); + } - } else { - // Move controls from extraControls to basicControls if possible, else do nothing. - ArrayList movingChildren = new ArrayList<>(); - int movingWidth = 0; - int startIndex = extraControls.getChildCount() - minExtraControlsChildCount - 1; - for (int index = startIndex; index >= 0; index--) { - View child = extraControls.getChildAt(index); - movingWidth += child.getWidth(); - if (bottomBarWidth + movingWidth > width) { + if (occupiedWidth > width) { + // We need to move some controls to extraControls. + if (overflowShowButton != null) { + overflowShowButton.setVisibility(View.VISIBLE); + occupiedWidth += getWidthWithMargins(overflowShowButton); + } + ArrayList controlsToMove = new ArrayList<>(); + // The last child of basicControls is the overflow show button, which shouldn't be moved. + for (int i = 0; i < endIndex; i++) { + View control = basicControls.getChildAt(i); + occupiedWidth -= getWidthWithMargins(control); + controlsToMove.add(control); + if (occupiedWidth <= width) { break; } - movingChildren.add(child); } - - if (!movingChildren.isEmpty()) { - extraControls.removeViews(startIndex - movingChildren.size() + 1, movingChildren.size()); - - for (View child : movingChildren) { - basicControls.addView(child, 0); + if (!controlsToMove.isEmpty()) { + basicControls.removeViews(/* start= */ 0, controlsToMove.size()); + for (int i = 0; i < controlsToMove.size(); i++) { + // The last child of extraControls is the overflow hide button. Add controls before it. + int index = extraControls.getChildCount() - 1; + extraControls.addView(controlsToMove.get(i), index); } } + } else { + // If extraControls are visible, hide them since they're now empty. + if (extraControlsScrollView != null + && extraControlsScrollView.getVisibility() == View.VISIBLE + && !overflowHideAnimator.isStarted()) { + overflowShowAnimator.cancel(); + overflowHideAnimator.start(); + } } } - private static int getWidth(@Nullable View v) { - return (v != null ? v.getWidth() : 0); + private static int getWidthWithMargins(@Nullable View v) { + if (v == null) { + return 0; + } + int width = v.getWidth(); + LayoutParams layoutParams = v.getLayoutParams(); + if (layoutParams instanceof MarginLayoutParams) { + MarginLayoutParams marginLayoutParams = (MarginLayoutParams) layoutParams; + width += marginLayoutParams.leftMargin + marginLayoutParams.rightMargin; + } + return width; } - private static int getHeight(@Nullable View v) { - return (v != null ? v.getHeight() : 0); + private static int getHeightWithMargins(@Nullable View v) { + if (v == null) { + return 0; + } + int height = v.getHeight(); + LayoutParams layoutParams = v.getLayoutParams(); + if (layoutParams instanceof MarginLayoutParams) { + MarginLayoutParams marginLayoutParams = (MarginLayoutParams) layoutParams; + height += marginLayoutParams.topMargin + marginLayoutParams.bottomMargin; + } + return height; } } diff --git a/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerView.java b/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerView.java index 81df78536d1..36e4a2ed620 100644 --- a/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerView.java +++ b/library/ui/src/main/java/com/google/android/exoplayer2/ui/StyledPlayerView.java @@ -59,7 +59,6 @@ import com.google.android.exoplayer2.source.ads.AdsLoader; import com.google.android.exoplayer2.text.Cue; import com.google.android.exoplayer2.text.TextOutput; -import com.google.android.exoplayer2.trackselection.TrackSelection; import com.google.android.exoplayer2.trackselection.TrackSelectionArray; import com.google.android.exoplayer2.ui.AspectRatioFrameLayout.ResizeMode; import com.google.android.exoplayer2.ui.spherical.SingleTapListener; @@ -582,8 +581,6 @@ public void setPlayer(@Nullable Player player) { oldVideoComponent.clearVideoTextureView((TextureView) surfaceView); } else if (surfaceView instanceof SphericalGLSurfaceView) { ((SphericalGLSurfaceView) surfaceView).setVideoComponent(null); - } else if (surfaceView instanceof VideoDecoderGLSurfaceView) { - oldVideoComponent.setVideoDecoderOutputBufferRenderer(null); } else if (surfaceView instanceof SurfaceView) { oldVideoComponent.clearVideoSurfaceView((SurfaceView) surfaceView); } @@ -610,9 +607,6 @@ public void setPlayer(@Nullable Player player) { newVideoComponent.setVideoTextureView((TextureView) surfaceView); } else if (surfaceView instanceof SphericalGLSurfaceView) { ((SphericalGLSurfaceView) surfaceView).setVideoComponent(newVideoComponent); - } else if (surfaceView instanceof VideoDecoderGLSurfaceView) { - newVideoComponent.setVideoDecoderOutputBufferRenderer( - ((VideoDecoderGLSurfaceView) surfaceView).getVideoDecoderOutputBufferRenderer()); } else if (surfaceView instanceof SurfaceView) { newVideoComponent.setVideoSurfaceView((SurfaceView) surfaceView); } @@ -734,9 +728,8 @@ public void setShutterBackgroundColor(int color) { /** * Sets whether the currently displayed video frame or media artwork is kept visible when the * player is reset. A player reset is defined to mean the player being re-prepared with different - * media, the player transitioning to unprepared media, {@link Player#stop(boolean)} being called - * with {@code reset=true}, or the player being replaced or cleared by calling {@link - * #setPlayer(Player)}. + * media, the player transitioning to unprepared media or an empty list of media items, or the + * player being replaced or cleared by calling {@link #setPlayer(Player)}. * *

      If enabled, the currently displayed video frame or media artwork will be kept visible until * the player set on the view has been successfully prepared with new media and loaded enough of @@ -1375,15 +1368,9 @@ private void updateForCurrentTrackSelections(boolean isNewPlayer) { closeShutter(); // Display artwork if enabled and available, else hide it. if (useArtwork()) { - for (int i = 0; i < selections.length; i++) { - @Nullable TrackSelection selection = selections.get(i); - if (selection != null) { - for (int j = 0; j < selection.length(); j++) { - @Nullable Metadata metadata = selection.getFormat(j).metadata; - if (metadata != null && setArtworkFromMetadata(metadata)) { - return; - } - } + for (Metadata metadata : player.getCurrentStaticMetadata()) { + if (setArtworkFromMetadata(metadata)) { + return; } } if (setDrawableArtwork(defaultArtwork)) { diff --git a/library/ui/src/main/java/com/google/android/exoplayer2/ui/TrackSelectionView.java b/library/ui/src/main/java/com/google/android/exoplayer2/ui/TrackSelectionView.java index 0ae38e83a98..b49afb63167 100644 --- a/library/ui/src/main/java/com/google/android/exoplayer2/ui/TrackSelectionView.java +++ b/library/ui/src/main/java/com/google/android/exoplayer2/ui/TrackSelectionView.java @@ -25,6 +25,7 @@ import android.widget.LinearLayout; import androidx.annotation.AttrRes; import androidx.annotation.Nullable; +import com.google.android.exoplayer2.C; import com.google.android.exoplayer2.Format; import com.google.android.exoplayer2.RendererCapabilities; import com.google.android.exoplayer2.source.TrackGroup; @@ -291,7 +292,7 @@ private void updateViews() { trackView.setText(trackNameProvider.getTrackName(trackInfos[trackIndex].format)); trackView.setTag(trackInfos[trackIndex]); if (mappedTrackInfo.getTrackSupport(rendererIndex, groupIndex, trackIndex) - == RendererCapabilities.FORMAT_HANDLED) { + == C.FORMAT_HANDLED) { trackView.setFocusable(true); trackView.setOnClickListener(componentListener); } else { diff --git a/library/ui/src/main/res/drawable-v21/exo_ripple_ffwd.xml b/library/ui/src/main/res/drawable-v21/exo_ripple_ffwd.xml deleted file mode 100644 index 5e4dd5550f9..00000000000 --- a/library/ui/src/main/res/drawable-v21/exo_ripple_ffwd.xml +++ /dev/null @@ -1,26 +0,0 @@ - - - - - - - - - - diff --git a/library/ui/src/main/res/drawable/exo_ripple_ffwd.xml b/library/ui/src/main/res/drawable/exo_ripple_ffwd.xml deleted file mode 100644 index 9f7e1fd0279..00000000000 --- a/library/ui/src/main/res/drawable/exo_ripple_ffwd.xml +++ /dev/null @@ -1,29 +0,0 @@ - - - - - - - - - - - - - - - diff --git a/library/ui/src/main/res/drawable/exo_ripple_rew.xml b/library/ui/src/main/res/drawable/exo_ripple_rew.xml deleted file mode 100644 index 5562b1352cb..00000000000 --- a/library/ui/src/main/res/drawable/exo_ripple_rew.xml +++ /dev/null @@ -1,29 +0,0 @@ - - - - - - - - - - - - - - - diff --git a/library/ui/src/main/res/layout-v23/exo_styled_player_control_ffwd_button.xml b/library/ui/src/main/res/layout-v23/exo_styled_player_control_ffwd_button.xml new file mode 100644 index 00000000000..18166f71f67 --- /dev/null +++ b/library/ui/src/main/res/layout-v23/exo_styled_player_control_ffwd_button.xml @@ -0,0 +1,20 @@ + + + + +