Skip to content

Commit

Permalink
[GTK][WPE] Start implementing MediaStream API
Browse files Browse the repository at this point in the history 
https://bugs.webkit.org/show_bug.cgi?id=185787

Source/WebCore:

Patch by Thibault Saunier  <[email protected]> and Alejandro G. Castro <[email protected]> on 2018-06-07
Reviewed by Philippe Normand.

We are adding all the required classes to make the
MediaStream API work, that means our own RealtimeMediaSourceCenterLibWebRTC
for the platform, the GStreamerCaptureDeviceManager, the audio/video capturers
and their respective audio/video sources as well as a dedicated GStreamer Source
that adds support for using MediaStream stream inside playbin3.
We are using the GstDeviceMonitor to list devices on the devices.

Enable mediastream tests.

* platform/GStreamer.cmake: Added the new files to the compilation.
* platform/audio/AudioStreamDescription.h: Added new GStreamer type.
* platform/audio/PlatformAudioData.h: Added new GStreamer type for
the GStreamerAudioData class.
* platform/graphics/gstreamer/GStreamerCommon.cpp:
(WebCore::simpleBusMessageCallback): This function and the next
one help us to connect a monitoring callback to a pipeline for
debugging.
(WebCore::connectSimpleBusMessageCallback): Ditto.
* platform/graphics/gstreamer/GStreamerCommon.h: Ditto
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::load): Make use of the loadFull() method.
(WebCore::MediaPlayerPrivateGStreamer::loadFull): Very similar to load()
but allows specifying what pipeline type to use (null to let the function determine
which one should be used). This is required as we force to always use playbin3 for the
mediastream source as it relies on the GstStream API.
(WebCore::MediaPlayerPrivateGStreamer::playbackPosition const): Style fix.
(WebCore::MediaPlayerPrivateGStreamer::naturalSize const): Added, use MediaStream specific information if available.
(WebCore::MediaPlayerPrivateGStreamer::updateTracks): Some style fixes.
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): Enhance dotfiles names.
(WebCore::MediaPlayerPrivateGStreamer::processTableOfContentsEntry): Minor formatting fix.
(WebCore::MediaPlayerPrivateGStreamer::sourceSetup): Set MediaStream on WebKitMediaStreamSource when setting it up.
(WebCore::MediaPlayerPrivateGStreamer::supportsType): Advertise that we support MediaStream if support is built.
(WebCore::MediaPlayerPrivateGStreamer::createGSTPlayBin): Make sure playbin3 is forced when loading a MediaStream.
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add a reference to the MediaStream object and declare loadFull and naturalSize override.
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
(WebCore::registerWebKitGStreamerElements): Register the new MediaStreamSrc element.
* platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp:
(WebCore::VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer): Make sure that MediaStream MAIN tracks are selected by default. We have no way to do it in MediaStreamSrc now as the GstStreamCollection is recreated by parsebin.
* platform/mediastream/RealtimeMediaSource.h: Make CaptureFailed a virtual method as in our mocks we require need to make
* platform/mediastream/RealtimeMediaSourceCenter.cpp:
(WebCore::RealtimeMediaSourceCenter::singleton): Remove the code
used for compilation for the platform when we do not have a
RealtimeMediaSourceCenterLibWebRTC. Now we return the proper class
for the platform.
* platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp:
Added class representing the RealtimeMediaSource for the Audio
with GStreamer.
* platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h:
Ditto.
* platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp: Added
this class that represents the GStreamer pipeline that captures
audio from the system devices, it inherits from GStreamerCapturer.
* platform/mediastream/gstreamer/GStreamerAudioCapturer.h: Dito.
* platform/mediastream/gstreamer/GStreamerAudioData.h: Added this
class implementing PlatformAudioData for the GStreamer platform,
used to pass the samples information.
* platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h:
Added this class implementing AudioStreamDescription to export the
information about the audio stream to libwebrtc.
* platform/mediastream/gstreamer/GStreamerCaptureDevice.h: Added
this base class for the audio and video capturing devices, it
implements general WebKit CaptureDevice class.
* platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp:
Added this class that implements the system monitor to get the
list of available devices in the system. It uses GstDeviceMonitor
to handle the operation. It uses two singleton device managers one
for audio and another one for video, as required by the
RealtimeMediaSourceCenter design.
* platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h: Ditto.
* platform/mediastream/gstreamer/GStreamerCapturer.cpp: Added this
base class representing how GStreamer captures the media from the
input devices in the system. Two classes inherit from this one to
capture audio and video. It setups the GStreamer pipeline and adds
functions to control it.
* platform/mediastream/gstreamer/GStreamerCapturer.h: Ditto.
* platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp: Added.
Implements a subclass of GstBin as a source element that will contain several
GstAppSrc, basically one per MediaStreamTrackPrivate of the MediaStreamPrivate
passed in parameter. It adds Observers on the MediaStreamTracks and
pushes the data to the sources as required. The element implements the GstURIHandler
interface so it can be used in playbin. The MediaPlayerPrivateGStreamer is responsible
for passing the MediaStreamPrivate object to the source when required.
(WebCore::webkitMediaStreamSrcPadProbeCb): Event probe that fixes stream_start events (setting the ID etc)
and finally add src pads to the pipeline.
* platform/mediastream/gstreamer/GStreamerMediaStreamSource.h: Ditto.
* platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp:
Added this RealtimeMediaSource representing the source of the
video data for the GStreamer platform. It handles the settings and
capabilities of the source and creates the capturer used to
control the operation of the stream.
* platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h: Ditto.
* platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp: Added
this class that inherits from the GStreamerCapturer and controls
the GStreamer pipelines of the video streams of the system.
* platform/mediastream/gstreamer/GStreamerVideoCapturer.h: Ditto.
* platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp: Added. Implementation of a Mock capturer for audio stream.
Subclasses GStreamerAudioCapturer and wraps a MockRealtimeAudioSource so that the behaviour is the same a MockRealtimeAudioSource
but still the GStreamer implementation code paths are tested.
* platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h: Ditto.
* platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp: Added. Implementation of a Mock capturer for video stream.
Subclasses GStreamerVideoCapturer and wraps a MockRealtimeVideoSource so that the behaviour is the same a MockRealtimeVideoSource
but still the GStreamer implementation code paths are tested.
* platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h: Ditto.
* platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp:
Added this class that implements the key RealtimeMediaSourceCenter
functions to configure the base class for the platform, using the
other GStreamer classes.
* platform/mock/MockRealtimeAudioSource.cpp: Do not build ::create if GStreamer implementation is built
* platform/mock/MockRealtimeVideoSource.cpp: Do not build ::create if GStreamer implementation is built

Tools:

Patch by Thibault Saunier <[email protected]> on 2018-06-07
Reviewed by Philippe Normand.

* Scripts/webkitpy/style/checker.py: Apply special formatting rules for new GObject subclasses.
* gstreamer/jhbuild.modules: Added a patch for the gst-plugins-base.
* gstreamer/patches/gst-plugins-base-0001-parsebin-Post-STREAM_COLLECTION-on-EVENT_STREAM_COLL.patch:
Added this fix to gst-plugings-base to fix the decodebin3. Merged as 89d0e9cc92a86aa0227ee87406737b6d31670aea

LayoutTests:

Patch by Thibault Saunier <[email protected]> on 2018-06-07
Reviewed by Philippe Normand.

* platform/gtk/TestExpectations: Reactivate MediaStream tests and enable all tests
related to the mediaDevice.enumerateDevices and MediaStream (not RTCPeerConnection nor
webaudio).

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@232589 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
[email protected] committed Jun 7, 2018
1 parent 3ac7cd5 commit 8245958
Show file tree
Hide file tree
Showing 44 changed files with 2,991 additions and 43 deletions.
11 changes: 11 additions & 0 deletions LayoutTests/ChangeLog
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
2018-06-07 Thibault Saunier <[email protected]>

[GTK][WPE] Start implementing MediaStream API
https://bugs.webkit.org/show_bug.cgi?id=185787

Reviewed by Philippe Normand.

* platform/gtk/TestExpectations: Reactivate MediaStream tests and enable all tests
related to the mediaDevice.enumerateDevices and MediaStream (not RTCPeerConnection nor
webaudio).

2018-06-07 Brent Fulgham <[email protected]>

Remove unused debug mode conditions
Expand Down
33 changes: 10 additions & 23 deletions LayoutTests/platform/gtk/TestExpectations
View file
Original file line number Diff line number Diff line change
Expand Up @@ -572,40 +572,31 @@ webkit.org/b/99036 fast/shadow-dom/pointerlockelement-in-slot.html [ Skip ]
webkit.org/b/85211 ietestcenter/css3/flexbox/flexbox-align-stretch-001.htm [ ImageOnlyFailure ]
webkit.org/b/85212 ietestcenter/css3/flexbox/flexbox-layout-002.htm [ ImageOnlyFailure ]

# Skipping until backend is implemented
fast/mediastream [ Skip ]
webrtc [ Skip ]
imported/w3c/web-platform-tests/webrtc/ [ Skip ]
http/tests/webrtc [ Skip ]
# The MediaStream implementation is also still not completed
webkit.org/b/79203 fast/mediastream/mock-media-source-webaudio.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/getUserMedia-webaudio.html [ Failure ]
webkit.org/b/79203 fast/mediastream/mock-media-source.html [ Failure ]
webkit.org/b/79203 fast/mediastream/MediaStream-video-element-track-stop.html [ Pass Crash ]
webkit.org/b/79203 fast/mediastream/MediaStream-video-element-track-stop.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-dtmf.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-icecandidate-event.html [ Failure Crash ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-media-setup-two-dialogs.html [ Failure Crash ]
webkit.org/b/79203 fast/mediastream/RTCRtpSender-replaceTrack.html [ Failure ]
# Crash is webkit.org/b/184292
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-addIceCandidate.html [ Failure Pass Crash ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-closed-state.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-ice.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-iceconnectionstatechange-event.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-iceconnectionstatechange-event.html [ Failure Timeout ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-ice.html [ Failure Timeout ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-inspect-answer.html [ Failure ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-inspect-offer-bundlePolicy-bundle-only.html [ Failure ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-inspect-offer.html [ Failure ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-legacy-stream-based-api.html [ Failure ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-media-setup-callbacks-single-dialog.html [ Crash Failure ]
webkit.org/b/79203 webaudio/mediastreamaudiosourcenode.html [ Failure ]
webkit.org/b/79203 fast/mediastream/MediaStream-video-element-displays-buffer.html [ Crash ]
webkit.org/b/79203 fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html [ Crash ]
webkit.org/b/79203 fast/mediastream/MediaStream-video-element.html [ Crash ]
webkit.org/b/79203 fast/mediastream/apply-constraints-advanced.html [ Crash ]
webkit.org/b/79203 fast/mediastream/MediaStream-video-element-video-tracks-disabled-then-enabled.html [ Timeout Failure ]
webkit.org/b/79203 fast/mediastream/MediaStream-video-element-displays-buffer.html [ Failure ]
webkit.org/b/79203 http/tests/media/media-stream/getusermedia-with-canvas.html [ Crash ]
webkit.org/b/79203 fast/mediastream/MediaStream-MediaElement-setObject-null.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/RTCPeerConnection-stats.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/captureStream/canvas2d-heavy-drawing.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/get-user-media-on-loadedmetadata.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/local-audio-playing-event.html [ Timeout ]
webkit.org/b/79203 fast/mediastream/media-stream-track-source-failure.html [ Timeout ]
webkit.org/b/79203 imported/w3c/web-platform-tests/mediacapture-streams/MediaStream-MediaElement-preload-none.https.html [ Failure ]
webkit.org/b/79203 imported/w3c/web-platform-tests/mediacapture-streams/MediaStreamTrack-end-manual.https.html [ Failure ]
webkit.org/b/151344 fast/mediastream/MediaStream-add-ended-tracks.html [ Timeout ]
Expand All @@ -616,6 +607,9 @@ webkit.org/b/173257 fast/mediastream/getUserMedia-grant-persistency3.html [ Pass

# Canvas captureStream support is not implemented
webkit.org/b/169811 fast/mediacapturefromelement [ Skip ]
webkit.org/b/169811 fast/mediastream/captureStream/ [ Failure ]
webkit.org/b/169811 fast/mediastream/captureStream/canvas2d.html [ Crash Timeout ]
webkit.org/b/169811 fast/mediastream/captureStream/canvas3d.html [ Crash Timeout ]

# Requires Resolution Media Query support
webkit.org/b/100137 fast/media/mq-resolution.html [ Failure ]
Expand Down Expand Up @@ -1241,25 +1235,18 @@ webkit.org/b/131546 media/track/track-in-band.html [ Crash Timeout Failure ]

webkit.org/b/163782 media/video-played-ranges-1.html [ Crash Pass ]

webkit.org/b/169811 fast/mediastream/captureStream/canvas2d.html [ Crash Timeout ]
webkit.org/b/169811 fast/mediastream/captureStream/canvas3d.html [ Crash Timeout ]

webkit.org/b/172281 accessibility/insert-children-assert.html [ Crash ]

webkit.org/b/172955 media/video-preload.html [ Crash Pass ]

webkit.org/b/175575 imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-with-slow-text-tracks.html [ Crash Pass ]

webkit.org/b/176801 fast/mediastream/apply-constraints-audio.html [ Crash ]
webkit.org/b/176801 fast/mediastream/argument-types.html [ Crash Pass ]

webkit.org/b/176803 http/tests/local/blob/send-hybrid-blob-using-open-panel.html [ Crash ]

webkit.org/b/176856 fast/mediastream/apply-constraints-video.html [ Crash ]

# See also webkit.org/b/141699
webkit.org/b/177534 fast/attachment/attachment-respects-css-size.html [ Crash ImageOnlyFailure ]
webkit.org/b/177535 fast/mediastream/media-device-info.html [ Crash Pass ]
# See also webkit.org/b/163528 Threaded compositor failures caused by one of r203496, r203497, or r203498
webkit.org/b/177536 media/video-object-fit.html [ Crash ImageOnlyFailure ]

Expand Down
117 changes: 117 additions & 0 deletions Source/WebCore/ChangeLog
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,120 @@
2018-06-07 Thibault Saunier <[email protected]> and Alejandro G. Castro <[email protected]>

[GTK][WPE] Start implementing MediaStream API
https://bugs.webkit.org/show_bug.cgi?id=185787

Reviewed by Philippe Normand.

We are adding all the required classes to make the
MediaStream API work, that means our own RealtimeMediaSourceCenterLibWebRTC
for the platform, the GStreamerCaptureDeviceManager, the audio/video capturers
and their respective audio/video sources as well as a dedicated GStreamer Source
that adds support for using MediaStream stream inside playbin3.
We are using the GstDeviceMonitor to list devices on the devices.

Enable mediastream tests.

* platform/GStreamer.cmake: Added the new files to the compilation.
* platform/audio/AudioStreamDescription.h: Added new GStreamer type.
* platform/audio/PlatformAudioData.h: Added new GStreamer type for
the GStreamerAudioData class.
* platform/graphics/gstreamer/GStreamerCommon.cpp:
(WebCore::simpleBusMessageCallback): This function and the next
one help us to connect a monitoring callback to a pipeline for
debugging.
(WebCore::connectSimpleBusMessageCallback): Ditto.
* platform/graphics/gstreamer/GStreamerCommon.h: Ditto
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::load): Make use of the loadFull() method.
(WebCore::MediaPlayerPrivateGStreamer::loadFull): Very similar to load()
but allows specifying what pipeline type to use (null to let the function determine
which one should be used). This is required as we force to always use playbin3 for the
mediastream source as it relies on the GstStream API.
(WebCore::MediaPlayerPrivateGStreamer::playbackPosition const): Style fix.
(WebCore::MediaPlayerPrivateGStreamer::naturalSize const): Added, use MediaStream specific information if available.
(WebCore::MediaPlayerPrivateGStreamer::updateTracks): Some style fixes.
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): Enhance dotfiles names.
(WebCore::MediaPlayerPrivateGStreamer::processTableOfContentsEntry): Minor formatting fix.
(WebCore::MediaPlayerPrivateGStreamer::sourceSetup): Set MediaStream on WebKitMediaStreamSource when setting it up.
(WebCore::MediaPlayerPrivateGStreamer::supportsType): Advertise that we support MediaStream if support is built.
(WebCore::MediaPlayerPrivateGStreamer::createGSTPlayBin): Make sure playbin3 is forced when loading a MediaStream.
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add a reference to the MediaStream object and declare loadFull and naturalSize override.
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
(WebCore::registerWebKitGStreamerElements): Register the new MediaStreamSrc element.
* platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp:
(WebCore::VideoTrackPrivateGStreamer::VideoTrackPrivateGStreamer): Make sure that MediaStream MAIN tracks are selected by default. We have no way to do it in MediaStreamSrc now as the GstStreamCollection is recreated by parsebin.
* platform/mediastream/RealtimeMediaSource.h: Make CaptureFailed a virtual method as in our mocks we require need to make
* platform/mediastream/RealtimeMediaSourceCenter.cpp:
(WebCore::RealtimeMediaSourceCenter::singleton): Remove the code
used for compilation for the platform when we do not have a
RealtimeMediaSourceCenterLibWebRTC. Now we return the proper class
for the platform.
* platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp:
Added class representing the RealtimeMediaSource for the Audio
with GStreamer.
* platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h:
Ditto.
* platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp: Added
this class that represents the GStreamer pipeline that captures
audio from the system devices, it inherits from GStreamerCapturer.
* platform/mediastream/gstreamer/GStreamerAudioCapturer.h: Dito.
* platform/mediastream/gstreamer/GStreamerAudioData.h: Added this
class implementing PlatformAudioData for the GStreamer platform,
used to pass the samples information.
* platform/mediastream/gstreamer/GStreamerAudioStreamDescription.h:
Added this class implementing AudioStreamDescription to export the
information about the audio stream to libwebrtc.
* platform/mediastream/gstreamer/GStreamerCaptureDevice.h: Added
this base class for the audio and video capturing devices, it
implements general WebKit CaptureDevice class.
* platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp:
Added this class that implements the system monitor to get the
list of available devices in the system. It uses GstDeviceMonitor
to handle the operation. It uses two singleton device managers one
for audio and another one for video, as required by the
RealtimeMediaSourceCenter design.
* platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h: Ditto.
* platform/mediastream/gstreamer/GStreamerCapturer.cpp: Added this
base class representing how GStreamer captures the media from the
input devices in the system. Two classes inherit from this one to
capture audio and video. It setups the GStreamer pipeline and adds
functions to control it.
* platform/mediastream/gstreamer/GStreamerCapturer.h: Ditto.
* platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp: Added.
Implements a subclass of GstBin as a source element that will contain several
GstAppSrc, basically one per MediaStreamTrackPrivate of the MediaStreamPrivate
passed in parameter. It adds Observers on the MediaStreamTracks and
pushes the data to the sources as required. The element implements the GstURIHandler
interface so it can be used in playbin. The MediaPlayerPrivateGStreamer is responsible
for passing the MediaStreamPrivate object to the source when required.
(WebCore::webkitMediaStreamSrcPadProbeCb): Event probe that fixes stream_start events (setting the ID etc)
and finally add src pads to the pipeline.
* platform/mediastream/gstreamer/GStreamerMediaStreamSource.h: Ditto.
* platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp:
Added this RealtimeMediaSource representing the source of the
video data for the GStreamer platform. It handles the settings and
capabilities of the source and creates the capturer used to
control the operation of the stream.
* platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h: Ditto.
* platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp: Added
this class that inherits from the GStreamerCapturer and controls
the GStreamer pipelines of the video streams of the system.
* platform/mediastream/gstreamer/GStreamerVideoCapturer.h: Ditto.
* platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp: Added. Implementation of a Mock capturer for audio stream.
Subclasses GStreamerAudioCapturer and wraps a MockRealtimeAudioSource so that the behaviour is the same a MockRealtimeAudioSource
but still the GStreamer implementation code paths are tested.
* platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h: Ditto.
* platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp: Added. Implementation of a Mock capturer for video stream.
Subclasses GStreamerVideoCapturer and wraps a MockRealtimeVideoSource so that the behaviour is the same a MockRealtimeVideoSource
but still the GStreamer implementation code paths are tested.
* platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h: Ditto.
* platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp:
Added this class that implements the key RealtimeMediaSourceCenter
functions to configure the base class for the platform, using the
other GStreamer classes.
* platform/mock/MockRealtimeAudioSource.cpp: Do not build ::create if GStreamer implementation is built
* platform/mock/MockRealtimeVideoSource.cpp: Do not build ::create if GStreamer implementation is built

2018-06-07 Brent Fulgham <[email protected]>

Handle Storage Access API calls in the absence of an attached frame
Expand Down
2 changes: 2 additions & 0 deletions Source/WebCore/PlatformGTK.cmake
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,14 @@ list(APPEND WebCore_INCLUDE_DIRECTORIES
"${WEBCORE_DIR}/platform/gtk"
"${WEBCORE_DIR}/platform/graphics/egl"
"${WEBCORE_DIR}/platform/graphics/glx"
"${WEBCORE_DIR}/platform/graphics/gstreamer"
"${WEBCORE_DIR}/platform/graphics/gtk"
"${WEBCORE_DIR}/platform/graphics/opengl"
"${WEBCORE_DIR}/platform/graphics/opentype"
"${WEBCORE_DIR}/platform/graphics/wayland"
"${WEBCORE_DIR}/platform/graphics/x11"
"${WEBCORE_DIR}/platform/mediastream/gtk"
"${WEBCORE_DIR}/platform/mediastream/gstreamer"
"${WEBCORE_DIR}/platform/mock/mediasource"
"${WEBCORE_DIR}/platform/network/gtk"
"${WEBCORE_DIR}/platform/network/soup"
Expand Down
11 changes: 11 additions & 0 deletions Source/WebCore/platform/GStreamer.cmake
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,17 @@ if (ENABLE_VIDEO OR ENABLE_WEB_AUDIO)
platform/graphics/gstreamer/mse/PlaybackPipeline.cpp
platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp
platform/graphics/gstreamer/mse/WebKitMediaSourceGStreamer.cpp

platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp
platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp
platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp
platform/mediastream/gstreamer/GStreamerCapturer.cpp
platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp
platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp
platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp
platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp
platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp
platform/mediastream/gstreamer/RealtimeMediaSourceCenterLibWebRTC.cpp
)

list(APPEND WebCore_SYSTEM_INCLUDE_DIRECTORIES
Expand Down
1 change: 1 addition & 0 deletions Source/WebCore/platform/audio/AudioStreamDescription.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ struct PlatformDescription {
enum {
None,
CAAudioStreamBasicType,
GStreamerAudioStreamDescription,
} type;
Variant<std::nullptr_t, const AudioStreamBasicDescription*> description;
};
Expand Down
1 change: 1 addition & 0 deletions Source/WebCore/platform/audio/PlatformAudioData.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ class PlatformAudioData {
enum class Kind {
None,
WebAudioBufferList,
GStreamerAudioData,
};

virtual Kind kind() const { return Kind::None; }
Expand Down
47 changes: 47 additions & 0 deletions Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
#include <mutex>
#include <wtf/glib/GLibUtilities.h>
#include <wtf/glib/GUniquePtr.h>
#include <wtf/glib/RunLoopSourcePriority.h>

#if ENABLE(VIDEO_TRACK) && USE(GSTREAMER_MPEGTS)
#define GST_USE_UNSTABLE_API
Expand Down Expand Up @@ -320,6 +321,52 @@ bool gstRegistryHasElementForMediaType(GList* elementFactories, const char* caps
return result;
}

static void simpleBusMessageCallback(GstBus*, GstMessage* message, GstBin* pipeline)
{
switch (GST_MESSAGE_TYPE(message)) {
case GST_MESSAGE_ERROR:
GST_ERROR_OBJECT(pipeline, "Got message: %" GST_PTR_FORMAT, message);
{
WTF::String dotFileName = String::format("%s_error", GST_OBJECT_NAME(pipeline));
GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(pipeline, GST_DEBUG_GRAPH_SHOW_ALL, dotFileName.utf8().data());
}
break;
case GST_MESSAGE_STATE_CHANGED:
if (GST_MESSAGE_SRC(message) == GST_OBJECT(pipeline)) {
GstState oldState, newState, pending;
gst_message_parse_state_changed(message, &oldState, &newState, &pending);

GST_INFO_OBJECT(pipeline, "State changed (old: %s, new: %s, pending: %s)",
gst_element_state_get_name(oldState),
gst_element_state_get_name(newState),
gst_element_state_get_name(pending));

WTF::String dotFileName = String::format("%s_%s_%s",
GST_OBJECT_NAME(pipeline),
gst_element_state_get_name(oldState),
gst_element_state_get_name(newState));

GST_DEBUG_BIN_TO_DOT_FILE_WITH_TS(GST_BIN(pipeline), GST_DEBUG_GRAPH_SHOW_ALL, dotFileName.utf8().data());
}
break;
default:
break;
}
}

void disconnectSimpleBusMessageCallback(GstElement *pipeline)
{
GRefPtr<GstBus> bus = adoptGRef(gst_pipeline_get_bus(GST_PIPELINE(pipeline)));
g_signal_handlers_disconnect_by_func(bus.get(), reinterpret_cast<gpointer>(simpleBusMessageCallback), pipeline);
}

void connectSimpleBusMessageCallback(GstElement *pipeline)
{
GRefPtr<GstBus> bus = adoptGRef(gst_pipeline_get_bus(GST_PIPELINE(pipeline)));
gst_bus_add_signal_watch_full(bus.get(), RunLoopSourcePriority::RunLoopDispatcher);
g_signal_connect(bus.get(), "message", G_CALLBACK(simpleBusMessageCallback), pipeline);
}

}

#endif // USE(GSTREAMER)
3 changes: 3 additions & 0 deletions Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,9 @@ inline GstClockTime toGstClockTime(const MediaTime &mediaTime)
}

bool gstRegistryHasElementForMediaType(GList* elementFactories, const char* capsString);
void connectSimpleBusMessageCallback(GstElement *pipeline);
void disconnectSimpleBusMessageCallback(GstElement *pipeline);

}

#ifndef GST_BUFFER_DTS_OR_PTS
Expand Down
Loading

0 comments on commit 8245958

Please sign in to comment.