Skip to content

Commit

Permalink
Add conceptual documentation for Audio - Mixer
Browse files Browse the repository at this point in the history
NOTRY=true

Bug: webrtc:12570
Change-Id: Iece5588c5a45a8619afb32c812ff671a161e48f3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/215929
Reviewed-by: Henrik Lundin <[email protected]>
Reviewed-by: Artem Titov <[email protected]>
Commit-Queue: Alessio Bazzica <[email protected]>
Cr-Commit-Position: refs/heads/master@{#33806}
  • Loading branch information
alebzk authored and Commit Bot committed Apr 22, 2021
1 parent feb6eb9 commit 39e2385
Show file tree
Hide file tree
Showing 3 changed files with 57 additions and 1 deletion.
1 change: 1 addition & 0 deletions g3doc/sitemap.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
* AudioEngine
* [ADM](/modules/audio_device/g3doc/audio_device_module.md)
* [Audio Coding](/modules/audio_coding/g3doc/index.md)
* [Audio Mixer](/modules/audio_mixer/g3doc/index.md)
* AudioProcessingModule
* [APM](/modules/audio_processing/g3doc/audio_processing_module.md)
* Video
Expand Down
2 changes: 1 addition & 1 deletion modules/audio_mixer/OWNERS
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
aleloi@webrtc.org
alessiob@webrtc.org
[email protected]
55 changes: 55 additions & 0 deletions modules/audio_mixer/g3doc/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
<?% config.freshness.owner = 'alessiob' %?> <?% config.freshness.reviewed =
'2021-04-21' %?>

# The WebRTC Audio Mixer Module

The WebRTC audio mixer module is responsible for mixing multiple incoming audio
streams (sources) into a single audio stream (mix). It works with 10 ms frames,
it supports sample rates up to 48 kHz and up to 8 audio channels. The API is
defined in
[`api/audio/audio_mixer.h`](https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/api/audio/audio_mixer.h)
and it includes the definition of
[`AudioMixer::Source`](https://source.chromium.org/search?q=symbol:AudioMixer::Source%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h),
which describes an incoming audio stream, and the definition of
[`AudioMixer`](https://source.chromium.org/search?q=symbol:AudioMixer%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h),
which operates on a collection of
[`AudioMixer::Source`](https://source.chromium.org/search?q=symbol:AudioMixer::Source%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h)
objects to produce a mix.

## AudioMixer::Source

A source has different characteristic (e.g., sample rate, number of channels,
muted state) and it is identified by an SSRC[^1].
[`AudioMixer::Source::GetAudioFrameWithInfo()`](https://source.chromium.org/search?q=symbol:AudioMixer::Source::GetAudioFrameWithInfo%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h)
is used to retrieve the next 10 ms chunk of audio to be mixed.

[^1]: A synchronization source (SSRC) is the source of a stream of RTP packets,

identified by a 32-bit numeric SSRC identifier carried in the RTP header so as
not to be dependent upon the network address (see
[RFC 3550](https://tools.ietf.org/html/rfc3550#section-3)).

## AudioMixer

The interface allows to add and remove sources and the
[`AudioMixer::Mix()`](https://source.chromium.org/search?q=symbol:AudioMixer::Mix%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h)
method allows to generates a mix with the desired number of channels.

## WebRTC implementation

The interface is implemented in different parts of WebRTC:

* [`AudioMixer::Source`](https://source.chromium.org/search?q=symbol:AudioMixer::Source%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h):
[`audio/audio_receive_stream.h`](https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/audio/audio_receive_stream.h)
* [`AudioMixer`](https://source.chromium.org/search?q=symbol:AudioMixer%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h):
[`modules/audio_mixer/audio_mixer_impl.h`](https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/modules/audio_mixer/audio_mixer_impl.h)

[`AudioMixer`](https://source.chromium.org/search?q=symbol:AudioMixer%20file:third_party%2Fwebrtc%2Fapi%2Faudio%2Faudio_mixer.h)
is thread-safe. The output sample rate of the generated mix is automatically
assigned depending on the sample rate of the sources; whereas the number of
output channels is defined by the caller[^2]. Samples from the non-muted sources
are summed up and then a limiter is used to apply soft-clipping when needed.

[^2]: [`audio/utility/channel_mixer.h`](https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/audio/utility/channel_mixer.h)
is used to mix channels in the non-trivial cases - i.e., if the number of
channels for a source or the mix is greater than 3.

0 comments on commit 39e2385

Please sign in to comment.