Deep Xi: A Deep Learning Approach to A Priori SNR Estimation

Deep Xi (where the Greek letter 'xi' or ξ is ponounced /zaɪ/) is a deep learning approach to a priori SNR estimation that was proposed in [1]. It can be used by minimum mean-square error (MMSE) approaches to speech enhancement like the MMSE short-time spectral amplitude (MMSE-STSA) estimator, the MMSE log-spectral amplitude (MMSE-LSA) estimator, and the Wiener filter (WF) approach. It can also be used to estimate the ideal ratio mask (IRM) and the ideal binary mask (IBM). Deep Xi can be used as a front-end for robust ASR, as shown in Figure 1. DeepXi is implemented in TensorFlow.

Figure 1: Deep Xi used as a front-end for robust ASR. The back-end (Deep Speech) is available here. The noisy speech magnitude spectrogram, as shown in (a), is a mixture of clean speech with voice babble noise at an SNR level of -5 dB, and is the input to Deep Xi. Deep Xi estimates the a priori SNR, as shown in (b). The a priori SNR estimate is used to compute an MMSE approach gain function, which is multiplied elementwise with the noisy speech magnitude spectrum to produce the clean speech magnitude spectrum estimate, as shown in (c). MFCCs are computed from the estimated clean speech magnitude spectrogram, producing the estimated clean speech cepstrogram, as shown in (d). The back-end system, Deep Speech, computes the hypothesis transcript, from the estimated clean speech cepstrogram, as shown in (e).

Figure 2: ResBLSTM a priori SNR estimator.

Figure 3: ResLSTM a priori SNR estimator.

Current Models

The scripts for each of the following models can be found in the ./ver directory:

• c2.7a is a TCN (temporal convolutional network) that has 2 million parameters.
• c1.13a is a ResLSTM (residual long short-term memory network) with 10.8 million parameters, as shown in Figure 2.
• n1.9a is a ResBLSTM (residual bidirectional long short-term memory network) with 21.3 million parameters, as shown in Figure 3.

A trained model for c2.7a can be found in the ./model directory. Trained models for c1.13a and n1.9a will be made available shortly.

Installation

It is recommended to use a virtual environment for installation.

Prerequisites:

• TensorFlow r1.11 (installed in a virtual environment). Will be updated to r2.0 in the near future.
• Python3
• MATLAB (only required for .mat output files)

To install:

1. git clone https://github.com/anicolson/DeepXi.git
2. pip install -r requirements.txt

How to Use the Deep Xi Scripts

Inference:

cd ver/c2/7/a
python3 deepxi.py --test 1 --out_type y --gain mmse-lsa --gpu 0

y for --out_type specifies enhanced speech .wav output. mmse-lsa specifies the used gain function (others include mmse-stsa, wf, irm, ibm, srwf, cwf).

Training:

cd ver/c2/7/a
python3 deepxi.py --train 1 --verbose 1 --gpu 0

Other options can be found in the deepxi.py script.

Datasets Used For Training

The .wav files used for training are single-channel, with a sampling frequency of 16 kHz.

The following speech datasets were used:

• The train-clean-100 set from Librispeech corpus, which can be found here.
• The CSTR VCTK corpus, which can be found here.
• The si and sx training sets from the TIMIT corpus, which can be found here (not open source).

The following noise datasets were used:

• The QUT-NOISE dataset, which can be found here.
• The Nonspeech dataset, which can be found here.
• The Environemental Background Noise dataset, which can be found here.
• The noise set from the MUSAN corpus, which can be found here.
• Multiple packs from the FreeSound website, which can be found here

References

[1] A. Nicolson and K. K. Paliwal, "Deep Learning For Minimum Mean-Square Error Approaches to Speech Enhancement", Submitted with revisions to Speech Communication.