Hardware

Browse all details of the hardware used for the Vibravox project.

Software

Browse all details of the software used for the Vibravox project.

Noise

Noise for the QiN step

After standardizing the sampling frequency, channels and format of the AudioSet files. Single-channel noise is obtained from 32 10-second samples of the following 90 classes:

[‘Drill’, ‘Truck’, ‘Cheering’, ‘Tools’, ‘Civil defense siren’, ‘Police car (siren)’, ‘Helicopter’, ‘Vibration’, ‘Drum kit’, ‘Telephone bell ringing’, ‘Drum roll’, ‘Waves, surf’, ‘Emergency vehicle’, ‘Siren’, ‘Aircraft engine’, ‘Idling’, ‘Fixed-wing aircraft, airplane’, ‘Vehicle horn, car horn, honking’, ‘Jet engine’, ‘Light engine (high frequency)’, ‘Heavy engine (low frequency)’, ‘Engine knocking’, ‘Engine starting’, ‘Motorboat, speedboat’, ‘Motor vehicle (road)’, ‘Motorcycle’, ‘Boat, Water vehicle’, ‘Fireworks’, ‘Stream’, ‘Train horn’, ‘Foghorn’, ‘Chainsaw’, ‘Wind noise (microphone)’, ‘Wind’, ‘Traffic noise, roadway noise’, ‘Environmental noise’, ‘Race car, auto racing’, ‘Railroad car, train wagon’, ‘Scratching (performance technique)’, ‘Vacuum cleaner’, ‘Tubular bells’, ‘Church bell’, ‘Jingle bell’, ‘Car alarm’, ‘Car passing by’, ‘Alarm’, ‘Alarm clock’, ‘Smoke detector, smoke alarm’, ‘Fire alarm’, ‘Thunderstorm’, ‘Hammer’, ‘Jackhammer’, ‘Steam whistle’, ‘Distortion’, ‘Air brake’, ‘Sewing machine’, ‘Applause’, ‘Drum machine’, “Dental drill, dentist’s drill”, ‘Gunshot, gunfire’, ‘Machine gun’, ‘Cap gun’, ‘Bee, wasp, etc.’, ‘Beep, bleep’, ‘Frying (food)’, ‘Sampler’, ‘Meow’, ‘Toilet flush’, ‘Whistling’, ‘Glass’, ‘Coo’, ‘Mechanisms’, ‘Rub’, ‘Boom’, ‘Frog’, ‘Coin (dropping)’, ‘Crowd’, ‘Crackle’, ‘Theremin’, ‘Whoosh, swoosh, swish’, ‘Raindrop’, ‘Engine’, ‘Rail transport’, ‘Vehicle’, ‘Drum’, ‘Car’, ‘Animal’, ‘Inside, small room’, ‘Laughter’, ‘Train’]

This represents 8 hours of audio.

Loudness normalization

from ffmpeg_normalize import FFmpegNormalize

normalizer = FFmpegNormalize(normalization_type="ebu",
                            target_level = -15.0,
                            loudness_range_target=5,
                            true_peak = -2,
                            dynamic = True,
                            print_stats=False,
                            sample_rate = 48_000,
                            progress=True)

normalizer.add_media_file(input_file='tot.rf64',
                          output_file='tot_normalized.wav')
normalizer.run_normalization()

Spatialization

The direction of sound is sampled uniformly on the unit sphere using the inverse cumulative distribution function.

Noise for the SiN step

The noise used for the final stage of the recording was captured with a ZYLIA ZR-1 Portable. It consists of applause, demonstrations and opera for a total of 2h40.

Recording protocol

Procedure

The recording process consists of four steps:

• Speak in Silence: For a duration of 15 minutes, the participant reads sentences sourced from the French Wikipedia. Each utterance generates a new recording and the transcriptions are preserved.

• Quiet in Noise: During 2 minutes and 24 seconds, the participant remains silent in a noisy environment created from the AudioSet samples. These samples have been selected from relevant classes, normalized in loudness, pseudo-spatialized and are played from random directions using a spatialization sphere equipped with 56 loudspeakers. The objective of this phase is to gather realistic background noises that will be combined with the Speak in Silence recordings to maintain a clean reference.

• Quiet in Silence: The procedure is repeated for 54 seconds in complete silence to record solely physiological and microphone noises. These samples can be valuable for tasks such as heart rate tracking or simply analyzing the noise properties of the various microphones.

• Speak in Noise: The final phase (54 seconds) will primarily serve to test the different systems (Speech Enhancement, Automatic Speech Recognition, Speaker Identification) that will be developed based on the recordings from the first three phases. This real-world testing will provide valuable insights into the performance and effectiveness of these systems in practical scenarios. The noise was recorded using the ZYLIA ZR-1 Portable Recorder from spatialized scenes and replayed in the spatialization sphere with ambisonic processing.

Post-processing

Please refer to the Vibravox paper for more information on the dataset curation.

Analysis

Coherence functions

The coherence functions of all microphones are shown in the figure below during an active speech phase.

Consent form

Ensure compliance with GDPR

A consent form for participation in the VibraVox dataset has been drafted and approved by the Cnam lawyer. This form mentions that the dataset will be released under the Creative Commons BY 4.0 license, which allows anyone to share and adapt the data under the condition that the original authors are cited. All Cnil requirements have been checked, including the right to oblivion. This form must be signed by each VibraVox participant.

The consent form

The voice recordings collected during this experiment are intended to be used for research into noise-resilient microphones, as part of the thesis project of Mr. Julien HAURET, PhD candidate at the Conservatoire national des arts et métiers (Cnam) (julien.hauret@lecnam.net).

The purpose of this form is to obtain the consent of each of the participants in this project to the collection and storage of their voice recordings, necessary for the production of the results of this research project.

The recordings collected will be anonymized and shared publicly at vibravox.cnam.fr under a Creative Commons BY 4.0 license, it being understood that this license allows anyone to share and adapt your data.

This processing of personal data is recorded in a computerized file by Cnam.

The data may be kept by Cnam for up to 50 years.

The recipients of the data collected will be the above-mentioned doctoral researcher and his thesis supervisor, Mr. Éric Bavu.

In accordance with the General Data Protection Regulation EU 2016/679 (GDPR) and laws no. 2018-493 of June 20, 2018 relating to the protection of personal data and no. 2004-801 of August 6, 2004 relating to the protection of individuals with regard to the processing of personal data and amending law no. 78-17 of January 6, 1978 relating to data processing, you have the right to access, rectify, oppose, delete, limit and port your personal data, i.e. your voice recordings.

To exercise these rights, or if you have any questions about the processing of your data under this scheme, please contact vibravox@cnam.fr. Although your right to be forgotten remains applicable for the duration of the data retention period, we advise you to exercise this right before the final publication of the database, which is scheduled for 01/10/2023.

If, after contacting us, you feel that your rights have not been respected, you can contact the Cnam-établissement public data protection officer directly at ep_dpo@lecnam.netonmicrosoft.com. You can also lodge a complaint with Cnil.