Category: Environmental Sound

New Tools & Data for Soundscape Synthesis and Online Audio Annotation

10/10/2017

We're glad to announce the release of two open-source tools and a new dataset developed as part of the SONYC project we hope will be of use to the community:

Scaper: a library for soundscape synthesis and augmentation

Automatically synthesize soundscapes with corresponding ground truth annotations
Useful for running controlled ML experiments (ASR, sound event detection, bioacoustic species recognition, etc.)
Useful for running controlled experiments to assess human annotation performance
Potentially useful for generating data for source separation experiments (might require some extra code)
Potentially useful for generating ambisonic soundscapes (definitely requires some extra code)

AudioAnnotator: a javascript web interface for annotating audio data

Developed in collaboration with Edith Law and her students at the University of Waterloo's HCI Lab
A web interface that allows users to annotate audio recordings
Supports 3 types of visualization (waveform, spectrogram, invisible)
Useful for crowdsourcing audio labels and running controlled experiments on crowdsourcing audio labels
Supports feedback mechanisms for providing real-time feedback to the user based on their annotations

URBAN-SED dataset: a new dataset for sound event detection

Includes 10,000 soundscapes with strongly labeled sound events generated using scaper
Totals almost 30 hours and includes close to 50,000 annotated sound events
Baseline convnet results on URBAN-SED are included in the scaper-paper.

Further information about scaper, the AudioAnnotator and the URBAN-SED dataset, including controlled experiments on the quality of crowdsourced human annotations as a function of visualization and soundscape complexity, are provided in the following papers:

Seeing sound: Investigating the effects of visualizations and complexity on crowdsourced audio annotations
M. Cartwright, A. Seals, J. Salamon, A. Williams, S. Mikloska, D. MacConnell, E. Law, J. Bello, and O. Nov.
Proceedings of the ACM on Human-Computer Interaction, 1(2), 2017.

Scaper: A Library for Soundscape Synthesis and Augmentation
J. Salamon, D. MacConnell, M. Cartwright, P. Li, and J. P. Bello.
In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, Oct. 2017.

We hope you find these tools useful and look forward to receiving your feedback (and pull requests!).

Cheers, on behalf of the entire team,
Justin Salamon & Mark Cartwright.
â€‹--

0 Comments

New Book Chapter: Sound Analysis in Smart Cities

3/10/2017

0 Comments

This chapter introduces the concept of smart cities and discusses the importance of sound as a source of information about urban life. It describes a wide range of applications for the computational analysis of urban sounds and focuses on two high-impact areas, audio surveillance, and noise pollution monitoring, which sit at the intersection of dense sensor networks and machine listening. For sensor networks we focus on the pros and cons of mobile versus static sensing strategies, and the description of a low-cost solution to acoustic sensing that supports distributed machine listening. For sound event detection and classification we focus on the challenges presented by this task, solutions including feature design and learning strategies, and how a combination of convolutional networks and data augmentation result in the current state of the art. We close with a discussion about the potential and challenges of mobile sensing, the limitations imposed by the data currently available for research, and a few areas for future exploration.

Sound analysis in smart cities
J. P. Bello, C. Mydlarz, and J. Salamon.
In T. Virtanen, M. D. Plumbley, and D. P. W. Ellis, editors, Computational Analysis of Sound Scenes and Events, pages 373–397. Springer International Publishing, 2018.
[Springer][PDF][BibTeX]

0 Comments

Can the Internet of Things & AI Solve Urban Noise?

26/4/2017

0 Comments

New York City is a loud place. In fact, 9 of 10 adults in NYC are exposed to harmful levels of noise, which can lead to sleep loss, hearing loss and even heart disease. Can anything be done about it? The SONYC research project combines internet of things sensing technology, machine learning, data science and citizen science to tackle noise pollution head on at city scale. Come hear about a sensor that can recognize sounds like a human and how we plan to use it to improve quality of life in NYC.

The talk will be followed by my colleague and astrophysicist Federica Bianco with her talk:
Twinkle twinkle little city-light!

When: Wednesday, April 26, 2017, 7:30pm 10:00pm
Where: SingleCut Beersmiths, 19-33 37th Street NY, 11105 United States (map)
Registration: https://tasteofscience.org/ny-events/thescientificcity

0 Comments

Deep Convolutional Neural Networks and Data Augmentation For Environmental Sound Classification

20/1/2017

0 Comments

The ability of deep convolutional neural networks (CNN) to learn discriminative spectro-temporal patterns makes them well suited to environmental sound classification. However, the relative scarcity of labeled data has impeded the exploitation of this family of high-capacity models. This study has two primary contributions: first, we propose a deep convolutional neural network architecture for environmental sound classification. Second, we propose the use of audio data augmentation for overcoming the problem of data scarcity and explore the influence of different augmentations on the performance of the proposed CNN architecture. Combined with data augmentation, the proposed model produces state-of-the-art results for environmental sound classification. We show that the improved performance stems from the combination of a deep, high-capacity model and an augmented training set: this combination outperforms both the proposed CNN without augmentation and a “shallow” dictionary learning model with augmentation. Finally, we examine the influence of each augmentation on the model’s classification accuracy for each class, and observe that the accuracy for each class is influenced differently by each augmentation, suggesting that the performance of the model could be improved further by applying class-conditional data augmentation.

For further details see our paper:

Deep Convolutional Neural Networks and Data Augmentation For Environmental Sound Classification
J. Salamon and J. P. Bello
IEEE Signal Processing Letters, In Press, 2017.
[IEEE][PDF][BibTeX][Copyright]

0 Comments

SONYC featured in New York Times, NPR, Wired and more

7/11/2016

0 Comments

Today SONYC was featured on several major news outlets including the New York Times, NPR and Wired! This follows NYU's press release about the official launch of the SONYC project.

Needless to say I'm thrilled about the coverage the project's launch is receiving. Hopefully it is a sign of the great things yet to come from this project, though, I should note, it has already resulted in several scientific publications.

Here's the complete list of media articles (that I could find) covering SONYC. The WNYC radio segment includes a few words from yours truly :)

To Create a Quieter City, They’re Recording the Sounds of New York

BBC World Service - World Update (first minute, then from 36:21)

Mapping New York City's Excessively Loud Sounds

New York, come usare i microfoni per una città più silenziosa

Scientists Are Tracking New York Noisiness in Order to Quiet It Down

NYU Scientists are Trying to Reduce Noise Pollution in New York City

Researchers Are Recording New York to Make it Quieter

Sounds of New York City (German Public Radio)

NYC’s $5 Million Noise Pollution Project

Mapping the Sounds of New York City Streets

New UrbanEars project has NYU teaming up with Ohio State to battle noise pollution

NYU Launches Research Initiative to Combat NYC Noise Pollution

Smart microphones are recording city sounds to help create a quieter New York

NYU Moves Forward with Study of City Noise

How to Take on NYC’s Scary Noise Problem

Research Initiative Looks to Tame Urban Noise Pollution

If you're interested to learn more about the SONYC project have a look at the SONYC website. You can also check out the SONYC intro video:

0 Comments

SONYC awarded major grant by the National Science Foundation

7/11/2016

0 Comments

I'm extremely excited to report that our Sounds of New York City (SONYC) project has been granted a Frontier award from the National Science Foundation (NSF) as part of its initiative to advance research in cyber-physical systems as detailed in the NSF’s press release.

NYU has issued a press release providing further information about the SONYC project and the award. From the NYU press release:

The project – which involves large-scale noise monitoring – leverages the latest in machine learning technology, big data analysis, and citizen science reporting to more effectively monitor, analyze, and mitigate urban noise pollution. Known as Sounds of New York City (SONYC), this multi-year project has received a $4.6 million grant from the National Science Foundation and has the support of City health and environmental agencies.

Further information about the project project can be found on the SONYC website. You can also check out the SONYC intro video:

0 Comments

New Tools & Data for Soundscape Synthesis and Online Audio Annotation

New Book Chapter: Sound Analysis in Smart Cities

Can the Internet of Things & AI Solve Urban Noise?

Deep Convolutional Neural Networks and Data Augmentation For Environmental Sound Classification

SONYC featured in New York Times, NPR, Wired and more

SONYC awarded major grant by the National Science Foundation

NEWS

Archives

Categories