Justin Salamon
  • Home
  • News
  • Research
  • Publications
  • Code/Data
  • Melody Extraction
  • PhD Thesis
  • Contact
    • Music
    • Music Technology

New Tools & Data for Soundscape Synthesis and Online Audio Annotation

10/10/2017

0 Comments

 
We're glad to announce the release of two open-source tools and a new dataset developed as part of the SONYC project we hope will be of use to the community: 

Scaper: a library for soundscape synthesis and augmentation
  • Automatically synthesize soundscapes with corresponding ground truth annotations 
  • Useful for running controlled ML experiments (ASR, sound event detection, bioacoustic species recognition, etc.)
  • Useful for running controlled experiments to assess human annotation performance
  • Potentially useful for generating data for source separation experiments (might require some extra code)
  • Potentially useful for generating ambisonic soundscapes (definitely requires some extra code)

AudioAnnotator: a javascript web interface for annotating audio data
  • Developed in collaboration with Edith Law and her students at the University of Waterloo's HCI Lab
  • A web interface that allows users to annotate audio recordings
  • Supports 3 types of visualization (waveform, spectrogram, invisible)
  • Useful for crowdsourcing audio labels and running controlled experiments on crowdsourcing audio  labels
  • Supports feedback mechanisms for providing real-time feedback to the user based on their annotations

URBAN-SED dataset: a new dataset for sound event detection
  • Includes 10,000 soundscapes with strongly labeled sound events generated using scaper
  • Totals almost 30 hours and includes close to 50,000 annotated sound events
  • Baseline convnet results on URBAN-SED are included in the scaper-paper.

Further information about scaper, the AudioAnnotator and the URBAN-SED dataset, including controlled experiments on the quality of crowdsourced human annotations as a function of visualization and soundscape complexity, are provided in the following papers:

Seeing sound: Investigating the effects of visualizations and complexity on crowdsourced audio annotations
M. Cartwright, A. Seals, J. Salamon, A. Williams, S. Mikloska, D. MacConnell, E. Law, J. Bello, and O. Nov.
Proceedings of the ACM on Human-Computer Interaction, 1(2), 2017.

Scaper: A Library for Soundscape Synthesis and Augmentation
J. Salamon, D. MacConnell, M. Cartwright, P. Li, and J. P. Bello.
In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, Oct. 2017.

We hope you find these tools useful and look forward to receiving your feedback (and pull requests!).

Cheers, on behalf of the entire team,
Justin Salamon & Mark Cartwright.
​--
Picture
Picture
Picture
Picture
0 Comments

Can the Internet of Things & AI Solve Urban Noise?

26/4/2017

0 Comments

 
New York City is a loud place. In fact, 9 of 10 adults in NYC are exposed to harmful levels of noise, which can lead to sleep loss, hearing loss and even heart disease. Can anything be done about it? The SONYC research project combines internet of things sensing technology, machine learning, data science and citizen science to tackle noise pollution head on at city scale. Come hear about a sensor that can recognize sounds like a human and how we plan to use it to improve quality of life in NYC.

The talk will be followed by my colleague and astrophysicist Federica Bianco with her talk:
​Twinkle twinkle little city-light!

When: Wednesday, April 26, 2017, 7:30pm  10:00pm
Where:  SingleCut Beersmiths, 19-33 37th Street NY, 11105 United States (map)
Registration: https://tasteofscience.org/ny-events/thescientificcity
Picture
0 Comments

Meet SONYC: Sounds of New York City

15/10/2015

0 Comments

 
Over the past two years I've been working together with a fantastic team of researchers on the SONYC: Sounds of New York City project. Check out our new video!

The objectives of SONYC are to create technological solutions for: (1) the systematic, constant monitoring of noise pollution at city scale; (2) the accurate description of acoustic environments in terms of its composing sources; (3) broadening citizen participation in noise reporting and mitigation; and (4) enabling city agencies to take effective, information-driven action for noise mitigation.

Noise pollution is one of the topmost quality of life issues for urban residents in the United States. It has been estimated that 9 out of 10 adults in New York City (NYC) are exposed to excessive noise levels, i.e. beyond the limit of what the EPA considers to be harmful. When applied to U.S. cities of more than 4 million inhabitants, such estimates extend to over 72 million urban residents.

To learn more about the SONYC project please check out the project website: wp.nyu.edu/sonyc

To read our publications on automatic urban sound classification as well as the development of low-cost, high-quality acoustic sensors, check out the project's publication page: wp.nyu.edu/sonyc/publications
0 Comments

Feature Learning with Deep Scattering for Urban Sound Analysis

10/6/2015

0 Comments

 
Picture
In this paper we evaluate the scattering transform as an alternative signal representation to the mel-spectrogram in the context of unsupervised feature learning for urban sound classification. We show that we can obtain comparable (or better) performance using the scattering transform whilst reducing both the amount of training data required for feature learning and the size of the learned codebook by an order of magnitude. In both cases the improvement is attributed to the local phase invariance of the representation. We also observe improved classification of sources in the background of the auditory scene, a result that provides further support for the importance of temporal modulation in sound segregation.

For further details please see our paper:

J. Salamon and J. P. Bello. "Feature Learning with Deep Scattering for Urban Sound Analysis", 2015 European Signal Processing Conference (EUSIPCO), Nice, France, August 2015.
[EURASIP][PDF][BibTex]

0 Comments

Announcing the  Urban  Sound  dataset and taxonomy

23/10/2014

0 Comments

 
Picture
 We are pleased to announce the release of UrbanSound, a dataset containing 27 hours of field-recordings with over 3000 labelled sound source occurrences from 10 sound classes. The dataset focuses on sounds that occur in urban acoustic environments.

To facilitate comparable research on urban sound source classification, we are also releasing a second version of this dataset, UrbanSound8K, with 8732 excerpts limited to 4 seconds (also with source labels), and pre-sorted into 10 stratified folds. In addition to the source ID both datasets also include a (subjective) salience label for each source occurrence: foreground / background.

The datasets are released for research purposes under a Creative Commons Attribution Noncommercial License, and are available online at the dataset companion website:


http://urbansounddataset.weebly.com/

This companion website also contains further information about each dataset, including the Urban Sound Taxonomy from which the 10 sound classes in this dataset were selected.

The datasets and taxonomy will be presented at the ACM Multimedia 2014 conference in Orlando in a couple of weeks. For those interested, please see our paper:

J. Salamon, C. Jacoby and J. P. Bello, "A Dataset and Taxonomy for Urban Sound Research", in Proc. 22nd ACM International Conference on Multimedia, Orlando USA, Nov. 2014.

For those attending ISMIR 2014 next week, I will also be there if you would like to discuss the datasets and taxonomy.

I hope you find the datasets useful for your work and look forward to seeing some of you at ISMIR and ACM-MM in the coming weeks!

0 Comments

    NEWS

    Machine listening research, code, data & hacks!

    Archives

    March 2023
    April 2022
    November 2021
    October 2021
    June 2021
    January 2021
    October 2020
    June 2020
    May 2020
    April 2020
    January 2020
    November 2019
    October 2019
    June 2019
    May 2019
    March 2019
    February 2019
    January 2019
    November 2018
    October 2018
    August 2018
    July 2018
    May 2018
    April 2018
    February 2018
    October 2017
    August 2017
    July 2017
    June 2017
    April 2017
    March 2017
    January 2017
    December 2016
    November 2016
    October 2016
    August 2016
    June 2016
    May 2016
    April 2016
    February 2016
    January 2016
    November 2015
    October 2015
    July 2015
    June 2015
    April 2015
    February 2015
    November 2014
    October 2014
    September 2014
    June 2014
    April 2014
    March 2014
    February 2014
    December 2013
    September 2013
    July 2013
    May 2013
    February 2013
    January 2013
    December 2012
    November 2012
    October 2012
    August 2012
    July 2012
    June 2012

    Categories

    All
    ACM MM'13
    ACM MM'14
    Acoustic Ecology
    Acoustic Event Detection
    Acoustic Sensing
    AES
    Applied Acoustics
    Article
    Audio-annotator
    Audio To Midi
    Auditory Scene Analysis
    Avian
    Award
    Baseball
    Beer
    Best Oral Presentation
    Best Paper Award
    Best Student Paper Award
    BigApps
    Bioacoustics
    BirdVox
    Book
    Chapter
    CHI
    Citizen Science
    Classification
    Computer Vision
    Conference
    Connected Cities
    Convolutional Neural Networks
    Cornell Lab Of Ornithology
    Coursera
    Cover Detection
    CREPE
    Crowdcrafting
    Crowdsourcing
    CUSP
    CVPR
    Data Augmentation
    Data Science
    Dataset
    Data Structures
    Dcase
    Deep Learning
    Domain
    Education
    Entrepreneurship
    Environmental Sound
    Essentia
    Eusipco
    Eusipco2015
    Evaluation
    Few-shot Learning
    Flight Calls
    Girl Scouts
    Grant
    Hackathon
    Hackday
    Hackfest
    HCI
    Hildegard Von Bingen
    ICASSP
    ICASSP 2020
    IEEE Signal Processing Letters
    Ieee Spm
    Indian Classical Music
    Interface
    Interspeech
    Interview
    Ismir 2012
    Ismir2014
    Ismir2015
    Ismir2016
    Ismir2017
    Ismir2020
    ITP
    Jams
    Javascript
    JNMR
    Journal
    Machine Learning
    Machine Listening
    Map
    Media
    Melodia
    Melody Extraction
    Metric Learning
    Midi
    Migration Monitoring
    MIR
    Mir_eval
    MOOC
    MTG-QBH
    Music Informatics
    Music Information Retrieval
    Music Similarity
    National Science Foundation
    Neumerator
    New York Times
    Noise Pollution
    Notebook
    NPR
    NSF
    NYC
    NYU
    Open Source
    Pitch
    Pitch Contours
    Pitch Tracking
    Plos One
    Plug In
    Plug-in
    Presentation
    Press
    PRI
    Prosody
    Publication
    Python
    Query By Humming
    Query-by-humming
    Radio
    Representation Learning
    Research
    Robots
    Scaper
    Science And The City
    Science Friday
    Self-supervision
    Sensor Network
    Sensors
    Sight And Sound Workshop
    Smart Cities
    Software
    SONYC
    Sound Classification
    Sound Education
    Sound Event Detection
    Soundscape
    Sounds Of New York City
    Sound Workshop
    Speech
    STEM
    Synthesis
    Taste Of Science
    Taxonomy
    Technical Report
    Time Series
    Tonic ID
    Tony
    Tutorial
    Unsupervised Feature Learning
    Urban
    Urban Sound Analysis
    Urban Sound Tagging
    Vamp
    Version Identification
    Visualization
    Vocaloid
    Vocoder
    Warblers
    Wav To Midi
    Welcome
    Wired
    WNYC
    Women In Science
    Workshop
    World Domination
    Wsf14
    Youtube

    RSS Feed

Powered by Create your own unique website with customizable templates.
  • Home
  • News
  • Research
  • Publications
  • Code/Data
  • Melody Extraction
  • PhD Thesis
  • Contact
    • Music
    • Music Technology