Harmonious Monk

A Northwestern University EECS352 Final Project
Developed by Daniel Felix-Kim, Kevin Li, and Michelle Carbo.
Instructor: Professor Bryan Pardo

Record Yourself

Record yourself singing or talking. Don’t worry, we won’t judge the quality of your voice.

Example Input

Run the Program

Open the command line and let Harmonious Monk work it’s magic!

Download

Listen and Enjoy

Kick back and enjoy your “jazzified” voice while drinking a scotch by the fireplace.

Example Output

Watch the video that inspired the creation of Harmonious Monk

About

Learn more about the mission and the development of the HM project.

Why We Made HM

Harmonious Monk (HM), allows a user to instantly become a jazz composer by automatically harmonizing speech or a melody. Though pitch tracking and harmonization have been done before, HM presents a different approach: harmonization based on jazz standards. Named after the great Thelonious Monk, HM transcribes speech into pitches and harmonizes it with jazz chords, drawing its chordal vocabulary and harmonic progressions from the tunes it analyzed from the iRealPro database.

How HM Works

HM transcribes a monophonic melody or speech using the Yin pitch tracking algorithm, and harmonizes the transcription with jazz chords. We built hidden Markov models (HMMs) based on a large sample of jazz standards, and these HMMs determine how the melody is harmonized. HM then uses the Viterbi algorithm to pick the most probable chord progression. The final product is the original audio with a sequence of harmonized piano chords laid underneath.

How HM Was Built/Tested

An XML parser was constructed to extract the key/chords from a database of jazz standards. After transposing all of the tunes to the same key, HM derives transition probabilities from chord to chord, which are used to create hidden Markov models. To transcribe the input melody, HM uses the Yin pitch tracker and the librosa onset detector to output an array of notes and frames, which are then used to find the most probable chords associated with the given melody notes using the Viterbi algorithm. HM then adds pre-recorded piano chords at the onset frames and writes a new .wav file. HM was tested using melodies from jazz standards, and the onsets were compared to ground truth onsets from Audacity.



Top: An example sound wave of a singing male voice with onsets marked.
Mid: The results of the pitch tracker, with 0 = C pitch class.
Bot: Pitch tracker confidence level, with a threshold of 0.8.

Results

The results of our methodology were quite successful. HM is able to harmonize any sung melody or spoken sentence with relative speed, accuracy, and ease. The jazz chord progressions sound like real chord progressions that are even recognizable from certain tunes. Our pitch tracker and onset detection work accurately as long as the onsets are not too frequent.

Resources

iRealPro: database of jazz standards
Aubio: pitch tracker source
Librosa: onset detector source
HMMs in Python: hidden Markov model source
XML Parser: reference used for xml parsing
Viterbi Algorithm: reference used for our Viterbi implementation

Testimonials

Hear what users have to say about Harmonious Monk!

I’ve been trying to make it in the jazz industry for decades. After using Harmonious Monk, I finally made it! I’m now recognized as a real jazz musician!

KennyG

Kenny G., Aspiring Jazz Musician

The news has been reporting that the American people don’t like my speeches. FAKE NEWS! SAD! Anyways, I was looking for a way to make my speeches more interesting. Thanks Harmonious Monk. Make America Jazz again!

donald

Donald Trump, POTUS

When I heard what was coming out of Harmonious Monk, I turned in my grave… in a good way! This program is killin’!

Thelonious

Thelonious Monk, Jazz Legend

Fun Statistics

119 Hours Spent
682 Lines of Code Written
132 Unique Piano Chords Recorded
32 Jazz Standards Analyzed

Meet the Team

We are a team of Northwestern University undergraduate students.
Learn more about us here.

Daniel

Daniel Felix-Kim

Python Developer

Daniel is an Electrical Engineering and Guitar Dual Degree student. This past summer, he worked as a Product Development Intern at Ford Motor Company.

Kevin

Kevin Li

Python and Web Developer

Kevin is a Computer Science and Music double major. This past summer, he worked as a Process Control Systems Engineering Intern at Arconic

Michelle

Michelle Carbo

Python Developer

Michelle is a Computer Science major. This past summer, she remodeled and worked on the Northwestern University McCormick website.

Get in touch

Questions, comments, concerns? Shoot us a message.

  • Daniel: danielfelixkim2016@u.northwestern.edu
  • Kevin: kevinli2017@u.northwestern.edu
  • Michelle: michellecarbo2016@u.northwestern.edu