How to Convert Audio and Video to MuseScore Notation Using Klangio Piano2Notes
In this post, I will explain my experience converting a video to a MuseScore sheet music notation file using Klangio’s Piano2Notes software tool, and will provide my initial thoughts on how well the software works. The Klangio tool can also be used to convert MP3 audio.
One of my favorite jazz tunes is Strasbourg St. Denis, which was composed by jazz trumpet player Roy Hargrove, and I wanted to find sheet music for the piano part, played by Gerald CLayton on the recording. Fortunately, I was able to find a PDF transcription of the CLayton solo here, but not for the rest of the song.
The best resource I could find is this lovely arrangement of the song by James Douglas, in which he provided a scrolling score. In addition, I like his solo (a lot), but I really want to learn Gerald Clayton’s solo. So I wanted to create a MuseScore sheet music file containing Jame’s notation for most of the song, and to replace James’ (very nice) solo with the Clayton solo.
MuseScore is a popular, free, open-source music notation software for macOs, Windows and Linux. MuseScore has a playback feature.
My initial thought was to create a set of screenshots from James’ video, pausing as the score moved across the screen to grab another screenshot, print them out, and then, referring to the printouts, create a MuseScore file by hand. But that would be rather tedious, and I wondered if there is a way to just convert audio to MuseScore notation.
I found this YouTube video from music AI company Klangio, who markets the product Piano2Notes. You can drop a YouTube video URL or an MP3 audio file into Piano2Notes tool, which converts the audio into either a PDF file or a music XML file (your choice), each of which can be imported into MuseScore, as shown in the Klangio video, to create a MuseScore sheet music file and a separate PDF file.
The Klangio Piano2Notes conversion is free for video or MP3 files less than 30 seconds long. I paid Klangio four dollars (US) to convert the entire video file. The fee was well worth it, because I avoided having to annotate the entire song in MuseScore from screenshots! The YouTube video to MuseScore conversion was straightforward, and satisfying.
For illustration, the first four measures of the resultant seven page long MuseScore sheet are shown in this image.
A few things stand out to me about the machine-generated score. For example, notation is important to the musician reading and playing a score, and the machine-generated notation is not really how a human would notate music, particularly with respect to the way the chords are notated. For example, in measures 2 and 3, the lower chord notes in the treble staff should be notated in the bass staff, which would make the score much easier to read while playing. That is, in measure 3, on the “and” of beat 2, there is a Bb notated in the bass clef, and in the treble clef, F, Ab, C, Db and F are notated. The consensus way to notate is bass clef (Bb, F, Ab), and treble clef (D, Db, F). Proper notation facilitates effective reading and playing.
However, in terms of the sound, the machine generated score does captures James’ arrangement pretty well. Here is James playing (in the YoutTube video):
Here is what the machine generated MuseScore version sounds like:
Setting aside the inferior sound quality of the MuseScore playback, the takeway is that the machine generated score is pretty good.
However, the MuseScore rythm is different at places than James’ arrangement. Here is James playing at measure 16:
And here is the MuseScore version:
The difference is easily discerned: James’ version of those phrases is hip, and MuseScore’s version is not, such that the rythmic notation in those phrases would need to be edited.
An upper interval note (Ab) is seen in measures 2 and 4 of the machine generated score, which were not played by James in his arrangement. However, they WERE played in by Gerald Clayton in the Roy Hargrove quintet recording of Strasbourg St. Denis. Did Piano2Notes perhaps validate the score it generated against the original Roy Hargrove quintet recording, and add in the upper interval notes?
Eagle eye readers may observe that the MuseScore version starts 1.5 beats after James’ moving transcription starts. That is because, for whatever reason, Piano2Notes did not capture the first 1.5 beats of the video sound. Again, the MuseScore version could be tweaked to include the missing notes at the beginning of the score.
Over all, I think Piano2Notes is a fascinating tool that has great potential to accelerate music transcription projects. The user should, however, go in with a positive attitude and realize that while some tweaking will be required, but the tweaking effort will likely be less laborious than notating the entire score by hand in MuseScore.
As an aside, I found that if I want to work with screenshots of James’ score in support of the tweaking work, the notes appear in white against a black background, which makes visualizing the notes on the staff rather difficult.
Fortunately, the color of the notes and the background can easily be inverted using the Apple Preview Adjust Color feature, with the result that the staff is easily seen.
I am not paid by Klangio to discuss their product in this blog post. This page does contain an Amazon affiliate link, and I earn a commission if you make a purchase through the link, at no cost to you. As an Amazon associate, I earn from qualifying purchases.
