Check out my
new book!
HTML5 Games book
HTML5 audio visualizations For a while now I've been playing around with music visualization, you can see some of my previous endeavours here. The canvas element gives us everything we need to draw all sorts of cool stuff in the browser but the audio part still needs Flash to be any fun. While the new HTML5 audio element does allow for audio playback, it doesn't give us much in terms of audio data information - which is crucial if the visualizations are to react to the music. In another experiment of mine I tried to solve it by doing a pre-analysis of the MP3 file and saving the data as JavaScript. The result was decent, I think, but having to pre-bake the data is a pretty severe limitation.

However, a few people have been hacking away at Firefox, attempting to provide just that functionality for the audio element. Some documentation of the new API is available here and if you're interested, you can follow the development in this bugzilla thread.

Anyway, I've been wanting to mess around with this and I figured that retro-fitting my "Pocket Full of Canvas" experiment to use HTML5 audio would be a fairly easy task. You can check out the result here.

At the moment you'll need a patched version of Firefox to get access to these extras but if you're not comfortable with building your own version, there are builds available here. It will still run in other (canvas-enabled) browsers but it will fall back to using SoundManager2 for the audio. While Flash audio is not that exciting anymore, it lets it run in eg. Chromium and Opera - and man, that's something else. Even if Firefox/Minefield has the upper hand with the audio stuff, it is feeling reaaally sluggish compared to those two (and you should definitely be using a nightly Minefield build). Chromium has always been speedy and Opera has gotten really crazy fast as well, running the visualizations in full-screen is just great (click the viz area to toggle).

And if you feel like playing around with the demos yourself, just click the "Show code" button and start hacking. I made a quick write-up of the very simple API here. The emphasis has been on making it easy to get from zero to something and not so much on making a lot of fancy functionality, so it is not very advanced.

Try it out: http://www.nihilogic.dk/labs/pocket_full_of_html5/

More examples:
⇓ 28 comments Anonymous

Too bad there's no Linux x64 build available :(

April 15, 2010 at 1:57 PM
Beben Koben

hmmm...what is this, i am not understand, just looking :)

April 20, 2010 at 2:29 PM
general fuzz

I'm seriously impressed that you were able to build in HTML 5. Good work! I clearly need to investigate its potential . . . .

April 27, 2010 at 8:25 AM
Jacob Seidelin

Great, glad you like it :-) And thanks for the tunes, they are most delicious.

April 27, 2010 at 12:00 PM
Scott

Somehow I missed seeing this earlier, great stuff; glad to see you're looking at the Audio data API patch for Firefox. I think it has a lot of potential, hopefully it can get in and adopted etc. It's a huge piece missing from the current Audio API spec (even if only at v1.0 itself), in my opinion.

Re: SM2, you may be able to get higher framerates if you flip on useHighPerformance and useFastPolling = true. (It doesn't look like either are on right now.) The useHighPerformance affects flash movie on-screen positioning (fixed vs. absolute, long story short, fixed = on-screen = better performance) - useFastPolling means a smaller interval within Flash for increased rate of whileplaying() calls, thus faster UI redraw.

Also FWIW, comment #3 re: "Epic" data is pure spam; identical posts all over the web elsewhere. ;)

April 27, 2010 at 3:01 PM
Jacob Seidelin

@Scott: Yes, I know about those settings but they (or at least one of them..) were giving me trouble with Firefox not correctly loading SM2 unless I turned them off (or turned Flash debugging on). Thanks though!

April 28, 2010 at 12:24 AM
Schill

Hm, crap; that's not supposed to happen. If you want to try it again (or run into the issue again in future), let me know; my goal is to have things "just work" in all configurations - the trick is that there happen to be lots of configurations. ;) If you can file a bug, even better!

JS-flash gets a bit weird in some cases, in Firefox if wmode is anything other than the default the movie will not load if the SWF is off-screen etc. (Again, should be handled, but I digress.)

April 29, 2010 at 5:31 PM
Scott

Hm, crap; that's not supposed to happen. If you want to try it again (or run into the issue again in future), let me know; my goal is to have things "just work" in all configurations - the trick is that there happen to be lots of configurations. ;) If you can file a bug, even better!

JS-flash gets a bit weird in some cases, in Firefox if wmode is anything other than the default the movie will not load if the SWF is off-screen etc. (Again, should be handled, but I digress.)

April 29, 2010 at 5:38 PM
www.richonet.com

Thanks for your content

Thanks,
www.oracledba.in
www.richonet.com

May 12, 2010 at 11:02 PM
Cena

Hi, I am giving this party, and I have got a beamer. I thought it would be nice to show some nice visualizations on the big screen.(And show track names.) Is there such a program?
And no window media player, since I have had some bad experiences with in the past.
Thanks

June 9, 2010 at 1:58 PM
Kampe

Amazing work man, keep it up!

June 13, 2010 at 2:04 AM
Xeallexx

Hey, Can you make this unreliant on the song thats being played so you could play your own music to it? I'd truly enjoy that. Another thing i would like to ask is if you could convert it to an embed-able format to use anywhere.

Other than that...

Wow! I really enjoy all projects posted on this blog and this one just blows my mind, thanks for all your effort!

June 17, 2010 at 5:22 PM
Havuz

thanks it is for me ı take it.

July 7, 2010 at 4:57 AM
Unknown

This is very interesting game...........I would like to visit once in a week....... I would like to play it again.......
online games

July 7, 2010 at 7:05 AM
Fotoğrafçı

thanks you for this wonderfull game. ı am playing it twice a day.

July 8, 2010 at 2:25 AM
Havuz Kimyasalları

thanks.

July 8, 2010 at 2:26 AM
Peruk

thanks you. ı am triying to work with asp.net are they ant good think for me.

July 12, 2010 at 2:03 AM
Social Security Number Lookup

I have tried this new HTML 5 and it is very powerful even the iphone 4 is using the same software. So keep exploring more on this software and let us all know about it.

July 28, 2010 at 10:24 PM
Ted Meyer

Hi, I've been combing the web for about 6 hours now, looking for an HTML5 audio visualizer. this is the closest i got, but i cant use it on my own site. if theres a way to download, please let me know, if not, are there any others that are available for export to my website? thanks.

September 16, 2010 at 7:00 PM
Fenerbahçe Beşiktaş Maçını İzle

thanks very much Fenerbahçe Beşiktaş Maçını İzle

September 17, 2010 at 4:25 AM
Unknown

The canvas element gives they everything necessary to draw all sorts of cool stuff in the browser, but the audio continues to flash a little funny. While the new HTML5 audio element allows audio playback, It does not give they much in terms of audio data - crucial if the visualisations are reacting to the music.

Free Poker Money

November 9, 2010 at 4:22 AM
Gray

Woah, I'm surprised this article doesn't have more relevant comments. This is great :/.

I'm trying to do something similar in AS3, but ran into the issue of getting the framebuffer of a playing mp3 hosted on another domain.

January 2, 2011 at 4:50 AM
knut

I want to visualize bass strokes. I'm not even close to know how to extract the bass level from the frameBuffer property on the MozAudioAvailable event. Can you please point me in the right direction? What are the numbers in the frameBuffer array? Which of them has to do with bass?

January 5, 2011 at 1:22 PM
Kosmös

very cool stuff - by the way, I am using the latest build of Firefox 3 (3.6.13) and the visualizations run without issue. It appears to me you're looping images at a variable framerate tied to events in the music, correct? From the size of the initial file transfer, it would appear this is the case?

January 27, 2011 at 8:05 PM
Anonymous

WOW!!

March 4, 2011 at 3:57 AM
misha

Thanks for sharing an amazing stuff ..cool..

vinyl siding installation virginia

March 24, 2011 at 10:32 AM
Gabriel Guerreiro

try:

(function() {
return function() {
quality(1);
decay(0.05);
zoom(.95)
move(0.025,0);
drawPath([[0,.2],[0.05,0.215],[0.05,0.785],[0,0.8]], false, "hsl(" + (((time * 10) % 360) >> 0) + ",100%,15%)");
for(var i = 0; i < 11; i++) drawRect(.005 * i, 0.5 - soundData.waveDataL[i * 12] * .5, .005, soundData.waveDataL[i * 12], "#FFF");
commit();
};
})();

December 24, 2011 at 7:24 AM
Gokul

Excellent work!! Could you please tell what do you mean by "pre-analysis of the MP3 file" ?? which software did you use to get the soundData ??

October 5, 2012 at 9:15 AM
Post a Comment