HTML5 audio visualizations
For a while now I've been playing around with music visualization, you can see some of my previous endeavours here. The canvas element gives us everything we need to draw all sorts of cool stuff in the browser but the audio part still needs Flash to be any fun. While the new HTML5 audio element does allow for audio playback, it doesn't give us much in terms of audio data information - which is crucial if the visualizations are to react to the music. In another experiment of mine I tried to solve it by doing a pre-analysis of the MP3 file and saving the data as JavaScript. The result was decent, I think, but having to pre-bake the data is a pretty severe limitation.
However, a few people have been hacking away at Firefox, attempting to provide just that functionality for the audio element. Some documentation of the new API is available here and if you're interested, you can follow the development in this bugzilla thread.
Anyway, I've been wanting to mess around with this and I figured that retro-fitting my "Pocket Full of Canvas" experiment to use HTML5 audio would be a fairly easy task. You can check out the result here.
At the moment you'll need a patched version of Firefox to get access to these extras but if you're not comfortable with building your own version, there are builds available here. It will still run in other (canvas-enabled) browsers but it will fall back to using SoundManager2 for the audio. While Flash audio is not that exciting anymore, it lets it run in eg. Chromium and Opera - and man, that's something else. Even if Firefox/Minefield has the upper hand with the audio stuff, it is feeling reaaally sluggish compared to those two (and you should definitely be using a nightly Minefield build). Chromium has always been speedy and Opera has gotten really crazy fast as well, running the visualizations in full-screen is just great (click the viz area to toggle).
And if you feel like playing around with the demos yourself, just click the "Show code" button and start hacking. I made a quick write-up of the very simple API here. The emphasis has been on making it easy to get from zero to something and not so much on making a lot of fancy functionality, so it is not very advanced.
Try it out: http://www.nihilogic.dk/labs/pocket_full_of_html5/
More examples:
For a while now I've been playing around with music visualization, you can see some of my previous endeavours here. The canvas element gives us everything we need to draw all sorts of cool stuff in the browser but the audio part still needs Flash to be any fun. While the new HTML5 audio element does allow for audio playback, it doesn't give us much in terms of audio data information - which is crucial if the visualizations are to react to the music. In another experiment of mine I tried to solve it by doing a pre-analysis of the MP3 file and saving the data as JavaScript. The result was decent, I think, but having to pre-bake the data is a pretty severe limitation.
However, a few people have been hacking away at Firefox, attempting to provide just that functionality for the audio element. Some documentation of the new API is available here and if you're interested, you can follow the development in this bugzilla thread.
Anyway, I've been wanting to mess around with this and I figured that retro-fitting my "Pocket Full of Canvas" experiment to use HTML5 audio would be a fairly easy task. You can check out the result here.
At the moment you'll need a patched version of Firefox to get access to these extras but if you're not comfortable with building your own version, there are builds available here. It will still run in other (canvas-enabled) browsers but it will fall back to using SoundManager2 for the audio. While Flash audio is not that exciting anymore, it lets it run in eg. Chromium and Opera - and man, that's something else. Even if Firefox/Minefield has the upper hand with the audio stuff, it is feeling reaaally sluggish compared to those two (and you should definitely be using a nightly Minefield build). Chromium has always been speedy and Opera has gotten really crazy fast as well, running the visualizations in full-screen is just great (click the viz area to toggle).
And if you feel like playing around with the demos yourself, just click the "Show code" button and start hacking. I made a quick write-up of the very simple API here. The emphasis has been on making it easy to get from zero to something and not so much on making a lot of fancy functionality, so it is not very advanced.
Try it out: http://www.nihilogic.dk/labs/pocket_full_of_html5/
More examples:
Too bad there's no Linux x64 build available :(
April 15, 2010 at 1:57 PM Beben Kobenhmmm...what is this, i am not understand, just looking :)
April 20, 2010 at 2:29 PM general fuzzI'm seriously impressed that you were able to build in HTML 5. Good work! I clearly need to investigate its potential . . . .
April 27, 2010 at 8:25 AM Jacob SeidelinGreat, glad you like it :-) And thanks for the tunes, they are most delicious.
April 27, 2010 at 12:00 PM ScottSomehow I missed seeing this earlier, great stuff; glad to see you're looking at the Audio data API patch for Firefox. I think it has a lot of potential, hopefully it can get in and adopted etc. It's a huge piece missing from the current Audio API spec (even if only at v1.0 itself), in my opinion.
April 27, 2010 at 3:01 PM Jacob SeidelinRe: SM2, you may be able to get higher framerates if you flip on useHighPerformance and useFastPolling = true. (It doesn't look like either are on right now.) The useHighPerformance affects flash movie on-screen positioning (fixed vs. absolute, long story short, fixed = on-screen = better performance) - useFastPolling means a smaller interval within Flash for increased rate of whileplaying() calls, thus faster UI redraw.
Also FWIW, comment #3 re: "Epic" data is pure spam; identical posts all over the web elsewhere. ;)
@Scott: Yes, I know about those settings but they (or at least one of them..) were giving me trouble with Firefox not correctly loading SM2 unless I turned them off (or turned Flash debugging on). Thanks though!
April 28, 2010 at 12:24 AM SchillHm, crap; that's not supposed to happen. If you want to try it again (or run into the issue again in future), let me know; my goal is to have things "just work" in all configurations - the trick is that there happen to be lots of configurations. ;) If you can file a bug, even better!
April 29, 2010 at 5:31 PM ScottJS-flash gets a bit weird in some cases, in Firefox if wmode is anything other than the default the movie will not load if the SWF is off-screen etc. (Again, should be handled, but I digress.)
Hm, crap; that's not supposed to happen. If you want to try it again (or run into the issue again in future), let me know; my goal is to have things "just work" in all configurations - the trick is that there happen to be lots of configurations. ;) If you can file a bug, even better!
April 29, 2010 at 5:38 PM www.richonet.comJS-flash gets a bit weird in some cases, in Firefox if wmode is anything other than the default the movie will not load if the SWF is off-screen etc. (Again, should be handled, but I digress.)
Thanks for your content
May 12, 2010 at 11:02 PM CenaThanks,
www.oracledba.in
www.richonet.com
Hi, I am giving this party, and I have got a beamer. I thought it would be nice to show some nice visualizations on the big screen.(And show track names.) Is there such a program?
June 9, 2010 at 1:58 PM KampeAnd no window media player, since I have had some bad experiences with in the past.
Thanks
Amazing work man, keep it up!
June 13, 2010 at 2:04 AM XeallexxHey, Can you make this unreliant on the song thats being played so you could play your own music to it? I'd truly enjoy that. Another thing i would like to ask is if you could convert it to an embed-able format to use anywhere.
June 17, 2010 at 5:22 PM HavuzOther than that...
Wow! I really enjoy all projects posted on this blog and this one just blows my mind, thanks for all your effort!
thanks it is for me ı take it.
July 7, 2010 at 4:57 AM AnonymousThis is very interesting game...........I would like to visit once in a week....... I would like to play it again.......
July 7, 2010 at 7:05 AM Fotoğrafçıonline games
thanks you for this wonderfull game. ı am playing it twice a day.
July 8, 2010 at 2:25 AM Havuz Kimyasallarıthanks.
July 8, 2010 at 2:26 AM Perukthanks you. ı am triying to work with asp.net are they ant good think for me.
July 12, 2010 at 2:03 AM Social Security Number LookupI have tried this new HTML 5 and it is very powerful even the iphone 4 is using the same software. So keep exploring more on this software and let us all know about it.
July 28, 2010 at 10:24 PM Ted MeyerHi, I've been combing the web for about 6 hours now, looking for an HTML5 audio visualizer. this is the closest i got, but i cant use it on my own site. if theres a way to download, please let me know, if not, are there any others that are available for export to my website? thanks.
September 16, 2010 at 7:00 PM Fenerbahçe Beşiktaş Maçını İzlethanks very much Fenerbahçe Beşiktaş Maçını İzle
September 17, 2010 at 4:25 AM UnknownThe canvas element gives they everything necessary to draw all sorts of cool stuff in the browser, but the audio continues to flash a little funny. While the new HTML5 audio element allows audio playback, It does not give they much in terms of audio data - crucial if the visualisations are reacting to the music.
November 9, 2010 at 4:22 AM GrayFree Poker Money
Woah, I'm surprised this article doesn't have more relevant comments. This is great :/.
January 2, 2011 at 4:50 AM knutI'm trying to do something similar in AS3, but ran into the issue of getting the framebuffer of a playing mp3 hosted on another domain.
I want to visualize bass strokes. I'm not even close to know how to extract the bass level from the frameBuffer property on the MozAudioAvailable event. Can you please point me in the right direction? What are the numbers in the frameBuffer array? Which of them has to do with bass?
January 5, 2011 at 1:22 PM Kosmösvery cool stuff - by the way, I am using the latest build of Firefox 3 (3.6.13) and the visualizations run without issue. It appears to me you're looping images at a variable framerate tied to events in the music, correct? From the size of the initial file transfer, it would appear this is the case?
January 27, 2011 at 8:05 PM AnonymousWOW!!
March 4, 2011 at 3:57 AM mishaThanks for sharing an amazing stuff ..cool..
March 24, 2011 at 10:32 AM Gabriel Guerreirovinyl siding installation virginia
try:
December 24, 2011 at 7:24 AM Gokul(function() {
return function() {
quality(1);
decay(0.05);
zoom(.95)
move(0.025,0);
drawPath([[0,.2],[0.05,0.215],[0.05,0.785],[0,0.8]], false, "hsl(" + (((time * 10) % 360) >> 0) + ",100%,15%)");
for(var i = 0; i < 11; i++) drawRect(.005 * i, 0.5 - soundData.waveDataL[i * 12] * .5, .005, soundData.waveDataL[i * 12], "#FFF");
commit();
};
})();
Excellent work!! Could you please tell what do you mean by "pre-analysis of the MP3 file" ?? which software did you use to get the soundData ??
October 5, 2012 at 9:15 AM