Making a Javascript Video Player
Another afternoon wasted, another semi-useless yet also fun experiment. This time the topic is making Javascript do video. The goal was simple, make a video player using just Javascript, the DOM and a bit of XHR.
My first thought was to read binary video files using a technique like the Andy Na posted about here, figuring that there must be some really simple to parse video formats around, but I soon changed directions and decided to make up a whole new video format. Enter.. JSONVid. Using a player like mplayer, it is easy to export all frames in a movie clip to individual jpeg files, and using whichever language you prefer it is also fairly trivial to collect these files, base64 encode the bunch of them and throw them all together in a nice JSON file (I used this PHP script).
So we now have a video format that looks like this:
Pretty straight forward and easy to retrieve and parse using Javascript and XHR.
First strategy was to create an Image object for each frame and render it on a canvas element using drawImage(). That worked fine and performance was nice (although Opera used a lot of CPU), but I figured I'd try just using a regular image tag and just change the src property to another data:uri each frame. The change was barely noticeably in Firefox and Safari and it ran a bit better in Opera, so I lost the canvas and stuck with plain old images.
Now, it seems that Firefox will eat up all the memory in the world if you keep throwing new data:uris at the same image tag, which led to another change, so for each frame a new Image object was created and saved for later and as the video played, the previous frame Image was replaced by the new Image object. That seemed to work, but introduced an annoying delay as all these Image objects were created before playing, so I ended up moving the Image creation to actual render cycle where it simply checks if the frame Image has already been created, and if not, creates it.
"But", you say, "there is no sound!". Indeed. We could embed an mp3 (or other format) track using old school embedding, but there is a delay between inserting the embedded element (or calling the Play() method) until it actually starts playing. Since we don't know how long this delay is each time, you'd only get out of sync audio. Using something like Scott Schiller's very nice SoundManager sound API, I suppose one could get better synced sound, but then we're back to using Flash and why do this at all, then..
We remain silent for now.
Finally, as it is now, the whole video file has to be downloaded before playback can begin. However, it shouldn't be a problem to add some kind of streaming support in the player, although the streaming on the server would need either a bit of serverside code or chopping the file into smaller pieces. That's for another day.
So, Flash definitely, and obviously, does this better, but I suppose you could use this to have short soundless clips playing without plugins of any kind. "Oh, like a GIF file", you say. Yea, I suppose, but with playback control and you could take advantage of PNG's alpha. But yea, as I said at the top, I'm not sure how useful this is, and the lack of support in IE kills it a bit, but I hear IE8 has data:uri support, so maybe there's hope.
The result is being able to do something like this:
There are two test files, both around 4 Mb: Test 1 and Test 2
Another afternoon wasted, another semi-useless yet also fun experiment. This time the topic is making Javascript do video. The goal was simple, make a video player using just Javascript, the DOM and a bit of XHR.
My first thought was to read binary video files using a technique like the Andy Na posted about here, figuring that there must be some really simple to parse video formats around, but I soon changed directions and decided to make up a whole new video format. Enter.. JSONVid. Using a player like mplayer, it is easy to export all frames in a movie clip to individual jpeg files, and using whichever language you prefer it is also fairly trivial to collect these files, base64 encode the bunch of them and throw them all together in a nice JSON file (I used this PHP script).
So we now have a video format that looks like this:
{
frm : "JSVID", // format id tag
ver : 1, // version number of format
width : 320, // width of video
height : 240, // height of video
rate : 15, // framerate (frames per second)
frames : 495, // number of frames in file
data : {
video : [ // here comes 495 data:uris containing base64 encoded jpeg image frames
"data:image/jpeg;base64,/9j/4AAQSkZJRgABAgEASABIAAD/7gAOQWRv ... ",
"data:image/jpeg;base64,/9j/4AAQSkZJRgABAgEASABIAAD/7gAOQWRv ... ",
...
]
}
}
Pretty straight forward and easy to retrieve and parse using Javascript and XHR.
First strategy was to create an Image object for each frame and render it on a canvas element using drawImage(). That worked fine and performance was nice (although Opera used a lot of CPU), but I figured I'd try just using a regular image tag and just change the src property to another data:uri each frame. The change was barely noticeably in Firefox and Safari and it ran a bit better in Opera, so I lost the canvas and stuck with plain old images.
Now, it seems that Firefox will eat up all the memory in the world if you keep throwing new data:uris at the same image tag, which led to another change, so for each frame a new Image object was created and saved for later and as the video played, the previous frame Image was replaced by the new Image object. That seemed to work, but introduced an annoying delay as all these Image objects were created before playing, so I ended up moving the Image creation to actual render cycle where it simply checks if the frame Image has already been created, and if not, creates it.
"But", you say, "there is no sound!". Indeed. We could embed an mp3 (or other format) track using old school embedding, but there is a delay between inserting the embedded element (or calling the Play() method) until it actually starts playing. Since we don't know how long this delay is each time, you'd only get out of sync audio. Using something like Scott Schiller's very nice SoundManager sound API, I suppose one could get better synced sound, but then we're back to using Flash and why do this at all, then..
We remain silent for now.
Finally, as it is now, the whole video file has to be downloaded before playback can begin. However, it shouldn't be a problem to add some kind of streaming support in the player, although the streaming on the server would need either a bit of serverside code or chopping the file into smaller pieces. That's for another day.
So, Flash definitely, and obviously, does this better, but I suppose you could use this to have short soundless clips playing without plugins of any kind. "Oh, like a GIF file", you say. Yea, I suppose, but with playback control and you could take advantage of PNG's alpha. But yea, as I said at the top, I'm not sure how useful this is, and the lack of support in IE kills it a bit, but I hear IE8 has data:uri support, so maybe there's hope.
The result is being able to do something like this:
<html>
<head>
<script src="jsvideo.js" type="text/javascript"></script>
</head>
<body>
<div videosrc="myvideo.jsvid" videoautoplay="true"></div>
</body>
</html>
There are two test files, both around 4 Mb: Test 1 and Test 2
"So, Flash definitely, and obviously, does this better, but I suppose you could use this to have short soundless clips playing without plugins of any kind."
April 18, 2008 at 9:24 PM Jacob SeidelinSo why not just use an animated gif?
A valid question, and one I've asked myself. You don't get playback control with a gif, though. But yea..
April 18, 2008 at 9:49 PM Anonymous"another semi-useless yet also fun experiment" ah - that sums it up quite nice. It might been fun, chalenging, but mostly just useless ;-)
April 21, 2008 at 11:19 PM AnonymousSeems to be quiet useless if there is no avi, wmv etc to picture converter.. How did you put every single picture in there?!
April 22, 2008 at 7:47 AM AnonymousThomas
You could grab the data urls on the fly from the response with `if (xhr.readystate == 3) ...` and have streaming video.
April 23, 2008 at 6:06 AM Jacob SeidelinThe JS APIs of media player embeds are pretty horrible, but it might be possible to figure out the current position and sync it to the video (+ drop video frames if the video lags.)
With preloaded audio and video, it should be possible to sync them. Resyncing a stopped stream might go bad if audio plugin doesn't give accurate timing data. But heck do I know, try it out? :)
That's a nice suggestion, but it would have to be a different file format then, since a partial response wouldn't be valid javascript as it is now. About the embedded sound: Yea, I think I'm just going to leave it soundless as it is. I don't need the headache trying to get that to work.
April 23, 2008 at 8:12 AM AnonymousIf you want to shorten the load-time, you can replace the base64 uris by true urls. I have built a similar player that does around 10fps by loading pictures individually on a local area network. Trough a DSL connection, it's more like 2fps. Best is if your web server supports HTTP/1.1 keep-alive to avoid connections round-trips.
April 24, 2008 at 6:44 AM AnonymousAnother crazy idea would be to use the HTTP Range header[1] to get parts of a multipart/jpeg file. Ranges would be acquired at load-time and used to retrieve a particular image with an xhr request.
[1] : http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35
Or you could "stream" the images using Content-Type: multipart/x-mixed-replace
April 25, 2008 at 1:05 AM AnonymousFantastic, really! I like all the javascript stuff. Its free and that´s what Software development should be.
May 25, 2008 at 1:29 PM Anonymousion cell cleanseion cell cleanse
June 5, 2008 at 1:27 AM Anonymousion cleanseion cleanse
why do you have to load the whole movie before playing it? couldn't you set up a circular buffer of frames, and a javascript timer to refresh the buffer as frames get "used up" by being viewed? how about a traditional player control, with a "seek" bar? to those who say this is useless -- some very simple security cam systems output a stream of jpeg images at a fixed FPS, so this would be a reasonable way of presenting that. maybe flash would be better, i don't know anything about flash (should learn!)
September 17, 2008 at 11:44 AM Paulo GonçalvesGreat!
October 28, 2008 at 9:33 AM AnonymousThis can be even better, and replace flash.
I suggest you set this up as an open source project using SVN. Perhaps on google. I am sure others including myself would love working on this project. I work with EXT JS and it seems that the only thing I really need flash for is Video. I don't see why the issues of sound sync and streaming can't be worked out.
March 14, 2009 at 3:29 PM AishwarAwesome idea! simple (I know its not always easy to think out simple ideas :) ) and clever. Good job
May 26, 2009 at 7:49 PM AnonymousI've been looking for a way to stream multiple jpegs via javascript but I don't want to have to wait for the whole thing to load. Can this script be modified to do this?
June 13, 2009 at 5:54 PM AnonymousGreat job. One the most interesting project I've ever seen because for now HTML5 sucks and Flash will suck forever.
February 19, 2010 at 11:59 PM Unknownhttp://www.virginmedia.com/music/video/player/i-blame-coco/in-spirit-golden/621774267001/#vid-621774267001
October 2, 2010 at 5:36 AM vividI'm not sure this is completely useless.. If a bridge from a server could be created to dish up a 'real' video (sound permitting) you suddenly have a great way to show video on iDevices (iPhone and iPad) without using the default player, fullscreen etc. I think this has great potential given some compression and sound could be worked out.
February 11, 2012 at 10:15 AM Movie Trailerson the test page there is nothing loading after i hover the mouse. i want to create a custom movie player for onchannel.net site
July 20, 2014 at 10:25 AM