Support Hybi10 binaryType

[ebidel]

Just landed in Chromium: http://code.google.com/p/chromium/issues/detail?id=93652

Allows sending Blob or ArrayBuffer (binary data).

[einaros]

The reason why I didn't implement this part of the hybi draft is simply that the other transports would struggle with it. It'll be better to add this when the draft stabilizes, and more browsers support it.

[ebidel]

By other transports do you mean other protocol versions? Regardless of whether or not the current spec is stable, expect binary support to stay.

[ebidel]

Is this fixed with v0.85? I see binary support in the release notes.

[einaros]

Well it's in the parser at least, but it has yet to be connected to anything meaningful.

Now is a good time to ask: What binary data do you actually want to submit?

[ebidel]

Use cases are many. Here are 2 off the top of my head:

• stream audio to connected clients
• realtime screen collaboration tool: stream the binary images across the wire, render to canvas.

https://github.com/Worlize/WebSocket-Node seems to support binary transfer.

[einaros]

Supporting binary transfer is, as I said, no big deal. I just struggle to see real use cases, seeing as js on the client has no binary data type.

Stream audio: How would you decode the binary efficiently, and assign them to the float arrays used by the sound apis (which are still not standard)?

Stream binary images: Canvas painting would either (a) be a video element which is drawn to a canvas, (b) an image which is drawn to a canvas, (c) rgba number values set to the bitmapdata of the canvas. Of the three, neither could really use binary data sent via websockets. And (c), being the closest to even remotely possible, is insanely slow.

Edit: I see that Chrome at least support receiving arraybuffers now, which was ironically your opening statement for this issue, so I should have caught that. Anyhoo, the audio bit should be doable as you say, and probably canvas as well if it now has a setter for the typed arrays.

[ebidel]

js on the client has no binary data type.

Yep. They're many new types in JS to work with binary data: Typed Arrays, ArrayBuffer, Blob, File.
They're used all over the place now (webgl, audio apis, file apis, xhr2). I presented an entire section on this at Google IO:
http://www.htmlfivewow.com/slide11

Decoding audio:

The Web Audio API for example (now supported in Chrome and WebKit nightlies) has methods for decoding audio:
https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioContext-section

You send it an ArrayBuffer.

streaming binary images

I was thinking more like a remote desktop client. Images are rapidly sent over a websocket (without the need to base64 encode, etc.) and rendered onto a canvas. c.) could also work, since the updated protocol allows passing ArrayBuffers.

I believe they're swamping out canvas's CanvasPixelArray to use ArrayBuffer instead. Regardless, you'll
be able to .getBlob() from the canvas and create an image file from it:
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-canvas-toblob

[einaros]

I was familiar with the floating point types, but not blob / file and alike. Granted those, and also canvas' push for ArrayBuffer, it makes sense to complete this implementation sooner rather than later.

I'll see if I can find time to do so later this week.

[ebidel]

You da man! Thank you!

[mmarcon]

Was this ever implemented? I am curious to play with images on canvas to be sent to the server for heavy processing and ArrayBuffer over socket.io seems like a pretty solution.

[einaros]

I implemented this partially in socket.io, but fully in 'ws', my up-to-date websocket client/server for node. See https://github.com/einaros/ws

[ebidel]

What is missing on the socket.io side of things?

[einaros]

• Support across the other transports.
• Merge of my pullreq which uses ws over my old parsers in socket.io, OR:
• Switch to engine.io+websocket.io and exposure of ws' binary features through websocket.io.

I added more binary samples to ws, by the way.

[ebidel]

einarios, this would be killer for socket.io to support. It's by far the most adopted
websocket implementation for node. Unfortunately, I'm seeing a lot of people starting
to abuse dataURLs (33% encoding overhead) for sending files. If socket.io supported
binary out of the box, we could speed up the web! :)

Please let me know how I can help.

[einaros]

Well in terms of websockets, socket.io will just be a front for ws, which already supports binary. The challenge at this point is the other transports. Someone will have to decide to deal with those.

[ebidel]

Understood. We're in a good place with XHR2 supporting binary. I'm sure flash has some special sauce to plug into, but I can't speak for the flashsocket transport.

[einaros]

Flash itself should be able to do it, but it's running against websocket hixie, so it'd either require a hybi rewrite (quite a bit of work), or have binary data base64'd (ouch). In terms of the js / flash bridge, I don't know how the different flash versions would be able to deal with typed arrays and blobs. XHR2 may be the only reasonable option for non hybi browsers.

In either case, this will be after the engine.io switch. And someone will still have to step up to the task for xhr ;)

[ebidel]

Feel free to ping me when that happens. Glad to hack something up.

[einaros]

It's up to @guille and @3rd-Eden. I keep track of websockets.

[briprowe]

I just wanted to add a vote for this feature. It would be very useful for the things I'm working on.

[adambratt]

Is there any chance of this being added? Would really be useful.

[ekryski]

+1 Would be great. Willing to hack on it for sure.

[itsatony]

+1

[rmzi]

https://github.com/liamks/Delivery.js -- Experimental Solution

[vote539]

So, Firefox and Chromium and even IE 10 now support the WebSocket.binaryType property.

If you turn on the experimental "serialization" option in my implementation of file uploads over Socket.IO, it is worth noting that in exchange for a 10-fold transmission overhead (my 1.9 MB binary file serialized to 25.2 MB of data), Socket.IO already provides a near-seamless transition between W3C's ArrayBuffer and Node's Buffer by way of serialization.

This seems like a good time to support WebSocket binary transmissions in Socket.IO; in cases that it's not supported, it is simple enough to internally resort to a base64 approach.

[Zolmeister]

+1

[rauchg]

@vote539 I agree with this completely. I'll focus on binary after I get 1.0 out

[Rob-pw]

@guille Could we have a sitrep on this feature?

[mekwall]

I am also wondering what's going on with this. I am looking to use it to stream audio and video.

[vongosling]

+1