Some of my students noticed the websockets didn’t work in the latest versions of Google Chrome. This is because Google chrome uses another version of the websocket protocol. If you enjoy reading specs, you can find it at http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-10 :-)

Anyway, took me quite some work, but I finally got it working. I was able to port some of the code of the Bauglir Internet Library (http://www.webnt.eu/index.php) to actionscript. AIRServer now supports both hybi-00 and hybi-10 of the protocol.

You can download the updated library and demo’s here. Enjoy!

Update on http://blog.aboutme.be/2011/12/23/airserver-0-5-socket-byte-concatenation/

So, I booted up my Windows pc, and threw together a UDP bridge in C#, which sends the skeleton information as a JSON encoded string to a target IP + port. I’ve coded a little adapter in Actionscript 3, which translates the JSON object into TUIO cursors. After that, it was a piece of cake to get multitouch working in Adobe AIR, using the skeleton information from the kinect.

I’m happy to share the C# project for the Kinect UDP bridge, and an AIR sample project, which allows you to transform a square, using the touch events from the kinect. Have fun!

Update: I’ve also uploaded a binary, run setup.exe in the zip file to install. Make sure the kinect is connected when you start up the application.

The zip file is actually a Flash Builder 4.5 workspace, containing the following projects:

- AIRServer: the library project containing the AIRServer classes
- CocoonP2P: the cocoon P2P library
- DevineSpaceGameAIRServer: the AIR server, containing the space game
- DevineSpaceGameControllerAMF: A mobile controller, using an AMF socket connection
- DevineSpaceGameControllerP2P: A mobile controller, using CocoonP2P

When the game is running, you can control it through websockets aswel: surf to http://labs.aboutme.be/airserver/spaceshooter/ with an iPhone4 / iPod Touch 2 or iPad to control the game.

Enjoy!

Users can join a 30-second spacegame. Each user controls a spaceship with their smartphone. The game itself was built with Adobe AIR and the AIRServer actionscript library. To join the game users can choose between a native application built with Adobe AIR, or an html5/js websockets client in their browser.

I’ll release the source files after some cleaning up, so you can play around with it aswell!

Update: find the source files in the follow-up post!

Smartphone controlled game on Vimeo.

It’s another option to create a multiuser game with different inputs. Right now you can use flash sockets, websockets and rtmfp (P2P) connectivity in a mixed environment to communicate with eachother.

server = new AIRServer();
//adding endpoints
server.addEndPoint(new SocketEndPoint(1234, new AMFSocketClientHanderFactory(new NativeObjectSerializer())));
server.addEndPoint(new SocketEndPoint(1235, new WebSocketClientHandlerFactory(new JSONSerializer())));
server.addEndPoint(new CocoonP2PEndPoint("be.aboutme.airserver.demos.Messages"));
//add event listeners
server.addEventListener(AIRServerEvent.CLIENT_ADDED, clientAddedHandler, false, 0, true);
server.addEventListener(AIRServerEvent.CLIENT_REMOVED, clientRemovedHandler, false, 0, true);
server.addEventListener(MessageReceivedEvent.MESSAGE_RECEIVED, messageReceivedHandler, false, 0, true);
//start the server
server.start();

Adding a cocoon endpoint, is even simpler than adding socket endpoints, all you have to do is pass an instance of the CocoonP2PEndpoint class to the addEndpoint function of the server, and specify a groupname for the P2P netgroup. You can handle the messages just like any other message on the server, it doesn’t matter if the message comes from a socket, websocket or via rtmfp.

You can download an updated version of the library project and an updated version of the demo’s with a cocoon P2P client. Big thanks to Peter Elst for CocoonP2P, it makes P2P on the local network a lot easier to implement!

Update on http://blog.aboutme.be/2011/12/23/airserver-0-5-socket-byte-concatenation/

First of all, download the zip file containing the library and some demo projects. Import the library project into Flash Builder. Next step could be checking out the demo’s: you’ll need the server demo (MessagesDemoServer), which is an Adobe AIR app.

This MessagesDemoServer example, listens for amf sockets on port 1234 and websockets on port 1235:

server = new AIRServer();
//add the listening endpoints
server.addEndPoint(new SocketEndPoint(1234, new AMFSocketClientHanderFactory(new NativeObjectSerializer())));
server.addEndPoint(new SocketEndPoint(1235, new WebSocketClientHandlerFactory(new JSONSerializer())));
//add event listeners
server.addEventListener(AIRServerEvent.CLIENT_ADDED, clientAddedHandler, false, 0, true);
server.addEventListener(AIRServerEvent.CLIENT_REMOVED, clientRemovedHandler, false, 0, true);
server.addEventListener(MessageReceivedEvent.MESSAGE_RECEIVED, messageReceivedHandler, false, 0, true);
//start the server
server.start();

You’ll also notice the need to specify a “ClientHandler Factory”. This way, you can specify custom object serializers if needed and custom crossdomain xmls (optional). Right now, there’s support for JSON encoding (for text-socket communication like websockets) and native encoding (for communication with flash client sockets).

There are two clients in the examples aswell: an AMF Client (which is basically a flex app to connect & send messages to the server), and a HTML5/Javascript Client (same functionality as the flex application).

So, download & enjoy this library. You can report bugs, or send in suggestions in the comments… Stay tuned for some more updates the next couple of days…

Update on http://blog.aboutme.be/2011/12/23/airserver-0-5-socket-byte-concatenation/

With an adobe air serversocket, most of the time, the clients will be other flash-based applications (swfs or native apps built with Adobe AIR). HTML5 introduced the concept of websockets: a way to create your own socket connections from an html page (using javascript) to a server. You’ll need to implement the right protocol to handle these connections, but with some work, you’re able to create an html/js mobile controller for an Adobe AIR application. Big advantage here is you don’t need to install a native application, you can just surf to a url, and start sending data between your client and your server.

Server side

First of all, I’ll go through the code on the server. I assume you’ve created an Adobe AIR 2.0 (or higher) project in your AS editor of choice. First of all, you’ll need to create an instance of the ServerSocket class and hook up some event listeners:

serverSocket = new ServerSocket();
serverSocket.addEventListener(ServerSocketConnectEvent.CONNECT, clientConnectHandler, false, 0, true);
serverSocket.bind(1235);
serverSocket.listen();

Whenever a client connects, I create an instance of an own-written client handler class, which will manage the connected client. I manage all clienthandlers in a global clientHandlers array.

private function clientConnectHandler(event:ServerSocketConnectEvent):void
{
   var clientHandler:ClientHandler = new ClientHandler(event.socket);
   clientHandler.addEventListener(Event.CLOSE, clientHandlerCloseHandler, false, 0, true);
   clientHandlers.push(clientHandler);
}

The clienthandler instances, manage all communication between the server and the client. This is where some of the websocket-specific stuff comes in: the first message the server receives, is a handshake. We’ll need to send back a specific message to the client, otherwise the client will drop the connection with the server. Read more about websockets and this handshake mechanism on wikipedia.

We listen for data in this clientHandler class:

public function ClientHandler(socket:Socket)
{
   this.socket = socket;
   socket.addEventListener(ProgressEvent.SOCKET_DATA, socketDataHandler, false, 0, true);
}

And inside this socketDataHandler, we’ll find the handshake logic:

private function socketDataHandler(event:ProgressEvent):void
{
   if(socket.bytesAvailable > 0)
   {
      if(!firstRequestProcessed)
      {
         firstRequestProcessed = true;
         var message:String = socketBytes.readUTFBytes(socketBytes.bytesAvailable);
         if(message.indexOf("GET ") == 0)
         {
            var messageLines:Array = message.split("\n");
            var fields:Object = {};
            var requestedURL:String = "";
            for(var i:uint = 0; i < messageLines.length; i++)
            {
               var line:String = messageLines[i];
               if(i == 0)
               {
                  var getSplit:Array = line.split(" ");
                  if(getSplit.length > 1)
                  {
                     requestedURL = getSplit[1];
                  }
               }
               else
               {
                  var index:int = line.indexOf(":");
                  if(index > -1)
                  {
                     var key:String = line.substr(0, index);
                     fields[key] = line.substr(index + 1).replace( /^([\s|\t|\n]+)?(.*)([\s|\t|\n]+)?$/gm, "$2" );
                  }
               }
            }
            //check the websocket version
            if(fields["Sec-WebSocket-Version"] != null)
            {
               //NOT SUPPORTED YET
            }
            else
            {
               if(fields["Sec-WebSocket-Key1"] != null && fields["Sec-WebSocket-Key2"] != null)
               {
                  //draft-ietf-hybi-thewebsocketprotocol-00
                  //send a response
                  var result:* = fields["Sec-WebSocket-Key1"].match(/[0-9]/gi);
                  var key1Nr:uint = (result is Array) ? uint(result.join("")) : 1;
                  result = fields["Sec-WebSocket-Key1"].match(/ /gi);
                  var key1SpaceCount:uint = (result is Array) ? result.length : 1;
                  var key1Part:Number = key1Nr / key1SpaceCount;
 
                  result = fields["Sec-WebSocket-Key2"].match(/[0-9]/gi);
                  var key2Nr:uint = (result is Array) ? uint(result.join("")) : 1;
                  result = fields["Sec-WebSocket-Key2"].match(/ /gi);
                  var key2SpaceCount:uint = (result is Array) ? result.length : 1;
                  var key2Part:Number = key2Nr / key2SpaceCount;
 
                  //calculate binary md5 hash
                  var bytesToHash:ByteArray = new ByteArray();
                  bytesToHash.writeUnsignedInt(key1Part);
                  bytesToHash.writeUnsignedInt(key2Part);
                  bytesToHash.writeBytes(socketBytes, socketBytes.length - 8);
 
                  //hash it
                  var hash:String = MD5.hashBytes(bytesToHash);
 
                  var response:String = "HTTP/1.1 101 WebSocket Protocol Handshake\r\n" +
                     "Upgrade: WebSocket\r\n" +
                     "Connection: Upgrade\r\n" +
                     "Sec-WebSocket-Origin: " + fields["Origin"] + "\r\n" +
                     "Sec-WebSocket-Location: ws://" + fields["Host"] + requestedURL + "\r\n" +
                     "\r\n";
                  var responseBytes:ByteArray = new ByteArray();
                  responseBytes.writeUTFBytes(response);
 
                  for(i = 0; i < hash.length; i += 2)
                  {
                     responseBytes.writeByte(parseInt(hash.substr(i, 2), 16));
                  }
 
                  responseBytes.writeByte(0);
                  responseBytes.position = 0;
                  socket.writeBytes(responseBytes);
                  socket.flush();
                  //stop right here
                  socketBytes.clear();
                  return;
               }
            }
         }
      }
      else
      {
         //do something else with the data
      }
   }
}

This will accept a websocket connection and send back the correct handshake. So, how do you connect from html / javascript to this AIR server?

HTML5 Websocket client

You’ll connect to the AIR server using javascript. This is pretty straightforward: you create a WebSocket object, link the correct event handlers and connect to the air server on the correct port:

function init()
{
   try
   {
      socket = new WebSocket("ws://localhost:1235");
      socket.onopen = socketOpenHandler;
      socket.onmessage = socketMessageHandler;
      socket.onclose = socketCloseHandler;
   }
   catch(exception)
   {
      alert("Error: " + exception);
   }
}
 
function socketOpenHandler()
{
   //connected to the server
}
 
function socketMessageHandler(message)
{
   //message received: message.data
}
 
function socketCloseHandler()
{
   //connection closed
}

That’s it. Fire up your Adobe AIR server, and you should be able to connect to the Adobe AIR application from javascript!

I’ll post some examples & a utility library in a follow-up post, so stay tuned :-)

Update: Check http://blog.aboutme.be/2011/11/24/airserver-support-for-websocket-version-8/ for a library and demos.

There are a few methods around:

- AS3Kinect: http://www.as3kinect.org/ – using libfreenect, and sockets to get the 2 webcam images (depth and color) into flash.
- TUIOKinect (https://code.google.com/p/tuiokinect) + TUIO AS3 (http://bubblebird.at/tuioflash/tuio-as3-library) – track “blobs” and get them as multitouch cursors in your flash app. More info at http://silviopaganini.posterous.com/openkinect-flash-tuio-udp-flash-bridge
- OSCeleton (https://github.com/Sensebloom/OSCeleton) sends out skeleton data in the OSC format to a UDP destination

Getting the images in flash is really slow, it needs to compress (at best) each frame and send that information over a socket / UDP / Localconnection to flash. It’s quite heave, and all you’ve got to work with are images.

The blob tracking from TUIOKinect is more interesting: we only send blob tracking info to our flash app, which is a lot “lighter” to transfer. However, TUIOKinect needs to be configured (what z-depth do you want to detect) and generates quite a lot false positives (stuff you don’t want to track). You don’t know if a blob is a hand or not. The fun part of TUIOKinect is that it’s TUIO, which means that you can track a blob, and use it as a touch input.

Then we have OSCeleton. With OSCeleton, you can get real skeleton data into flash. After standing in a calibration pose, it spits out the coordinates of you skeleton joints (head, left arm, right knee, …) to a UDP connection.

I decided to combine AS3TUIO with OSCeleton, so we can translate skeleton joint information and use skeleton joins (say: left hand, right hand) as touch points. This way, you can use the standard multitouch events, only you aren’t really touching anything, but waving in front of a kinect camera.

Setting it up

First things first: getting OSCeleton running. There’s an excellent guide over at https://github.com/Sensebloom/OSCeleton – so follow those steps first.

After you’ve got OSCeleton up and running, you can get the data into your AIR application using my library.

Download the source or swc file, and add it to your project.

Download some example projects (AIR, Flash Builder).

Download the library + demo’s at http://blog.aboutme.be/wp-content/uploads/ckeditor_flex_air/ckeditor_flex_air.zip

Again, some release notes (same as FCKEditor actually :-)):

When you extract the zip file, you will find an swc file, an assets folder and de demos folder. If you want to use CKEditor in your own Flex or AIR projects, add the swc to your library path, and the assets folder to your src folder (just like the demo projects).

When you use the component in a Flex project, you need to ensure you set the wmode property to ‘opaque’. Also, if you run it on your local filesystem with Internet Explorer, you won’t see the component. It works fine if you run it on a webserver.

You can check out a demo at http://labs.aboutme.be/flex_fckeditor/

You can download my library + demo’s at http://blog.aboutme.be/wp-content/uploads/flex_fckeditor/flex_fck_editor_0.1.zip

When you extract the zip file, you will find an swc file, an assets folder and de demos folder. If you want to use FCKEditor in your own Flex or AIR projects, add the swc to your library path, and the assets folder to your src folder (just like the demo projects).

When you use the component in a Flex project, you need to ensure you set the wmode property to ‘opaque’. Also, if you run it on your local filesystem with Internet Explorer, you won’t see the component. It works fine if you run it on a webserver.

Currently it’s possible to get / set the htmlText, and change the toolbar set. I hope you enjoy this component, if you find any bugs or have feature request, feel free to post them in the comments.