Wouter Verweirder

My Personal Blog

Websocket Version 8 Support in AIR Server

It’s been a while since my last update on AIRServer – an actionscript library which enables you to set up a socket server in Adobe AIR, which listens to multiple socket inputs (native AMF sockets, websockets and P2P input).

Some of my students noticed the websockets didn’t work in the latest versions of Google Chrome. This is because Google chrome uses another version of the websocket protocol. If you enjoy reading specs, you can find it at http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-10 :–)

Anyway, took me quite some work, but I finally got it working. I was able to port some of the code of the Bauglir Internet Library (http://www.webnt.eu/index.php) to actionscript. AIRServer now supports both hybi-00 and hybi-10 of the protocol.

You can download the updated library and demo’s here. Enjoy!

You can find the latest version on github

Flex Mobile Tabbed View Navigator Button Badge

I wanted to display a “badge text” in a Flex Mobile project. Badge texts are the red circles on iOS, where you can see a number of updates, unread messages, …

Badge texts on tabbed view navigator

This isn’t included in the default flex mobile components / skins, so I decided to extend & skin the tabbed view navigator, so you’re able to add this functionality in your applications.

You simple use my BadgeViewNavigator instead of ViewNavigators, and set a badgeText value to display in the badge:

1
2
3
4
5
6
7
8
<views:BadgeViewNavigator label="One" width="100%" height="100%"
      firstView="be.aboutme.flex.mobile.badgeViewNavigator.views.OneView"
      badgeText="23"/>
<views:BadgeViewNavigator label="Two" width="100%" height="100%"
      firstView="be.aboutme.flex.mobile.badgeViewNavigator.views.TwoView"
      badgeText="2"/>
<views:BadgeViewNavigator label="Three" width="100%" height="100%"
      firstView="be.aboutme.flex.mobile.badgeViewNavigator.views.ThreeView"/>

You can download the sources & demo in this fxp file. Enjoy!

Kinect Native Extension for Adobe AIR

Update: This sample only supports the beta1 SDK on windows. Please check out http://www.as3nui.com, where further development is happening (newer SDK versions, support for OSX).

I’ve been playing with native extensions for air for a couple of weeks now. One of the things I wanted to do, was to get the Kinect working through a native extension. I’ve posted some sample libraries before, where the kinect date was sent to Adobe AIR through UDP sockets. However, the bandwith is quite limited, and there are noticable delays.

Using a native extension, we don’t have those limitations anymore. Another pro is that we don’t need to run a seperate program to send the data to our flash application.

I’ve got multiple skeleton tracking working, together with the video and the depth video.

1
2
3
4
5
6
7
8
9
10
11
12
kinect = new Kinect();
kinect.addEventListener(KinectEvent.SKELETON_TRACKED, skeletonTrackedHandler, false, , true);
kinect.startTracking();

kinectVideo = new KinectVideo(kinect);
kinectVideo.play();
addChild(kinectVideo);

kinectDepth = new KinectDepth(kinect);
kinectDepth.x = 650;
kinectDepth.play();
addChild(kinectDepth);

displaying the skeleton is pretty straightforward:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
private function drawSkeletons():void
{
    skeletonsContainer.graphics.clear();
    for each(var skeleton:Skeleton in kinect.skeletons)
    {
        for each(var joint:Joint in skeleton.joints)
        {
            drawJoint(joint);
        }
        //draw some bones
        drawBone(skeleton.joints[Joint.SHOULDER_CENTER], skeleton.joints[Joint.SHOULDER_LEFT]);
        drawBone(skeleton.joints[Joint.SHOULDER_LEFT], skeleton.joints[Joint.ELBOW_LEFT]);
        drawBone(skeleton.joints[Joint.ELBOW_LEFT], skeleton.joints[Joint.WRIST_LEFT]);
        drawBone(skeleton.joints[Joint.WRIST_LEFT], skeleton.joints[Joint.HAND_LEFT]);
        drawBone(skeleton.joints[Joint.SHOULDER_CENTER], skeleton.joints[Joint.SHOULDER_RIGHT]);
        drawBone(skeleton.joints[Joint.SHOULDER_RIGHT], skeleton.joints[Joint.ELBOW_RIGHT]);
        drawBone(skeleton.joints[Joint.ELBOW_RIGHT], skeleton.joints[Joint.WRIST_RIGHT]);
        drawBone(skeleton.joints[Joint.WRIST_RIGHT], skeleton.joints[Joint.HAND_RIGHT]);
        drawBone(skeleton.joints[Joint.SHOULDER_LEFT], skeleton.joints[Joint.SPINE]);
        drawBone(skeleton.joints[Joint.SHOULDER_RIGHT], skeleton.joints[Joint.SPINE]);
        drawBone(skeleton.joints[Joint.SPINE], skeleton.joints[Joint.HIP_CENTER]);
        drawBone(skeleton.joints[Joint.HIP_CENTER], skeleton.joints[Joint.HIP_LEFT]);
        drawBone(skeleton.joints[Joint.HIP_LEFT], skeleton.joints[Joint.KNEE_LEFT]);
        drawBone(skeleton.joints[Joint.KNEE_LEFT], skeleton.joints[Joint.ANKLE_LEFT]);
        drawBone(skeleton.joints[Joint.ANKLE_LEFT], skeleton.joints[Joint.FOOT_LEFT]);
        drawBone(skeleton.joints[Joint.HIP_CENTER], skeleton.joints[Joint.HIP_RIGHT]);
        drawBone(skeleton.joints[Joint.HIP_RIGHT], skeleton.joints[Joint.KNEE_RIGHT]);
        drawBone(skeleton.joints[Joint.KNEE_RIGHT], skeleton.joints[Joint.ANKLE_RIGHT]);
        drawBone(skeleton.joints[Joint.ANKLE_RIGHT], skeleton.joints[Joint.FOOT_RIGHT]);
    }
}

private function drawJoint(joint:Joint):void
{
    skeletonsContainer.graphics.beginFill(0x000000);
    skeletonsContainer.graphics.drawCircle(joint.x * 640, joint.y * 480, 10 / joint.z);
    skeletonsContainer.graphics.endFill();
}

private function drawBone(from:Joint, to:Joint):void
{
    skeletonsContainer.graphics.lineStyle(1);
    skeletonsContainer.graphics.moveTo(from.x * 640, from.y * 480);
    skeletonsContainer.graphics.lineTo(to.x * 640, to.y * 480);
}

You can download the flash builder project and the visual studio project for the native extension. I didn’t have any C programming experience before, so there’s probably room for improvement on the C side. Currently, the extension is only available for windows using the kinect sdk. Make sure you’ve installed the microsoft kinect sdk, aswell as visual studio. You’ll also need to launch the flash builder project using my ant build script which is included in the flash builder project. You’ll want to update the path to your air sdk in the ant-debug.xml.

If you want to see it live in action, come check out our session “interREACT with the flas platform” at the FITC unconference at MAX (tuesday).

Linking Microsoft Kinect SDK to Adobe AIR & TUIO Multitouch

Finally had a chance to play around with the official Kinect SDK. First thing I wanted to try, was to link the Kinect SDK to Adobe AIR (with UDP), and adjust my previous kinect demos (with OSCeleton and the open source drivers) so I could do multitouch with the kinect (a bit like minority report). The official SDK is a lot more stable then the open source drivers, and another plus is you don’t need to stand in a calibration position for the skeleton to get tracked. Downside of the official SDK is that it’s windows only…

Kinect UDP Screen

So, I booted up my Windows pc, and threw together a UDP bridge in C#, which sends the skeleton information as a JSON encoded string to a target IP + port. I’ve coded a little adapter in Actionscript 3, which translates the JSON object into TUIO cursors. After that, it was a piece of cake to get multitouch working in Adobe AIR, using the skeleton information from the kinect.

Using your hands as multitouch cursors

I’m happy to share the C# project for the Kinect UDP bridge, and an AIR sample project, which allows you to transform a square, using the touch events from the kinect. Have fun!

Update: I’ve also uploaded a binary, run setup.exe in the zip file to install. Make sure the kinect is connected when you start up the application.

AIRServer Example: Multi-user Smartphone Controlled Game

I wrote this game on top of my AIRServer library, as part of some multi-user explorations with the flash platform. The graphics were created by my HOWEST colleague Jynse Cremers :–)

Users can join a 30-second spacegame. Each user controls a spaceship with their smartphone. The game itself was built with Adobe AIR and the AIRServer actionscript library. To join the game users can choose between a native application built with Adobe AIR, or an html5/js websockets client in their browser.

I’ll release the source files after some cleaning up, so you can play around with it aswell!

Update: find the source files in the follow-up post!

Smartphone controlled game on Vimeo.

Handling Websocket Connections With Adobe AIR ServerSocket

Since Adobe AIR 2.0, you can create your own socket-servers, using the flash.net.ServerSocket class. This makes it easy to hook external applications to your AIR applications, for example to create a game (server) with multiple devices as game controllers (socket clients).