MS Surface 2.0 Hacking
Update: As of kernel release 4.2, my driver is now in the mainline tree and works out-of-the-box with, e.g., Ubuntu 15.10!
Update: Here's the tale of how I nearly bricked the Surface and got it back to life.
Most people will have heard of Microsoft's Surface 2.0, which is a neatly packaged 40" multitouch screen with integrated PC. Of course, out of the box, it runs a modified version of Windows 7 and little else. Since I always consider that a bit of a challenge, I've decided to see if you could also run Linux on it.
As it turns out, running Linux on the Surface 2.0 is actually quite straightforward; thanks to the Media Informatics Group at LMU, I had the opportunity to do a bit of experimentation on their Surface. Just attach an USB harddisk with Ubuntu Linux, change the boot order in the BIOS and get a Ubuntu desktop in no time. All the usual suspects that had flaky Linux support in ye olden tymes (3D acceleration, sound, WLAN) work out of the box, the only thing that doesn't is the PixelSense touchscreen - which is the most interesting part after all.
Of course, I've decided to fix that, and the result is available at github: Open-Source Surface 2.0 Driver.
The process of writing that driver was rather straightforward: install the Surface SDK on a VirtualBox VM running Windows 7, attach the "Microsoft Vision Input Processor" USB device to the VM and dump the USB data from /sys/kernel/debug/usb/usbmon/0u on the Linux host system.
The video protocol is really simple: read a total of 960x540 bytes from USB bulk endpoint 0x82 in blocks of 16k and you have a video frame. Rinse, repeat.
The touch data protocol isn't much more complicated: read 512-byte packets from endpoint 0x86 and you'll get a list of 64-byte structures containing one touch/blob/object descriptor each.
To round things off, a brief video showing the raw data is below.
Note: if you look closely, you can see some of the screen content (i.e. the video window) "bleed" through into the sensor data. Doesn't seem to affect the sensor, though.
Update: Here's the tale of how I nearly bricked the Surface and got it back to life.
Most people will have heard of Microsoft's Surface 2.0, which is a neatly packaged 40" multitouch screen with integrated PC. Of course, out of the box, it runs a modified version of Windows 7 and little else. Since I always consider that a bit of a challenge, I've decided to see if you could also run Linux on it.
As it turns out, running Linux on the Surface 2.0 is actually quite straightforward; thanks to the Media Informatics Group at LMU, I had the opportunity to do a bit of experimentation on their Surface. Just attach an USB harddisk with Ubuntu Linux, change the boot order in the BIOS and get a Ubuntu desktop in no time. All the usual suspects that had flaky Linux support in ye olden tymes (3D acceleration, sound, WLAN) work out of the box, the only thing that doesn't is the PixelSense touchscreen - which is the most interesting part after all.
Of course, I've decided to fix that, and the result is available at github: Open-Source Surface 2.0 Driver.
The process of writing that driver was rather straightforward: install the Surface SDK on a VirtualBox VM running Windows 7, attach the "Microsoft Vision Input Processor" USB device to the VM and dump the USB data from /sys/kernel/debug/usb/usbmon/0u on the Linux host system.
The video protocol is really simple: read a total of 960x540 bytes from USB bulk endpoint 0x82 in blocks of 16k and you have a video frame. Rinse, repeat.
The touch data protocol isn't much more complicated: read 512-byte packets from endpoint 0x86 and you'll get a list of 64-byte structures containing one touch/blob/object descriptor each.
To round things off, a brief video showing the raw data is below.
Note: if you look closely, you can see some of the screen content (i.e. the video window) "bleed" through into the sensor data. Doesn't seem to affect the sensor, though.