The Project That Didn’t Want To Start

Before I start I also considered the following titles:

  • Developing on Windows for the first (and hopefully last) time
  • Putting the aarrrrrgggg in RnD (Shoutout to Ian Stephenson for that one)

As part of my final year at the NCCA I am doing a piece undergrad research. My research will be into “Integrating Live Skeleton data into a VR Environment”.

While this post will detail the lengthy process of getting the project off the ground, I will briefly describe the project for reference.

The idea is to have a VR environment that is a simulation of a room. Inside the real life room is a person walking around, using an Xbox Kinect we can track that person and map their skeleton onto a virtual skeleton. The VR client does not have to be linked to the Kinect, the data will be sent over the network.

The networking is handled using the MQTT protocol that uses a simple system of publishing to a broker and then subscribing to that broker to get the information. (The diagram explains it way better than I could, just look at that)

 

So step one of the project, set up an MQTT broker and publish to it and subscribe to get the messages back. This is implemented by a bit of software called mosquitto, a command line tool to handle setting up the broker and clients. This was pretty trivial to set up on my Mac (Screenshot below), or any Unix system for that matter. To set it up on Windows was a little more tricky for me since I’m not used to using cmd, but also pretty simple to get it running.

The day after I took this screenshot I got it working across my local network easily between a few machines. So far so good, feeling good about the project, working with this kinda thing is pretty fun so it was a nice way to start the project. But it gets worse don’t worry.

To add a bit more context, the project follows on from another that took place over the summer by another student at BU. Adam’s project set up MQTT in Unity using a C# library, which was nice since it was a piece of groundwork I didn’t have to do. This took a little bit of refactoring but I got it working on my Mac, but after discussing with my tutor we thought it would be best to use the same computer Adam used since it was on Windows, meaning I could do Kinect development on there too.

Seems like a good idea, saves the ‘works on my machine’ problem since the focus is on the research. Little did I know, this particular box of silicon and circuit boards had it in for me.

At this point, for those unfamiliar with the joys of Microsoft peripheral development I’ll do a little bit of Background. Kinect V1 came out for the Xbox 360. This was when the dominant Windows version was 7. The Kinect V1 also works on Windows * (The OS the machine was running when I received it). However it is not supported on Windows 10, only the Kinect V2 is supported on Windows 10 (And only 10, not 8 or 7). So when I received the Kinect V1 for use on the Windows 8 machine, no problems right?

Wrong.

Visual Studio Community 2017.

Microsoft are happy with the “Redefined fundamentals” of VS2017. However I found myself unhappy with how they redefined the “Sign In” button to act as a close button. This is particularly troublesome as signing in is required to use all of the great features, such as editing and running code. (A handy feature in for an IDE to have)

Microsoft’s Kinect SDK is tied in to Visual Studio, and rewriting the Kinect SDK for Unix didn’t seem like a measured response to this particular problem. (The thought crossed my mind a few times though) So what to do? Installing an older version of VS seems like a good plan, but it seems like the Sign in bug (or feature, I won’t judge) goes back to older versions of VS too.

So I decided to troubleshoot this problem with VS2017. I actually tried to debug the crash with an older version of VS but it still needed me to sign in, damn. I trawled the Microsoft forums a lot, but couldn’t find anything there to help. I tried turning off the internet, connect via LAN and WiFi, manually installing the license, at one low point I even tried a virus scan.

Reinstalling Windows seemed like a good idea at this point, however without a valid Windows 8 key, we had to use the free upgrade to Windows 10. Now some of you will have spotted the problem that I am about to run into, that’s right, Kinect V1 doesn’t run on Windows 10. It doesn’t even use the same SDK as the V2.

A few days spent waiting for the new Kinect, and a new operating system later, we seem to be ready to go. I sign in to VS2017, the sign in button seems to perform the function you would expect it to on Windows 10, so that’s a plus. I plug in the new Kinect, but something is wrong, I load up the Kinect config tool.

Oh no. Why. What did I do to in a previous life to deserve this, etc.

Kinect V2 can transfer much more data per second than its predecessor, in fact so much more that a regular USB 2.0 won’t cut it, it needs a USB 3.0. Something that the computer I was given does not provide, in fact at this point all it seems to provide is misery.

It has been around a week and a half of delays at this point, so waiting for a USB 3.0 PCI card to arrive didn’t seem to bad, I have Amazon Prime and just about made it for next day deliver so it wasn’t too bad at all. Inside the machine I had to move some components around to get the power cables to reach the new card, but all in all it wasn’t too bad. Remembering to plug in the Kinect helped a lot at this point, in fact it all just clicked into place after that.

Using cutting edge technology we can generate a 3D point cloud of a software student’s raw joy, as after 2 weeks of trying he successfully installed a fancy webcam.

Only took a change of OS, IDE, Kinect, USB Card, Moving the GPU, Downloading some drivers and some other bits and bobs and now I can begin to work on my Research.

 

To be honest it is fun to solve these kinds of problems, and in the long run I probably learnt some valuable lessons about Windows development (avoid it being the main one I think). And now I have set up a client that can send live skeleton data from a Kinect, through a Raspberry Pi broker, and back to whoever wants to subscribe to it and read the stream. (It is normally tracking me at my desk so I probably won’t share the ip, it’ll only be upsetting I’m sure). Which to be honest is pretty neat, I’ve never done anything like this before, and I’m not completely put off this kind of thing in future.

The moral of this story is don’t give up, the longer it takes the more satisfying it is when it finally works. And to a lesser extent, software that requires you to sign in to ‘improve user experience’ is a real ass sometimes.

Leave a Reply

Your email address will not be published. Required fields are marked *