illiWii: A Friendly Foray into Frugal Flying

Last edited April 29, 2008 by Yost Smith
Find this document at http://cheer.math.edu/reulab/web/ytsmith2
To return to the index of other students' work, click here.

By Yost Smith

Abstract:

The objective of this project is to take the Nintendo Wiimote and use its Bluetooth capabilities to wirelessly control any Syzygy application. For more information about the Nintendo Wii not provided here see this site. As will be explained below, the Wiimote is at heart very similar to any other piece of electronic equipment, it takes external stimuli and converts them to voltages/currents which are transmitted via bluetooth connection to the computer. Syzygy is made to take standardized input from a controller through the device's driver. In between Syzygy and the driver is a filter, created using a programming language called pForth, short for pseudo-Forth. When a Syzygy application is run, Syzygy looks in the current working directory for a parameters file (in our case called szg_parameters.xml). If none exists Syzygy uses the default parameters. By running a Syzygy application from a working directory in which there exists a modified version of the default parameters file (in which we've written a pForth filter to interpret the Wiimote's data stream) we can use the Wiimote to control the simulation. This project aims to create a filter called wii_rc which will can be used to give a realistic feeling of flying in the CAVE. One hand will be mapped to the roll and pitch of the head, and the other will be mapped to the throttle and yaw.

Background Information:

I. The Wiimote

     The Nintendo Wiimote shows much promise as a potential controller for moving through the three-space and four-space simulations that can be run in the CAVE and the CUBE due to its relatively cheap cost, flexibility and ease of programming. The Wiimote itself has an ADXL330 accelerometer which can sense acceleration in all three principal axes, and also uses a CCD to interpret infrared light from the sensor bar in order to determine the direction in which the Wiimote is pointing. This sensor bar could just as easily be replaced with two candles or two flashlights, both sources of infrared radiation. The accelerometers, the CCD, the directional pad, the analog stick, and the buttons all return voltages to the Wiimote's onboard computer, which takes these and converts them into either a boolean variable (0 or 1) for buttons and the D-pad directions or an axis variable (a float) for the accelerometers. The Wiimote then sends these via a bluetooth connection to whatever device/computer it is paired with (in the intended case the Wii). These capabilities, combined with the Wiimote's attractive price factor, make it a very viable controller for the CAVE and CUBE.

II. pForth

     pForth is a very straightforward language to use, and can be correctly described as "A series of words separated by whitespaces." There is a group of pForth words known as Standard Vocabulary which are built into pForth, and one can also define new words. The other primary concept to pForth, other than that of words and whitespaces is that of the stack. Imagine if you will a large stack of boxes into which numbers can be stored, and you have a visual representation of pForth's stack. There are two stacks in pForth, the Data Stack (the primary one with which we'll be working) and the Return Stack (primarily used by the system). The Data Stack is usually just referred to as the stack. When one places numbers on the stack the last number placed on the stack is on top. Operations can be performed on the stack's members and on the stack itself. Our use, and the common use of pForth, is to assign values to stack members and to manipulate them using default and user defined operators. pForth uses Reverse Polish Notation(RPN), so to add 1 and 2 we would have to enter "1 2 +". All this says is (1) Put 1 on the stack (2) Put two on the stack and (3) Pop the last two entries on the stack off and add them together, then put the resultant back on the stack. In our case, the driver by default defines two rotation matrices called the head and wand matricies, and all of our stack operations are designed to manipulate these.

III. Control Systems

     According to basic flight dynamics, there are three critical parameters, in this case rotations, to flight. These are known as roll, pitch and yaw, and refer respectively to rotations about the z, x and y axes. Traditionally the joystick in an airplane controls the throttle (when pushed forward/pulled back) and the roll (when pushed to the side). Yaw and pitch are manipulated using foot pedals. The control system I am designing for the Wiimote will use the Wiimote in the left hand to control throttle and yaw with its pitch and roll, respectively, and the Nunchuck in the right hand to control the pitch and the roll with its pitch and roll, respectively. To apply any of these parameters to the rotation matrix for our object (which really just ends up being the sum of all actions on the object) we simply multiply it by the appropriate matrix. The roll, pitch and yaw multiplication matricies are listed below, respectively.

The Code

As mentioned in the abstract, to use the Wiimote to control a Syzygy application we simply run that application from a directory in which there exists our custom szg_parameters.xml file. You can download a copy of that configuration file here. The szg_parameters.xml (in our case re-named to yost_parameters_final.xml) file contains several profiles, the name of which is at the top of each section, <param><name>NAME</name><value>. If you don't feel like reading through the entire code, I've put a sample segment of the code up here. This profile is called wii_rc and does quite a few things as you can read about below. It helps when trying to read pForth to have a copy of the documentation handy. It may also help to look at the button/axes mapping sheet that I've provided just above the code below.

Disclaimer: This was not intended to be a full commenting of the code. For that please see my hand annotated code.

The first half of the code, up to where we define sign, just creates a bunch of matrices, vectors and variables and gives some of them initial values. For instance, we store the translation matrix [0 5 0] into fixHeadMatrix. "define sign" just creates a command called sign that will return either a 0, 1 or -1 dependent upon what number its fed. In the next section of the code we basically just tell the program to set the head matrix (what we're looking at) to the wiimote, in this case defined by matrix_0 which we started out by filtering by. Next we filter matrix_1, which is the nunchuk matrix. We create the conditional that if the 12 button (in this case Z) is pressed the nunchuk's roll is extracted and inserted as an axis event for the zeroth index (which is also known as the yaw). If the twelve button is not depressed nothing happens. Almost done. Now all we do is use the 6th Euler Angle, which happens to be the pitch of the nunchuck, we multiply it by -1 (to reverse the direction) and insert it as an axis event with index 1 (traditionally interpreted as thrust or fore/aft motion). And that's it. We've just set the Wiimote's pitch and roll to control the pitch and roll of the head, respectively, and the Nunchuk's pitch and roll to control fore/aft motion and yaw, respectively.

Conclusion

     The general idea behind wii_rc was to give the user the feel of flying an airplane, with one hand they would control the throttle and yaw and with the other they would control the pitch and the roll. Unfortunately the actual feel of flying an airplane isn't easily attainable and was not completed by this project. This can be done but requires a scenegraph to be made for the particular application which includes the entire world. Rolls and pitches that should then control the airplane would instead just roll/pitch the entire world in the opposite direction. This could perhaps be included in a future project. I'd like to take this part of my site up to a huge thank you to everyone who helped me get this project done. Thanks go out to Jim Crowell, George Francis and Camille Goudeseune, in particular to Professor Francis and to Camille who selflessly helped me out in my final hour(s). I've learned a lot and I've had some fun, so thank you guys.

Random tidbits
There are a few things that Professor Francis wanted me to make sure and write down so that they weren't lost for future generations, so here goes:

     1. Whenever you are editing some important file, it is always a good idea to make a backup before hand.
     2. Ubuntu Feisty Fawn apparently doesn't really like the nVidia graphics card that was in our Dell, so whenever you want to change your input sources (i.e. switch from using a monitor to a projector) you have to to a total restart of the machine.
     3. The command in gvim to remove all the annoying ^M's that appear in a file after editiing it with some GUI text editor such as the one that comes with Ubuntu is: %/^Q^M// where ^Q and ^M correspond to hitting ctrl+M and ctrl+M. To do it in regular old vim you type: %s /^M// where ^M can be obtained by pressing ctrl+v ctrl+M.
     4. You can put gvim into compatibility mode (where it uses most of the same shortcuts as vim) by typing :set compatibility.
     5. You should restart the server that the Wiimote connects to in the cave every time you run a different application. Do this by typing: ./szgd /home/cube/dev/szg/bin/linux -r on the command line of the server in the back room that the bluetooth is connected to.
     6. For a .pdf of my presentation click here.

References:

1. "Nintendo Wiimote" - Wikipedia's article on the Nintendo Wiimote
2. "Syzygy Documentation: PForth Input-filtering Language" - Jim Crowell's pForth tutorial
3. "Rotation Matrix" - Wikipedia's article on rotation matricies
4. "Flight Dynamics" - Wikipedia's article on flight dynamics
5. Camille Goudeseune - Seriously, this guy's awesome.
6. Professor Francis - The one, the only.