Controlling ANY game with the P5, any way you want

You can play any game with the P5 by emulating the Mouse, Joystick, Keyboard, Midi input device, or Speech input (for games that use SAPI 5).

If you have a Windows PC, then you will need to download PIE (Programmable Input Emulator) to do this. You should also download and install PPJoy to emulate a joystick with PIE, ~MidiYoke to emulate a MIDI input device with PIE, and SAPI 5 to emulate speech input. You will need Windows 2000, or Windows XP to emulate the keyboard in ~DirectInput games with PIE.

If you don't have a Windows PC, then you can still [Mouse Mode Gaming|emulate a mouse using the P5's built in hardware Mouse Mode]. And you can probably find a program to emulate a Midi input device for your platform.

Styles of Game Control

There are several different ways you can use a Virtual Reality glove to control an existing game:

As a 3D mouse, 3D Joystick, or 6DOF controller

The simplest (and most boring) way is to think of the glove as an input device with up to 6 axes (up/down, left/right, forwards/backwards, roll, angle from side to side, and angle up/down) and 8 or 9 buttons (the five fingers plus the 3 or 4 buttons on the back of the hand).

Many games these days will support a joystick with this many axes and buttons.

Download and install PIE and PPJoy and see the ~DirectInputP5.PIE file to see how this is done. Or read the tutorial that comes with PIE.

Or you can download and install my old ~DirectInput Emulator and PPJoy if you really don't like PIE.

As 5 Axes of Finger Bends

You could also use the finger bends to control analog joystick or mouse axes. For example, using one finger to control throttle smoothly from 0 to 100%. Or you could use the thumb bend for horizontal movement and the index finger bend for vertical movement. Or any other analog controls you want to control with your fingers.

As Hand Gestures to Control Game Actions

Gestures are much more fun. You can use combinations of the hand's location, which way it is facing, which fingers are bent, what direction it is moving, and its speed to activate various game functions. If you want to be really fancy you can use sequences of gestures.

PIE features lots of P5 variables you can use to recognise gestures:

  • The hand's absolute location
    • x, y, z
    • ~TooFarForward, ~TooFarBack, ~TooFarLeft, ~TooFarRight, ~TooFarUp, ~TooFarDown
  • Which way the hand is facing (this is purely hand rotation and has nothing to do with finger or thumb bends)
    • These properties will be true when that side of the hand is facing closer to the specified direction than any of the other directions. For example ~PalmUp will be true when the palm is facing upwards, until you rotate it 45 degrees in any direction at which point it will change to ~PalmLeft, ~PalmForwards, ~PalmRight or ~PalmBack. You need a combination of two of these to specify a complete hand orientation for example "P5.PalmUp and P5.FingersForward" is the same as "P5.PalmUp and P5.ThumbRight".
      • ~PalmUp, ~PalmDown, ~PalmLeft, ~PalmRight, ~PalmForwards, ~PalmBack
      • ~FingersUp, ~FingersDown, ~FingersLeft, ~FingersRight, ~FingersForward, ~FingersBack
      • ~ThumbUp, ~ThumbDown, ~ThumbLeft, ~ThumbRight, ~ThumbForwards, ~ThumbBack
    • There are also diagonal versions, for example ~PalmLeftUp or ~PalmLeftUpForward.
    • There are also vague versions, for example ~PalmUpVague, which is true whenever the palm is closer to facing upwards than it is to facing downwards. These only have to be within 90 degrees of the specified orientation, instead of 45 degrees like the above versions.
    • There are also strict versions, for example ~PalmUpStrict, which is true whenever the palm is within 22.5 degrees of the specified orientation.
    • There are also versions for checking if it is within a certain angle of that direction. eg. if p5.PalmAngleFromUp > 30 degrees then ...
    • Roll, Pitch, and Yaw can be used when you want analog values, but they are a pain to use and I don't recommend them.
    • Mat11, Mat12, Mat13, Mat21, Mat22, Mat23, Mat31, Mat32, Mat33 to access the rotation matrix fields directly.
    • Rotation to access the whole rotation matrix
  • Which fingers are bent
    • You can use a patter of five letters to represent the state of each finger. "l" means straight, "r" means partly bent, "n" means completely bent (The shape of the letters matches the shape of the finger!) "x" means don't care.
      • eg. p5.xlnnn (pointing), p5.xllll (flat), p5.xnnnn (fist), p5.xrrrr (claw), etc.
    • You can also check each finger bend from 0 to 63
      • eg. if p5.index > 32 then ...
  • What direction, and how fast it is moving
    • p5.XVelocity, p5.YVelocity, p5.ZVelocity
    • p5.XAcceleration, p5.YAcceleration, p5.ZAcceleration
    • p5.PitchVelocity, p5.YawVelocity, p5.RollVelocity
    • p5.PitchAcceleration, p5.YawAcceleration, p5.RollAcceleration
    • p5.Speed
    • p5.Velocity
    • p5.Acceleration

for example, to recognise a moving thumbs-down gesture, we might do something like this:

var.ThumbsDown = p5.lnnnn and p5.ThumbDown and p5.YVelocity < -1000

Tracking Other Body Parts

You can also attach the glove to your feet (be careful not to kink the bend sensors when you do this) or your head for other ways of controlling games. For example you can play Dance Dance Revolution (actually you should use Step Mania) with a glove on each foot.

As Full Body Realistic Gestures

My favourite way of using the glove to control games is to use full-body gestures. This means:

Jump by jumping Crouch by crouching Punch by punching Walk by walking on the spot Turn by swinging your body in that direction Raise or lower your weapon by raising or lowering your hand Aim by aiming etc.

You need a complicated PIE script to recognise these full body actions just from your hand's position, but it can be done. See Doom3.PIE for an example of how it is done with a single glove on one hand.

It's also fun to play games like Tekken this way.

Using Speech and a P5

You can also use PIE to recognise voice commands and use them to control a game.

For example you could make a script to control a first person shooter with a glove plus have spoken commands like "rocket launcher" to change weapons.

eg. in PIE you could include these lines in a script to add speech recognition...

Key.One = said("pistol") Key.Two = said("shotgun") Key.Three = said("rocket launcher")