Tag Archives: Gestures

Detecting Keyboard Gestures Using Monogame

Recently, I got to the stage where I had just about finished one of my games, and suddenly realised that I had done nothing to deal with mouse gestures. Dealing with touch (which I have previously blogged about) can blind you to other input forms. Exactly how you deal with this will, of course, depend on what you’re trying to achieve.

Simple press and click – Touch

Let’s have a look at a piece of code to handle touch gestures:

private TouchCollection _currentTouch;

private void ProcessTouchInput()
{
    if (keypad == null) return;

    _currentTouch = TouchPanel.GetState();
    foreach( var touch in _currentTouch)
    {
        if (touch.State == TouchLocationState.Released)
        {
            keysPressed += keypad.GetKeyAt(touch.Position);
        }
    }
}

Okay, so this was ripped right out of the middle of a game. Most of it doesn’t matter, but `keypad` represents a customised on-screen keypad. So, to find out what’s been pressed, we simply call TouchPanel.GetState() and iterate the returned collection.

If we find that the TouchLocationState to be released then it must have been touched (how can you release something that was never touched?).

It then builds up a string containing the keys.

… And with mouse

Okay, here’s the first thing; this statement:

If we find that the TouchLocationState to be released then it must have been touched (how can you release something that was never touched?).

Is no longer true. `Released` where a mouse state is concerned is simply an absence of being pressed!

So, we need to track what has happened, and when:

private DateTime? _mousePressed;
private MouseState _currentMouse;

private void ProcessMouseInput()
{
    if (keypad == null) return;
           
    _currentMouse = Mouse.GetState();
    if (_currentMouse.LeftButton == ButtonState.Pressed)
         _mousePressed = DateTime.Now;

    if (_currentMouse.LeftButton == ButtonState.Released
       && _mousePressed != null)
    {
        _mousePressed = null;
        keysPressed += keypad.GetKeyAt( new Vector2(_currentMouse.X, _currentMouse.Y));               
    }
}

Okay, so as far as mouse press is concerned, we have to roll our own. This isn’t particularly complicated code though.

What about gestures?

You can do gestures; it’s not straight-forward and, again, it’s not particularly well supported.

Here’s how I handle dragging:

MouseState mouseState = Mouse .GetState();
Vector2 mousePosition = new Vector2(mouseState.X, mouseState.Y);

// First, work out where and whether the drag has started
if (mouseState.LeftButton == ButtonState.Pressed)
{
if (_lastPressed == null || _lastPressed < _lastRelease) dragStart = null; // The current mouse location must be in a place where it // is acceptable to start a drag if (CheckLocation(mousePosition, acceptableDragStart, 60)) { if (dragStart == null) { dragStart = mousePosition; } } _lastPressed = DateTime.Now; } // Now work out when the drag is released if (mouseState.LeftButton == ButtonState.Released) { // For some reason, ButtonState.Released is // always flagged unless the button is pressed if (_lastPressed == null) return; _lastPressed = null; // Set the last release here _lastRelease = DateTime.Now; // Do whatever it is that you need for drag release DragRelease(mousePosition, dragStart, PlayerPosition); } [/sourcecode] So the code is in two distinct sections; the first being to determine is, and where, the mouse was pressed, and to store that information; the latter to determine if and where it was released. Double tap, hold, and other such gestures make this code more and more complex. It feels a lot like the main issue is the Released state.

Detecting multiple gesture types using Monogame in Windows 8

Gestures in this environment are hard. Ridiculously hard. There is a ReadGesture function, but it doesn’t work. Let’s imagine that you want to detect the following gestures:

Tap
Double Tap
Drag

Not exactly complex. We’ll need to know some basic information about each gesture:

Tap: where have we tapped
Double tap: where have we double tapped (and that the two taps are in the same location)
Drag: the start and end positions of the drag.

Let’s have a look at some code:

     
TouchPanel.EnabledGestures =
GestureType.FreeDrag | GestureType.DragComplete | GestureType.DoubleTap | GestureType.Tap;

while (TouchPanel.IsGestureAvailable)
{
    GestureSample gs = TouchPanel.ReadGesture();
    switch (gs.GestureType)
    {
        case GestureType.FreeDrag:
            System.Diagnostics.Debug.WriteLine(string.Format("FreeDrag: {0}", gs.Position));
            break;

        case GestureType.DragComplete: 
            System.Diagnostics.Debug.WriteLine(string.Format("DragComplete: {0}", gs.Position));

            break;

        case GestureType.DoubleTap:
            System.Diagnostics.Debug.WriteLine("DoubleTap");

            break;

        case GestureType.Tap:
            System.Diagnostics.Debug.WriteLine("Tap");

            break;

    }
}

So, let’s run this and see what we get. We certainly get all the gestures. However, there are the following issues:

    1. The DragComplete doesn’t provide any information at all, other than the fact that the drag isn’t taking place anymore

    2. The drag works in bursts, so you get many drag events for a single drag

    3. The Tap event fires, but a Double Tap doesn’t replace the Tap, so you get both.

There’s a bit of rolling-your-own here. Let’s start with the drag, because it’s actually the easiest problem to solve. We know when the use finishes dragging, so a simple variable to say that they are dragging and where they started will solve most of this problem:

    private Vector2? dragStart = null;

Why is it nullable? Well, this variable stores two separate states: whether the user is dragging, and where they started.

Next, we need to know where they were last (if anyone from Microsoft ever reads this, please feel free to give a detailed description in the comments why this hoop jumping is necessary!):

    private Vector2? lastPosition = null;

Then we need to store and check these variables; here’s the new section of the switch statement:

    case GestureType.FreeDrag:
                                System.Diagnostics.Debug.WriteLine(string.Format("FreeDrag: {0}", gs.Position));
                        
        if (!dragStart.HasValue)
        {
            dragStart = gs.Position;
        }
        
        lastPosition = gs.Position;
            break;

       case GestureType.DragComplete:                    

           System.Diagnostics.Debug.WriteLine(string.Format("DragComplete: {0}", gs.Position));
                       
           if (dragStart.HasValue)
           {
               var gsDragDelta = lastPosition - dragStart.Value;
               if (gsDragDelta.Value.LengthSquared() > DRAG_TOLERANCE)
               {
                   if (gsDragDelta.Value.X != 0 || gsDragDelta.Value.Y != 0)
                   {
                       dragStart = null;
                       // Do stuff on drag complete
                       break;

That handles the drag. Next, the tap. Handling the Tap is easy, but what if, as I did, you want to handle Tap to do one thing, and Double Tap to do another? This fried my brain for a while; then something occurred to me. What if you delay the action:

    
    case GestureType.DoubleTap:    
        dblTap = true;
        // Do stuff on Double tap                  
        System.Diagnostics.Debug.WriteLine("DoubleTap");
        Task.Delay(100).ContinueWith((args) => { dblTap = false; });
        break;

    case GestureType.Tap:
        Task.Delay(100).ContinueWith((args) =>
        {
            if (!dblTap)
            {
                // DO stuff on Tap                
                System.Diagnostics.Debug.WriteLine("Tap");
            }
        });
        break;

I imagine the delay needs to be experimented with. 100ms seems to work for me.

As usual, if you have any suggestions, comments, or ways to do this better then please leave a comment.