Multi-Touch Handling

SIO2,

I think we need a bit of a change to touch handling. Currently, there is _SIO2windowtouchmove and _SIO2windowtap. _SIO2windowtap will return the state SIO2_WINDOW_TAP_UP or SIO2_WINDOW_TAP_DOWN. This wont work with multi touch. Here is an example interaction:

1. User touches down finger 1. ( SIO2windowtap -> SIO2_WINDOW_TAP_DOWN and sio2->_SIO2window->n_touch = 1 )
2. User touches down finger 2. ( SIO2windowtap -> SIO2_WINDOW_TAP_DOWN and sio2->_SIO2window->n_touch = 2 )
3. User touches down finger 3. ( SIO2windowtap -> SIO2_WINDOW_TAP_DOWN and sio2->_SIO2window->n_touch = 3 )
4. User lifts finger 2. ( SIO2windowtap -> SIO2_WINDOW_TAP_UP and sio2->_SIO2window->n_touch = 0 )

So, in step 4, there are still 2 fingers down but sio2->_SIO2window->n_touch is showing 0. Not only that, but there is no way to tell which finger was lifted.

For the case of only dealing with 1 touch at a time the current system works fine. When dealing with 2 or more touches, things start to break down. Maybe it would be a good idea to change the sio2->_SIO2window->touch to be an array of SIO2Touch where sio2Touch is something like this:

Code:


typedef struct {
  vec2d  pos;
  UITouchPhase phase;
  int tapCount;
} SIO2Touch;


You can get the UITouchPhase and tapCount from each UITouch object. UITouchPhase is defined in UITouch.h as:

Code:


typedef enum {
    UITouchPhaseBegan,            // whenever a finger touches the surface.
    UITouchPhaseMoved,            // whenever a finger moves on the surface.
    UITouchPhaseStationary,        // whenever a finger is touching the surface but hasn't moved since the previous event.
    UITouchPhaseEnded,            // whenever a finger leaves the surface.
    UITouchPhaseCancelled,        // whenever a touch doesn't end but we need to stop tracking (e.g. putting device to face)
} UITouchPhase;


It also seems like it would be useful for widgets to get a touch first with the option to then cancel the touch if they handle it so it doesn't get sent off to othe normal template touch handler.

If you think this is a good plan, it would be pretty easy to whip up the code to handle it. What do you think?

this is all fixed in 1.3.4...

Awesome. Is it in the svn tree yet? I didn't see any changes to the touch handling there last I checked...

sio2interactive wrote:this is all fixed in 1.3.4...

Can you give an example of how to use this fix? I've downloaded 1.3.5, and it still seems like this is an issue.

From what I can tell, when a touch even happens, it's simply stored inside an array in the SIO2 framework; that's how we access it from our apps. But there seems to be a fundamental problem with this (or I don't understand) - because, even if you have both fingers DOWN on the screen, if you only move ONE you only get ONE move event, and have no idea which finger it was!

EG:
1: Finger 1 down at 100x100 -> Get event of a down at 100x100
2: Finger 2 down at 300x300 -> Get event of a down at 300x300. Can assume this is the second finger because we haven't received an up yet?
3: Move Finger 2 to 200x200 -> Get event of a move at 200x200 -- but there's no way to tell WHICH finger moved, because n_touch is 1 no matter which finger it was.

Also, Tutorial 5 is supposed to demonstrate touch events, but just seems to have three empty functions at the bottom instead.

Kudos on the framework in general though - it's rocking my world right now =)

The problems are definitely still around. What we need is something like this:

Code:


typedef struct {
   vec2         pos;
   unsigned int   tapCount;
   unsigned int   phase;
} TouchObject;

typedef struct {
   unsigned int   n_touch;
   TouchObject      *touches[5];
} TouchEventObject;


- (TouchEventObject*)extractTouches:(NSSet *)touches
{
   UITouch *touch;
   CGPoint pos;
   TouchEventObject *touchEvent;
   
   for( touch in touches )
   {
      TouchObject *t;
      
      pos = [ touch locationInView:self ];
      
      t->pos.x = pos.y;
      t->pos.y = pos.x;
      t->tapCount = [touch tapCount];
      t->phase = [touch phase];
      
      touchEvent->touches[ touchEvent->n_touch ] = t;
      
      ++touchEvent->n_touch;
   }
   
   return touchEvent;
}

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
{
   TouchEventObject *touchEvent = [self extractTouches:touches];
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                       sio2->_SIO2window,
                       SIO2_WINDOW_TOUCHES_BEGAN,
                       touchEvent );
}

- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event
{
   TouchEventObject *touchEvent = [self extractTouches:touches];
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                       sio2->_SIO2window,
                       SIO2_WINDOW_TOUCHES_MOVED,
                       touchEvent );
}

- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event
{
   TouchEventObject *touchEvent = [self extractTouches:touches];
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                       sio2->_SIO2window,
                       SIO2_WINDOW_TOUCHES_ENDED,
                       touchEvent );
}


That should cover all possible use cases and provide enough information. Of course the code isnt complete as the sio2ResourceDispatchEvents function call doesn't accept a userData parameter as specified.

Hummmmm I see what you mean... it is indeed more convenient... but that require a bit of modification on the events system.

I'll include that asap...

Cheers,

Not sure if this issue is fixed yet, but in the meantime I thought I'd ask if it's possible to retrieve the position on a SIO2_WINDOW_TAP_UP event? I don't actually need to know which touch it was, just where.

Check tutorial05

I don't think what I'm looking for is in tutorial 5, unless I'm missing something.

What I need is to know where the finger leaves the surface in cases of multi-touch. If I could identify which specific touch was releasing, it would also be sufficient, but according to this thread it seems that can't be done, at least not without working with touchesEnded directly. Am I mistaken?

I'm pretty sure you can determine where a finger was released in the same way you figure out where it is when it placed. For example, my event handler contains something like:

Code:


for(int i = 0; i < sio2->_SIO2window->n_touch; i++)
    if(_state == SIO2_WINDOW_TAP_UP)
        CurrentGame->LocalUI()->MouseUpEvent(sio2->_SIO2window->touch[ i ].x, sio2->_SIO2window->touch[ i ].y, i);
    else
        CurrentGame->LocalUI()->MouseDownEvent(sio2->_SIO2window->touch[ i ].x, sio2->_SIO2window->touch[ i ].y, i);


You will need to use the touchesBegan/touchesMoved/touchesEnded yourself. SIO2 doesn't properly handle all touches yet and tracking touches oftentimes is made far easier by just retaining a touch on touchesBegan and comparing it in touchesMoved/touchesEnded.

I belive that's isn't the best solution, but may help someone, like me

In the EAGLView.mm replace those methods:

Code:


- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
{
   UITouch *touch;
   CGPoint pos;
   
   NSSet* allTouches = [event allTouches];
   sio2->_SIO2window->n_touch = 0;

   for( touch in allTouches )
   {
      pos = [ touch locationInView:self ];

      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].x = pos.y;
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].y = pos.x;
         
      ++sio2->_SIO2window->n_touch;
   }      
   
   sio2->_SIO2window->n_tap = [ [ touches anyObject ] tapCount ];
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                        sio2->_SIO2window,
                        SIO2_WINDOW_TAP,
                        SIO2_WINDOW_TAP_DOWN );
}


- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event
{
   UITouch *touch;
   CGPoint pos;
   
   NSSet* allTouches = [event allTouches];
   sio2->_SIO2window->n_touch = 0;
   
   for( touch in allTouches )
   {
      pos = [ touch locationInView:self ];
      
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].x = pos.y;
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].y = pos.x;
      
      ++sio2->_SIO2window->n_touch;
   }   
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                        sio2->_SIO2window,
                        SIO2_WINDOW_TOUCH_MOVE,
                        SIO2_WINDOW_TAP_DOWN );
}


- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event
{
   UITouch *touch;
   CGPoint pos;
   
   NSSet* allTouches = [event allTouches];
   sio2->_SIO2window->n_touch = 0;
   
   for( touch in allTouches )
   {
      if (touch.phase == UITouchPhaseEnded) {
         continue;
      }
      pos = [ touch locationInView:self ];
      
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].x = pos.y;
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].y = pos.x;
      
      ++sio2->_SIO2window->n_touch;
   }   
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                        sio2->_SIO2window,
                        SIO2_WINDOW_TAP,
                        SIO2_WINDOW_TAP_UP );
}


BUT the magic will really happens when you put this code in the first line of - (BOOL)createFramebuffer method:

Code:


[self setMultipleTouchEnabled:YES];


I hope helped someone

magictech wrote:I belive that's isn't the best solution, but may help someone, like me

In the EAGLView.mm replace those methods:

Code:


- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event
{
   UITouch *touch;
   CGPoint pos;
   
   NSSet* allTouches = [event allTouches];
   sio2->_SIO2window->n_touch = 0;

   for( touch in allTouches )
   {
      pos = [ touch locationInView:self ];

      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].x = pos.y;
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].y = pos.x;
         
      ++sio2->_SIO2window->n_touch;
   }      
   
   sio2->_SIO2window->n_tap = [ [ touches anyObject ] tapCount ];
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                        sio2->_SIO2window,
                        SIO2_WINDOW_TAP,
                        SIO2_WINDOW_TAP_DOWN );
}


- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event
{
   UITouch *touch;
   CGPoint pos;
   
   NSSet* allTouches = [event allTouches];
   sio2->_SIO2window->n_touch = 0;
   
   for( touch in allTouches )
   {
      pos = [ touch locationInView:self ];
      
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].x = pos.y;
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].y = pos.x;
      
      ++sio2->_SIO2window->n_touch;
   }   
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                        sio2->_SIO2window,
                        SIO2_WINDOW_TOUCH_MOVE,
                        SIO2_WINDOW_TAP_DOWN );
}


- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event
{
   UITouch *touch;
   CGPoint pos;
   
   NSSet* allTouches = [event allTouches];
   sio2->_SIO2window->n_touch = 0;
   
   for( touch in allTouches )
   {
      if (touch.phase == UITouchPhaseEnded) {
         continue;
      }
      pos = [ touch locationInView:self ];
      
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].x = pos.y;
      sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].y = pos.x;
      
      ++sio2->_SIO2window->n_touch;
   }   
   
   sio2ResourceDispatchEvents( sio2->_SIO2resource,
                        sio2->_SIO2window,
                        SIO2_WINDOW_TAP,
                        SIO2_WINDOW_TAP_UP );
}


BUT the magic will really happens when you put this code in the first line of - (BOOL)createFramebuffer method:

Code:


[self setMultipleTouchEnabled:YES];


I hope helped someone

Yes It helped Me, by default sio2 wasn't updating n_touch when another finger was putted on the screen

Hmmm.. the code works well to handle multi touch but it cause problem with the widgets.. when you press a widget, it calls SIO2widgettapdown but when you release it, it does'nt call SIO2widgettapup.. wich is a problem

edit : it seems that it's

Code:


if (touch.phase == UITouchPhaseEnded) {
         
         continue;
}


that cause the problem, but if you remove it the multitouch doesn't work well. I have no idea how to fix that...

Check the meditation garden source I demonstrate how to use the SIO2_WIDGET_VALIDATE_LOCK to achieve exactly that... Basically its simulating the phase end. The reason why is, if SIO2 have to be ported to another platform it will still work

The mod would really help me too but I get errors when I build:
Line Location EAGLView.mm:157: error: request for member 'x' in '*(sio2->SIO2::_SIO2window->SIO2window::touch + ((vec2**)(((unsigned int)sio2->SIO2::_SIO2window->SIO2window::n_touch) * 4u)))', which is of non-class type 'vec2*'

I am missing something, probably something real obvious to a real programmer. If one of you clever dudes could just put me straight I'd be grateful ...

cgen2 wrote:The mod would really help me too but I get errors when I build:
Line Location EAGLView.mm:157: error: request for member 'x' in '*(sio2->SIO2::_SIO2window->SIO2window::touch + ((vec2**)(((unsigned int)sio2->SIO2::_SIO2window->SIO2window::n_touch) * 4u)))', which is of non-class type 'vec2*'

I am missing something, probably something real obvious to a real programmer. If one of you clever dudes could just put me straight I'd be grateful ...

sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ].x
it's sio2->_SIO2window->touch[ sio2->_SIO2window->n_touch ]->x

for Y too and it's in every method

Ian,

Of course. Thanks heaps. You're a champion