Physics-based picking

View previous topic View next topic Go down

Physics-based picking

Post  matt on Fri Oct 03, 2008 5:11 pm

I've been trying to implement physics-based picking based on the Bullet demos, DemoApplication.cpp in particular.

First, I've added a pointer to the corresponding SIO2object for each btRigidBody:
Code:
...
      _SIO2physic->_btDiscreteDynamicsWorld->addRigidBody( _btRigidBody );
      
      _SIO2object->_btRigidBody = _btRigidBody;
      _btRigidBody->setUserPointer( _SIO2object ); // Enables getting a SIO2object* from a btRigidBody
   }
...
}

Then, I've implemented a SIO2 physics function which corresponds basically to what is implemented in Bullet's DemoApplication::mouseFunc and DemoApplication::getRayTo:
Code:
inline btVector3 getCameraPosition( SIO2camera *_SIO2camera )
{
   return btVector3( _SIO2camera->pos->x, _SIO2camera->pos->y, _SIO2camera->pos->z );
}

inline btVector3 getCameraTargetPosition( SIO2camera *_SIO2camera )
{
   return btVector3( _SIO2camera->tar->x, _SIO2camera->tar->y, _SIO2camera->tar->z );
}

inline btVector3 getCameraUpVector( SIO2camera *_SIO2camera ) {
   return btVector3( _SIO2camera->up->x, _SIO2camera->up->y, _SIO2camera->up->z );
}

btVector3 getRayTo( SIO2camera *_SIO2camera,
               float x,
               float y )
{
   // Several attributes which were hardcoded in the DemoApplication are available from SIO2,
   // e.g. the FoV is calculated there but we have it already available in the camera structure.

   float screenWidth = sio2->_SIO2window->size->x;
   float screenHeight = sio2->_SIO2window->size->y;
   float fov = _SIO2camera->fov * SIO2_DEG_TO_RAD;

   btVector3 rayFrom = getCameraPosition( _SIO2camera );
   btVector3 rayForward = ( getCameraTargetPosition( _SIO2camera ) - rayFrom );
   rayForward.normalize();
   float farPlane = _SIO2camera->cend;
   rayForward *= farPlane;
   
   btVector3 vertical = getCameraUpVector( _SIO2camera );
   
   btVector3 hor;
   hor = rayForward.cross( vertical );
   hor.normalize();
   
   float tanFov = tanf( 0.5f * fov );
   btScalar aspect = screenHeight / (btScalar) screenWidth;
   
   hor *= 2.f * farPlane * tanFov;
   vertical *= 2.f * farPlane * tanFov;
   
   if( aspect < 1 ) {
      hor /= aspect;
   } else {
      vertical *= aspect;
   }
   
   btVector3 rayToCenter = rayFrom + rayForward;
   btVector3 dHor = hor * 1.f / screenWidth;
   btVector3 dVert = vertical * 1.f / screenHeight;
   
   btVector3 rayTo = rayToCenter - 0.5f * hor + 0.5f * vertical;
   rayTo += x * dHor;
   rayTo -= y * dVert;
   
   return rayTo;
}

bool sio2PhysicPick( SIO2physic *_SIO2physic,
                  SIO2camera *_SIO2camera,
                  float x,
                  float y,
                  SIO2object** object,
                  float* px,
                  float* py,
                  float* pz )
{
   btVector3 rayTo = getRayTo( _SIO2camera, x, y );
   btVector3 camPos = getCameraPosition( _SIO2camera );
   btCollisionWorld::ClosestRayResultCallback rayCallback( camPos, rayTo );
   _SIO2physic->_btDiscreteDynamicsWorld->rayTest( camPos, rayTo, rayCallback );
   if( rayCallback.hasHit() )
   {
      btRigidBody* body = btRigidBody::upcast( rayCallback.m_collisionObject );
      if( body )
      {
         // For debugging purposes: check which object was hit.
         SIO2object *test = (SIO2object *) body->getUserPointer();
         if( !body->isStaticOrKinematicObject() ) {
            *object = (SIO2object*) body->getUserPointer();
            btVector3 pos = rayCallback.m_hitPointWorld;
            *px = pos.x();
            *py = pos.y();
            *pz = pos.z();
            return true;
         }
      }
   }
   return false;
}

Now I have a test scene with a huge cube as a container where the front and top faces are removed, and a little dynamic cube in the container which falls down once the application starts. Whenever I try to pick this little cube, I get the big one returned from this method.

Is someone willing to try this method out and help me out with it? I can't see any differences from the original Bullet code which could lead to problems.

Thanks,
Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  sio2interactive on Fri Oct 03, 2008 7:55 pm

Does this cube is an instance? (ex: Cube, Cube.001) If yes just get the handle of the parent instead...

SIO2object *tmp = _SIO2object->_SIO2object ? ( SIO2object * )_SIO2object->_SIO2object : _SIO2object;

By the way picking with bullet is relatively slow compare to the method used by SIO2 (like in tutorial06)... Depending on what kind of scene you have and the type of performance you are looking for maybe you should give a try to the embedded SIO2 method...
avatar
sio2interactive

Posts : 1526
Join date : 2008-08-26
Age : 38
Location : Shanghai

View user profile http://sio2interactive.com

Back to top Go down

Re: Physics-based picking

Post  matt on Sat Oct 04, 2008 2:45 am

sio2interactive wrote:Does this cube is an instance?
No, it shouldn't be. I'll double-check this.

[quote?"sio2interactive"]By the way picking with bullet is relatively slow compare to the method used by SIO2 (like in tutorial06)... Depending on what kind of scene you have and the type of performance you are looking for maybe you should give a try to the embedded SIO2 method...[/quote]
It's okay if you just need to know which object was clicked. But I also need the location in 3D world space where a ray from the users mouse position through the camera planes hits an object.

Best,
Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  sio2interactive on Sat Oct 04, 2008 3:14 am

sio2Unproject, will return you the XYZ position in worldspace from any coordinate on the screen (in other words 2D to 3D)
avatar
sio2interactive

Posts : 1526
Join date : 2008-08-26
Age : 38
Location : Shanghai

View user profile http://sio2interactive.com

Back to top Go down

Re: Physics-based picking

Post  matt on Sat Oct 04, 2008 5:38 pm

sio2interactive wrote:sio2Unproject, will return you the XYZ position in worldspace from any coordinate on the screen (in other words 2D to 3D)
Hm, mind to provide an example?

From my understanding, I would have to provide a z-buffer-value as the "winz" parameter. Normally, in OpenGL, you'd retrieve that value with glReadPixels and GL_DEPTH_COMPONENT as the type, but OpenGL|ES doesn't support that. So what's your solution? How do you use sio2Unproject?

Moreover, once this is solved, how can I quickly get the modelview/projection matrix without calling glGetDoublev... for all three? I guess SIO2 already has them available somewhere? Does the method already take into account that the OGL viewport has (0,0) at the bottom left or do I have to take care of this myself?

Best,
Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  sio2interactive on Sat Oct 04, 2008 6:46 pm

Same way you are using gluUnproject, however it is pretty slow due to the calculation, on a second thought maybe bullet method is faster cuz they already have a tree built...
avatar
sio2interactive

Posts : 1526
Join date : 2008-08-26
Age : 38
Location : Shanghai

View user profile http://sio2interactive.com

Back to top Go down

Re: Physics-based picking

Post  matt on Sun Oct 05, 2008 3:25 pm

I now mixed "best of both worlds" and use sio2UnProject for generating a test ray through far and near clipping plane, but still go for the Bullet stuff to find the hit object / point.

Code:
inline btVector3 getCameraPosition( SIO2camera *_SIO2camera )
{
   return btVector3( _SIO2camera->pos->x, _SIO2camera->pos->y, _SIO2camera->pos->z );
}

inline btVector3 getCameraTargetPosition( SIO2camera *_SIO2camera )
{
   return btVector3( _SIO2camera->tar->x, _SIO2camera->tar->y, _SIO2camera->tar->z );
}

inline btVector3 getCameraUpVector( SIO2camera *_SIO2camera ) {
   return btVector3( _SIO2camera->up->x, _SIO2camera->up->y, _SIO2camera->up->z );
}

btVector3 getRayTo( SIO2camera *_SIO2camera,
               float x,
               float y )
{
   GLfloat worldX1, worldY1, worldZ1;
   GLfloat worldX2, worldY2, worldZ2;
   GLfloat model[ 16 ];
   glGetFloatv( GL_MODELVIEW_MATRIX, model );
   GLfloat proj[ 16 ];
   glGetFloatv( GL_PROJECTION_MATRIX, proj );
   GLint viewport[ 4 ];
   glGetIntegerv( GL_VIEWPORT, viewport );
   
   y = viewport[ 3 ] - y;
   
   sio2UnProject( x, y, 0,
               model, proj, viewport,
               &worldX1, &worldY1, &worldZ1 );
   sio2UnProject( x, y, 1,
               model, proj, viewport,
               &worldX2, &worldY2, &worldZ2 );
   
   btVector3 v = btVector3( worldX2 - worldX1,
                        worldY2 - worldY1,
                        worldZ2 - worldZ1 );
   return getCameraPosition( _SIO2camera ) + v;
}

bool sio2PhysicPick( SIO2physic *_SIO2physic,
                  SIO2camera *_SIO2camera,
                  float x,
                  float y,
                  SIO2object** object,
                  float* px,
                  float* py,
                  float* pz )
{
   btVector3 rayTo = getRayTo( _SIO2camera, x, y );
   btVector3 camPos = getCameraPosition( _SIO2camera );
   btCollisionWorld::ClosestRayResultCallback rayCallback( camPos, rayTo );
   _SIO2physic->_btDiscreteDynamicsWorld->rayTest( camPos, rayTo, rayCallback );
   if( rayCallback.hasHit() )
   {
      btRigidBody* body = btRigidBody::upcast( rayCallback.m_collisionObject );
      if( body )
      {
         *object = (SIO2object*) body->getUserPointer();
         btVector3 pos = rayCallback.m_hitPointWorld;
         *px = pos.x();
         *py = pos.y();
         *pz = pos.z();
         return true;
      }
   }
   return false;
}

Works for me, maybe it's useful for others as well. Now I need to have a look how I could get more information about the triangle which was hit and its material...

Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  matt on Fri Oct 17, 2008 4:06 pm

Something must have changed from 1.1 to 1.2... I changed a test project to work with the new version. In 1.1, I could pick an object, move it around, and drop it. It was correctly rendered in all of these phases. In 1.2, after picking an object, it remains at the same place visually, but interacts with other objects at it's new position physically. Once I drop it, its rendered at correct position.

The position change takes place like in all the Bullet demos by changing a pivot point position on a btPoint2PointConstraint.

sio2interactive, do you have any ideas where I could look?

Best,
Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  sio2interactive on Fri Oct 17, 2008 5:37 pm

Hummmm try to comment _SIO2object->_btRigidBody->getActivationState( ) == ACTIVE_TAG is sio2_object.cc since when you pick your object is not necessarily activated... Lemme know if it work
avatar
sio2interactive

Posts : 1526
Join date : 2008-08-26
Age : 38
Location : Shanghai

View user profile http://sio2interactive.com

Back to top Go down

Re: Physics-based picking

Post  matt on Fri Oct 17, 2008 5:47 pm

sio2interactive wrote:Hummmm try to comment _SIO2object->_btRigidBody->getActivationState( ) == ACTIVE_TAG is sio2_object.cc
Yes, that's it. But I'm actually disabling deactivation when I pick the object...
Code:
selectedObject->_btRigidBody->setActivationState( DISABLE_DEACTIVATION );
Strange. Any ideas?

Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  sio2interactive on Fri Oct 17, 2008 7:24 pm

Heuuuuuu not strange, the state is DISABLE_DEACTIVATION and not == to ACTIVE_TAG Wink
avatar
sio2interactive

Posts : 1526
Join date : 2008-08-26
Age : 38
Location : Shanghai

View user profile http://sio2interactive.com

Back to top Go down

Re: Physics-based picking

Post  matt on Sat Oct 18, 2008 7:15 am

sio2interactive wrote:Heuuuuuu not strange, the state is DISABLE_DEACTIVATION and not == to ACTIVE_TAG Wink
Yeah, but doesn't DISABLE_ACTIVATION imply ACTIVE_TAG? Shouldn't your if include DISABLE_ACTIVATION as a possible condition? I'm not a Bullet expert so I might be wrong here but from my understanding DISABLE_ACTIVATION means the rigid body is active and remains in that state, while ACTIVE_TAG means it's active but could get deactivated by Bullet.

Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  sio2interactive on Sat Oct 18, 2008 5:13 pm


Yeah, but doesn't DISABLE_ACTIVATION imply ACTIVE_TAG? Shouldn't your if include DISABLE_ACTIVATION as a possible condition?

Yeah I think that'll do the trick...
avatar
sio2interactive

Posts : 1526
Join date : 2008-08-26
Age : 38
Location : Shanghai

View user profile http://sio2interactive.com

Back to top Go down

Can anyone send me an example to pick/move an object?

Post  ludi on Sun Oct 19, 2008 8:11 am

Hello,

Can anyone send me an example to pick/move an object? I can select an object but I do not have an idea how I can move the selected object. The problem is to convert between the touch coordinates to the 3d word coordinates.

Thanks a lot
Ludi

ludi

Posts : 14
Join date : 2008-10-19

View user profile

Back to top Go down

Re: Physics-based picking

Post  matt on Sun Oct 19, 2008 2:46 pm

ludi wrote:Can anyone send me an example to pick/move an object? I can select an object but I do not have an idea how I can move the selected object. The problem is to convert between the touch coordinates to the 3d word coordinates.
All relevant parts are provided right here in this thread. See my earlier post where I showed how I created a ray through near/far-clipping plane and then use Bullet to pick an object. One important thing if you want to use Bullet for picking is that you should set a btRigidBody's userPointer to the SIOobject* so that you know which body belongs to which SIO2 object. I have changed SIO2 in sio2PhysicsAdd...

Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  ludi on Mon Oct 20, 2008 1:25 am

I am quite new in Object-C and iPhone programming. It was not possible to create a working application with your hints. I will try it again.
It would be great if you can provide me an example.

Thanks
Ludi

ludi

Posts : 14
Join date : 2008-10-19

View user profile

Back to top Go down

Re: Physics-based picking

Post  ludi on Tue Oct 21, 2008 5:48 am

matt wrote:
ludi wrote:Can anyone send me an example to pick/move an object? I can select an object but I do not have an idea how I can move the selected object. The problem is to convert between the touch coordinates to the 3d word coordinates.
All relevant parts are provided right here in this thread. See my earlier post where I showed how I created a ray through near/far-clipping plane and then use Bullet to pick an object. One important thing if you want to use Bullet for picking is that you should set a btRigidBody's userPointer to the SIOobject* so that you know which body belongs to which SIO2 object. I have changed SIO2 in sio2PhysicsAdd...

Matt

Hi Matt,

I started with tutorial 6 and added your code and comments to the template.mm file.
Therefore I added your methods:
-getCameraPosition
-getCameraTargetPosition
-getCameraUpVector
-getRayTo
-sio2PhysicPick

Now I tried to add code to select the right object at the function templateScreenTap and move the object in templateScreenTouchMove.
I used the method getRayTo to calculate the touch coordinates to the coordinates in the 3D World and afterwards the sio2PhysicPick to move the object.
The program still does not work.

Can you give me an additional hint or a small example? That would be nice.

Thanks a lot
Ludi

ludi

Posts : 14
Join date : 2008-10-19

View user profile

Back to top Go down

Re: Physics-based picking

Post  matt on Tue Oct 21, 2008 9:14 am

Could you provide me with the code you already have so that I can see some more details on how you used the methods? Unfortunately, I cannot provide you with an isolated example myself because I've incorporated that into a larger project.

For example, did you remember to set the UserPointer in sio2PhysicAdd as shown in this posting? And do you also have the relevant properties set in Blender?

Best,
Matt

matt

Posts : 155
Join date : 2008-09-30

View user profile http://elfrun.net

Back to top Go down

Re: Physics-based picking

Post  sw on Tue Nov 25, 2008 6:51 am

thanks rom and matt, it is a very very good post.
avatar
sw

Posts : 73
Join date : 2008-10-12

View user profile

Back to top Go down

Re: Physics-based picking

Post  goldfrapp0x0309 on Sun Aug 09, 2009 3:14 am

After updating to 1.4, this picking method works after i call sio2getModel/Proj during render. The other problem is the actual picking precision... ? I have a horizontal plane, and when i pick on it, the point picked seems skewed... as if the frustrum was inverted or something. Its precise when i click in the middle of screen, but seems to have an ever growing offset when i click further from the center of the screen... ?

Any help appreciated, i use same code as on this post.

Thanks !
avatar
goldfrapp0x0309

Posts : 43
Join date : 2009-02-28

View user profile

Back to top Go down

Re: Physics-based picking

Post  sio2interactive on Sun Aug 09, 2009 3:16 am

Check tutorial062, 1.4 have 3D pick and drag based on physic...

_________________
SIO2 Interactive
Free Open Source 3D Game Engine for iPhone and iPod Touch
http://sio2interactive.com
avatar
sio2interactive

Posts : 1526
Join date : 2008-08-26
Age : 38
Location : Shanghai

View user profile http://sio2interactive.com

Back to top Go down

Re: Physics-based picking

Post  goldfrapp0x0309 on Mon Aug 10, 2009 2:43 am

Yup, forgot to reply, it was due to cend of camera. Wasn't far enough apparently and created that offset.

Thanks
avatar
goldfrapp0x0309

Posts : 43
Join date : 2009-02-28

View user profile

Back to top Go down

Re: Physics-based picking

Post  mordenkaim on Thu Aug 13, 2009 8:21 am

Mostly this makes sense to me, the only question I have is where do you get the values for the near and far clipping plane? In order for it to be a constant value it would have to be relative to the camera since the camera moves and this picking method still works. Are 1 and 0 just arbitrary values or is there a reason you picked them?

Also, I still haven't seen an explanation that I can understand that really explains the z coordinate for SIO2UnProject. The way I am thinking about it is the x, y is the location you click on the screen and the z would be the distance from the camera to the point you're trying to get the coordinates for in worldspace. So if the camera were looking straight down the y axis you would do camera.y - target.y to get the depth. So if my objects were at y=2.0 and the camera was located at y=6.0 the z coordinate for unproject would be 4.0. Of course this doesn't work if you're not looking directly down an axis, but for simplicity am I thinking the right way?

*Edit* As I've had more time to research I figured out that the z value is a float ranging 0.0 to 1.0 with 0.0 representing the near clipping plane and 1.0 representing the far clipping plane. So, .5 would be a depth of halfway between the near and far clipping plane, .33 would be 1/3 (closest to the near clipping plane) .66 would be a depth 2/3 of the distance between the near and far clipping plane and so on... Understanding that fundamental was really key in making sense of both of my questions...

mordenkaim

Posts : 21
Join date : 2009-05-19

View user profile

Back to top Go down

Re: Physics-based picking

Post  Sponsored content


Sponsored content


Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum