API review

Present at review:

• Jon B.
• Jon S.
• Gil
• Sachin
• Joe
• Peter
• Kaijen
• Matei

Overview

General review for the manipulation pipeline

We will be reviewing the overall structure of the manipulation pipeline. There are 4 stacks:

• object_manipulation

• pr2_object_manipulation

• tabletop_object_perception

• pr2_tabletop_manipulation_apps

All of these stacks, and the packages therein, have been documented. You can browse them at your convenience, and please signal anything that looks out of place, whether in an API, package/stack structure, documentation, etc. Please put all comments on this review page.

API review for object manipulation

The ROS API for the object manipulation pipeline is defined in object_manipulation_msgs, but is mostly discussed in object_manipulator. There are two main categories:

• the external interface to the pipeline, discussed under Actions and Services Provided. This is the most important part, as it defines how the pipeline should be used.

• communication inside the pipeline, discussed under Actions and Services Required. This is probably less crucial right now, but could use a good discussion. Will probably be mostly relevant to folks looking not to use the pipeline wholesale, but to substitute some of their own components for the default ones.

API review for tabletop object detection

On the same occasion, we will review the API for the tabletop_object_perception stack. This stack is not a development goal in itself. It is only meant to allow users to use the manipulation pipeline until better options mature. As such, it is never meant to go beyond 0.2 status. It's ROS API is in two packages:

• tabletop_object_detector

• tabletop_collision_map_processing

Please put all comments on this review page.

Question / concerns / comments

Enter your thoughts on the API and any questions / concerns you have here. Please sign your name. Anything you want to address in the API review should be marked down here before the start of the meeting.

Gil

object_manipulator

• Where does the definition of what constitutes an arm and what the arms are named come from?
• What's the thinking behind the grasp definition? Why is there a grasp posture in terms of joint states, a pre-grasp in terms of joint states, and a grasp pose in terms of a pose? Why not a pose for pre-grasp? Is there an assumption of inverse IK?
• Why define the discretization outside the message and the number of steps in the message? Does this exclude tiny and giant hands?
• In the medium/long term I'm worried we're going to head more expressiveness in defining approaches, as this seems pretty focused on inverse IK.
• Same concerns about place - maybe we can discuss in terms of substituting CHOMP for inverse IK as a use case?

pr2_object_manipulation

• pr2_gripper_grasp_controller has bad link GraspHandPostureExecution. Seems like there's some stuff in here which we aren't actually API reviewing.

tabletop_object_perception

• I can't find where you mention the symmetric requirement in tabletop_object_detector. It seems like you might want to mention in the section on assumption, which already contains 3 even though the text says 2. It also better explains why you can search in XY instead of needing yaw as well.
• In the package summary for tabletop_collision_map_processing the 'etc.' seems like you got bored documenting. I'd just end the list if that's all you think is important.

Hai

• In general there seems to be quite a large number of parameters in the PickUpGoal and PlaceGoal messages and that many can be made optional.

• I see the grasp process as being composed of 3 separate steps: move from pre-grasp, grasp, and lift. In this API, those steps are exposed as a single service call with parameters for the 3 separate steps in one call making the parameter list rather large. Although, in my experience, you seldom move to the pre-grasp configuration without making a grasp. So probably breaking this call into two calls would make sense with a service that performs pre-grasp & grasp then a separate service for the lifting component. This way, it might be possible for the user to change lift to other commands, i.e. twist (for bottle caps), pull (for handles), or a grasp with the other arm (for bimanual tasks).

• It looks like parameters in PlaceGoal such as desired_approach_steps, min_approach_steps, use_reactive_execution, and use_reactive_lift can have reasonable default values that should "work" for normal hand sized objects. Having a comment in the msg definition that demarcate everything underneath it to be optional might be nice for first time users. Also, I'm not sure on why some of the parameters are in units of steps. Would it be ok to have them in cm or other friendly SI units?

• I didn't understand the implications for what happens if the various collision frame names are left empty.

• In placement, does the controller really need a z component from the PoseStamped message? It seems that this would be defined by the surface.

• This isn't related to the grasping pipeline but the behaviors in reactive grasping (reactive approach, compliant close, grasp adjustment) seems quite useful in general manipulation. In our system, I used a lot of the grasping functions on El-E for all the other tasks (light switch, drawer opening, door opening, lamp pulling, etc). Maybe it's worth exposing these behaviors through service calls too.

Sachin

• Overall structure
  • why is grasp_planner_database in the object_manipulation stack?
• Overall structure
  • why is grasp_planner_database in the object_manipulation stack?
• Grasp message
  • how do grasp_posture and grasp_pose relate to each other?
  • if the grasp_pose is relative to the object, where is the object frame defined for non-database objects - point clouds?

• PickUpGoal

  • the 1 cm steps seems to be hard-coded - can we make this a parameter too?
    • call it something like approach_discretization
  • I agree with Hai that decoupling the lift from the grasp action would make this more general
  • object_collision_name is a little confusing so probably just call it object_name
    • also do the same for support_collision_name - maybe call it support_surface_name

• PlaceGoal

  • the same thing for object_collision_name and support_collision_name
  • again the 1 cm discretization should be configurable
  • how does this deal with non-database objects - is the point cloud corresponding to the objects retrieved based on the name of the object? Is there a bounding box that is specified somewhere?
  • add information (in the message) on the range of values for place_padding
  • it would be good to have some default values for everything but we need to figure out a way to see when the defaults need to be imposed.

• GraspPlanning

  • don't need to call the resultant grasps "planned_grasps" - just call them "grasps"

• GraspHandPostureExecution

  • Does the GraspHandPostureGoal message have to be a separate message? Why not put it into the goal directly?

  • What does it mean for a grasp to be "active"?

• ReactiveGrasp.action

  • Does it need some information about the object being grasped? This could work both for known and unknown objects

Advait

• Although arm_navigation is external to the grasping pipeline, some pointers on how one could use different methods for going from the starting arm position -> pre-grasp position -> grasp position would be nice.

  • For example, I could imagine wanting to use more sensor feedback and strainght line paths instead of the RRT motion planner to move the gripper until it makes contact with the object or the table.
  • How could I fit a behavior like this into the grasping pipeline?
• If we want to stop the grasp pipeline execution at any point of time, is there a topic that we could publish to? Sort of like the robot monitoring its own progress and stopping in case of something going wrong.
• Object placement:
  • Its possible that if we want to place objects on cluttered environments, like piling up objects, or putting objects in a bin, we might again want to move the object until it is in contact with the environment and then let go.
  • How could such a method be used with the object grasping pipeline.

Jon (B)

• We had made some modifications to the tabletop_object_detector that made it discriminate between different plane orientations. We should probably move that in at some point.

Meeting agenda

Current release:

• yaml file with hand description should be documented (done)

  • in the Pickup and Place messages users must fill out the arm name (done)

  • this will be in the tutorials (done)

• explain grasp message better, maybe in tutorial (done)

• fix the approach specification (done)

  • in the message:

    • float desired_approach_distance (done)

    • float min_approach_distance (done)

  • as a parameter of the object_manipulator:

    • the resolution of the approach

• ditto for placing (done)

• explain fields of messages in tutorial, including behavior elicited by the default values (done)

• explain empty collision names in the tutorial (done)

• replace "where the object should be placed" with "the desired pose of the object once it's been placed" (done)

• explain the origin of the point cloud for unknown objects (done)

• change *_collision_name to collision_*_name for pickup and place (done)

  • also use support_surface (done)

• explain place padding in tutorial (done)

• change PlannedGrasps to "grasps" (done)

• move GraspPostureGoal in the body of the action (done)

• get rid of empties (done)

• change: (done)

  • GraspHandPostureQuery to GraspStatus (done)

  • rename is_active to is_hand_occupied (done)

• add the GraspableObject to the reactive grasp action (done)

• add a use_reactive_place flag (done)

• add normal filtering for the tabletop_object_detector

• add a general error flag to the grasp planner return code (done)

• revisit manipulation result message (done)

• add a service to the db wrapper for getting tags and description (done)

Future releases:

• enable use of different planners for both approach and move

• replace table rectangle with a convex hull polygon

• Kaijen will add actions for sub-components of reactive grasping

Conclusion

Package status change mark change manifest)

• Action items that need to be taken.

• Major issues that need to be resolved