Overview

This package is a ROS wrapper for Alvar, an open source AR tag tracking library.

ar_track_alvar has 4 main functionalities:

1. Generating AR tags of varying size, resolution, and data/ID encoding
2. Identifying and tracking the pose of individual AR tags, optionally integrating kinect depth data (when a kinect is available) for better pose estimates.
3. Identifying and tracking the pose of "bundles" consisting of multiple tags. This allows for more stable pose estimates, robustness to occlusions, and tracking of multi-sided objects.
4. Using camera images to automatically calculate spatial relationships between tags in a bundle, so that the user does not have to manually measure and enter tag locations in an XML file to use the bundle functionality (**NOT CURRENTLY WORKING -- SEE BELOW).

Alvar is significantly newer and more advanced than the ARToolkit, which has been the basis for several other ROS AR tag packages. Alvar features adaptive thresholding to handle a variety of lighting conditions, optical flow based tracking for more stable pose estimation, and an improved tag identification method that does not significantly slow down as the number of tags increases.

Installation

$ sudo apt-get install ros-indigo-ar-track-alvar

Generating AR tags

Two pdf files are in the markers directory containing tags 0-8 and 9-17, respectively. The markers are 4.5 cm (although when printed and measured, came out to 4.4 cm for me). If you want to generate your own markers with different ID numbers, border widths, or sizes, run:

   1 rosrun ar_track_alvar createMarker

and instructions will appear describing the various options.

You can also download the aforementioned marker files here: ar_track_alvar/markers0to8.png ar_track_alvar/markers9to17.png

Usecase per number of tags

Detecting individual tags

The first use case for this package is to identify and track the poses of (possibly) multiple AR tags that are each considered individually. The node individualMarkers takes the following command line arguments:

1. marker_size (double) -- The width in centimeters of one side of the black square marker border

2. max_new_marker_error (double) -- A threshold determining when new markers can be detected under uncertainty

3. max_track_error (double) -- A threshold determining how much tracking error can be observed before an tag is considered to have disappeared

4. camera_image (string) -- The name of the topic that provides camera frames for detecting the AR tags. This can be mono or color, but should be an UNrectified image, since rectification takes place in this package

5. camera_info (string) -- The name of the topic that provides the camera calibration parameters so that the image can be rectified

6. output_frame (string) -- The name of the frame that the published Cartesian locations of the AR tags will be relative to

individualMarkers assumes that a kinect being used as the camera, so that depth data can be integrated for better pose estimates. If you are not using a kinect or do not desire to use depth data improvements, use individualMarkersNoKinect instead.

When using this package with the PR2, you can simply type:

   1 roslaunch ar_track_alvar pr2_indiv.launch

Concrete example can be found on the development repo (Indigo onward, Kinetic onward) can be used as an example of how to set these parameters in a reasonable way and start the node.

Otherwise, this file can be used as an example of how to set these parameters in a reasonable way and start the node.

Detecting multi-tag bundles

Sometimes it is advantageous to treat "bundles" of multiple tags as a single unit. For example, this can allow for the estimation of the pose of a many-sided object, even when some of the tags cannot be seen. It can also lead to more stable pose estimates and provide robustness to occlusion. A tag bundle is defined by an XML file that lists a set of tag IDs and their positions relative to a "master" tag. The master tag always comes first in the XML file and defines a coordinate system for the rest of the tags. IMPORTANT: this coordinate system is different from the standard system used in ROS! In this system, when facing the tag, positive-z comes out of the front of the tag toward the viewer, positive-x is to the right, and positive-y is up.

To create a bundle, first choose which tag you want to be the master tag. Treat the center of the master tag as (0,0,0). Then, after placing the rest of the tags, measure the x, y, and z coordinate for each of the 4 corners of all of the tags, relative to the master tag origin. Enter these measurements for each tag into the XML file starting with the lower left corner and progressing counter-clockwise around the tag. An example XML file showing this for a tagged table leg can be found in the ./bundles directory (Indigo onward, Kinetic onward).

Multiple bundles can be detected at once. There should be one file for each bundle that is to be detected. The node findMarkerBundles takes the following command line parameters:

1. marker_size (double) -- The width in centimeters of one side of the black square that makes up a marker

2. max_new_marker_error (double) -- A threshold determining when new markers can be detected under uncertainty

3. max_track_error (double) -- A threshold determining how much tracking error can be observed before an tag is considered to have disappeared

4. camera_image (string) -- The name of the topic that provides camera frames for detecting the AR tags. This can be mono or color, but should be an UNrectified image, since rectification takes place in this package

5. camera_info (string) -- The name of the topic that provides the camera calibration parameters so that the image can be rectified

6. output_frame (string) -- The name of the frame that the published Cartesian locations of the AR tags will be relative to

7. bundle_files (multiple strings) -- A list of XML file names, one for each bundle you wish to detect.

findMarkerBundles assumes that a kinect being used as the camera, so that depth data can be integrated for better pose estimates. If you are not using a kinect or do not desire to use depth data improvements, use findMarkerBundlesNoKinect instead.

When using this package with the PR2, you can simply type:

   1 roslaunch ar_track_alvar pr2_bundle.launch

Otherwise, this file (Indigo onward, Kinetic onward) can be used as an example of how to set these parameters in a reasonable way and start the node.

• • A video (by Ishijima Electronics (in Japanese) shows multiple tags on a piece of paper being detected as a bundle.

ROS API

Subscribed topic (common among usecases)

(e.g.) /kinect_head/rgb/camera_info (sensor_msgs/CameraInfo)

• Camera needs calibrated before this topic to become available.

• Image to be analyzed for markers.

API for individual markers

Published Topics

• This is an rviz message that when subscribed to (as a Marker in rviz), will display a colored square block at the location of each identified AR tag, and will also overlay these blocks in a camera image. Currently, it is set to display a unique color for markers 0-5 and a uniform color for all others.

• This is a list of the poses of all the observed AR tags, with respect to the output frame

Provided tf Transforms

• Provides a transform from the camera frame to each AR tag frame, named ar_marker_x, where x is the ID number of the tag.

API for detecting multiple markers

Published Topics

• This is an rviz message that when subscribed to (as a Marker in rviz), will display a red square block at the location of each identified "master" AR tag (one per bundle), a green square block at the location of each other tag, and will also overlay these blocks in a camera image.

• This is a list of the poses of all the observed "master" AR tags (one per bundle), with respect to the output frame

Provided tf Transforms

• Provides a transform from the camera frame to each "master" AR tag frame, named ar_marker_x, where x is the ID number of the tag.

Automatic XML bundle file generation

Rather than having to measure tag placements and construct an XML file by hand for each bundle, Alvar has a feature that allows this to be done automatically from multiple images in which 2 or more tags are visible. Running ./bin/trainMarkerBundle (see pr2_train.launch for an example) provides a keyboard interface for captuing images and optimizing the bundle structure. However, I was not able to get it working reliably. It would be great if someone took this up as a small project. Some tips from another group that uses Alvar can be found here.