Note: This tutorial assumes that you have completed the previous tutorials: Using Catkin Workspaces.
(!) Please ask about problems and questions regarding this tutorial on answers.ros.org. Don't forget to include in your question the link to this page, the versions of your OS & ROS, and also add appropriate tags.

Overlaying with catkin workspaces

Description: Describes how to overlay a catkin workspace on to an existing system for development and testing.

Keywords: catkin workspace

Tutorial Level: INTERMEDIATE

  Show EOL distros: 

Contents

  1. Prerequisites
    1. Prepare Environment
  2. Overlaying with a catkin Workspace
    1. Create a catkin Workspace
    2. Adding Packages to Your catkin Workspace
    3. Building Your catkin Workspace
    4. Testing Out Your Overlaid Packages
  3. Chaining catkin workspaces
  4. Limitations of overlaying
    1. Underlayed build artifacts shine through
    2. Dependent packages in overlay are not rebuild

Overlaying refers to building and using a ROS package from source on top of an existing version of that same package. In this way your new or modified version of the package "overlays" the installed one.

Prerequisites

For this tutorial we assume that you have installed the appropriate version of ROS ($ROS_DISTRO) and are using Ubuntu.

Prepare Environment

First start by installing the ros-tutorials ROS-package from apt-get: 

  • $ sudo apt-get install ros-$ROS_DISTRO-ros-tutorials

This should install several packages containing the examples from the ROS tutorials. You can confirm this by checking the path: 

  • $ roscd rospy_tutorials
$ pwd

/opt/ros/$ROS_DISTRO/share/rospy_tutorials

Note: If you get 'roscd: command not found' then you have probably neglected to source you environment, do that with this command: 

  • $ source /opt/ros/$ROS_DISTRO/setup.bash

As you can see the roscd command takes you to the installation location or the package rospy_tutorials, where shared resources are installed. We will next overlay the ros-tutorials debian with the same packages but from from source in a workspace.

Overlaying with a catkin Workspace

Next we will need to setup a workspace in order to build and source this overlaid package.

Create a catkin Workspace

Start by creating a folder to hold the new workspace: 

  • $ mkdir -p ~/overlay_ws/src
$ cd ~/overlay_ws

Adding Packages to Your catkin Workspace

Now you can add a few ROS packages from source to your src folder: 

$ cd ~/overlay_ws/src
$ git clone https://github.com/ros/ros_tutorials.git

The second command clones the default branch of the "ros_tutorials" repository.

Note: It is recommended to use rosinstall_generator and vcstool for managing workspaces to ensure that the right repositories and branches are used as well as to make it easier to e.g. update multiple working directories.

Building Your catkin Workspace

Now that your workspace has the packages you wish to overlay you need to build it. Before building the workspace, however, you need to make sure that the environment is setup the way we want it. In this case we want to build the package(s) in this workspace "on top of" the things you have installed from debs, you can ensure this by first sourcing the deb's environment setup file: 

  • $ source /opt/ros/$ROS_DISTRO/setup.bash

Now you can build our workspace by invoking catkin_make from the root of our catkin workspace: 

  • $ cd ~/overlay_ws/
$ catkin_make

The above command should have built your packages in this workspace into the devel space of this workspace (located at ~/overlay_ws/devel). Now once you source that devel space you will have overlaid the packages in this workspace on top of the packages installed in /opt/ros/$ROS_DISTRO, lets do that now: 

  • $ source ~/overlay_ws/devel/setup.bash

Testing Out Your Overlaid Packages

Now that the example package, rospy_tutorials, has been overlaid lets try our test again: 

  • $ roscd rospy_tutorials
$ pwd

~/overlay_ws/src/ros_tutorials/rospy_tutorials

As you can see, roscd does not take you to /opt/ros/$ROS_DISTRO/share/rospy_tutorials any more, even though the rospy_tutorials package still exists in /opt/ros/$ROS_DISTRO, but instead takes you to the sources in your overlay workspace.

Chaining catkin workspaces

It is possible to use several catkin workspaces at the same time, and thus to separate the build process of one group of catkin packages from another group of catkin packages. As an example you might want to install ROS core packages from source as one set of packages (an "underlay"), and packages specific to your lab as another set of packages.

Other than that, novice users should have little reason to use this feature, and work with just one workspace to avoid confusion. Having a chain of workspaces also forces the user to call make in workspaces in the right order, and can cause failures if overlaying packages contain packages on which packages in overlaid workspaces depend.

Still, for this tutorial we will chain a second workspace to our first one to demonstrate how it works.

So create a second workspace just like you created your first (called overlay_ws_overlay and overlay_ws, respectively): 

$ mkdir -p ~/overlay_ws_overlay/src
$ cd ~/overlay_ws_overlay/src
$ catkin_init_workspace
$ cd ~/overlay_ws_overlay
## THIS IS THE CRUCIAL PART FOR OVERLAYING
$ source ~/overlay_ws/devel/setup.bash
$ catkin_make

Note how the output tells you the overlay chain: 

...
-- This workspace overlays: /home/<user>/overlay_ws/devel;/opt/ros/$ROS_DISTRO
...

Also note the overlay requires that the workspace we want to depend on is active when we configure. So if you delete your build folder and configure again, you will have to remember to source the respective setup.sh again to create the chain of workspaces.

To test the overlay, open a new shell, then source the new setup.sh in overlay_ws_overlay: 

# open new shell
$ source ~/overlay_ws_overlay/devel/setup.bash
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws/src/ros_tutorials/rospy_tutorials

We can also create another copy of rospy_tutorials in our second overlay workspace: 

$ cd ~/overlay_ws_overlay/src
$ git clone https://github.com/ros/ros_tutorials.git

Now we can check with roscd whether the overlay works: 

$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws/src/ros_tutorials/rospy_tutorials

As you can see, it did NOT work. In order for ros tools to find a new package, we need to have configured it at least once. Also roscd works on a cache of locations which we need to update: 

$ cd ~/overlay_ws_overlay
$ catkin_make
# this updates the cache for roscd
$ rospack profile
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws_overlay/src/ros_tutorials/rospy_tutorials

Now you should see roscd taking you to the uppermost overlay overlay_ws_overlay.

Just as a recap, open a new shell(terminal window): 

$ roscd rospy_tutorials
roscd: command not found

$ source /opt/ros/$ROS_DISTRO/setup.bash 
$ roscd rospy_tutorials
$ pwd
/opt/ros/$ROS_DISTRO/share/rospy_tutorials

$ source ~/overlay_ws/devel/setup.bash
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws/src/ros_tutorials/rospy_tutorials

$ source ~/overlay_ws_overlay/devel/setup.bash 
$ roscd rospy_tutorials
$ pwd
/home/<user>/overlay_ws_overlay/src/ros_tutorials/rospy_tutorials$

Limitations of overlaying

When you overlay packages, you need to be aware of certain limitations. Assume we have this workspace layout with 2 workspaces: 

overlay_ws/
  src/
    package_a
    package_b  # depends on package_a
  devel/
    ...
overlay_ws_overlay/
  src/
    package_a
  devel/
    ...

Assume overlay_ws_overlay overlays overlay_ws. As you can see the package package_a exists in both workspaces, this one is overlaying the other package_a.

Underlayed build artifacts shine through

With catkin, overlay only means that where similarly named build artifacts (e.g. c++ header files) exist in overlay_ws_overlay and overlay_ws, those in overlay_ws_overlay will be used. Differently named artifacts, or artifacts deleted in the overlaying packages, can still visible to dependent packages. This depends on whether the dependent package depends on one package in the underlay workspace that has not been overlayed. The same goes for the overlaying package itself, i.e. cpp files may compile using headers of the overlayed package under the circumstances described above.

So as a consequence, if overlay_ws/src/package_a defines a header file that overlay_ws_overlay/src/package_a does not define, this header file will still be usable by any package in overlay_ws_overlay. It is not hidden by the overlay. The only way to prevent this for sure is to not use overlaying. Else users must be careful not to remove build artifacts in overlaying packages.

Dependent packages in overlay are not rebuild

The other limitation is that package_b, which depends on package_a, will with this setup never be rebuild, meaning it will not use any build artifacts of overlay_ws_overlay/src/package_a. This can cause certain kinds of failures.

To be on the safe side, users need to make sure that all such packages that depend on an overlaying package are also present as source in the overlaying workspace. This can also be achieved in the example above by creating a symbolic link from overlay_ws_overlay/src/ to overlay_ws/src/package_b.

Next: Using ROSBUILD with Catkin

Wiki: catkin/Tutorials/workspace_overlaying (last edited 2020-06-11 19:33:32 by DirkThomas)

Except where otherwise noted, the ROS wiki is licensed under the
Creative Commons Attribution 3.0