In a previous post we learned what rosbags are and how to use them to playback KITTI data. In this prequel-post we will show how to create the rosbag. The source code for the python module described is available here.

Installing the Python module

Grab the source code by cloning the repository, enter the source directory, then build and install the package. If all goes well and you have installed the KITTI dataset you are set to make your first conversion.

# clone the module source
git clone https://github.com/apresland/kitti-rosbag-utils

# enter source dir
cd kitti-rosbag-utils

# install the module
python3 -m build
python3 -m pip install

Using the Python module

Basic usage of the module involves choosing the data type from raw or odometry and setting the KITTI basedir and choosing the combination of KITTI datetime and drive that you want to convert to a rosbag.

# {basedir, datetime, drive} come from the KITTI dataset
python3 -m kitti_rosbag_utils -b {basedir} -t {raw} -dt {datetime}  -dr {drive}

For example, say we want to create a bag from the KITTI raw data labelled 2011_09_30_drive_0027 which we installed at /data/kitti/raw on the local filesystem. The using the following command will produce the rosbag with filename kitti_raw_2011_09_30_0027.bag

python3 -m kitti_rosbag_utils -b /data/kitti/raw -t raw -dt 2011_09_30  -dr 0027

How it works

The conversion process is simple and has these basic steps:

A lot of what we need is already available as python packages and we just need to write some glue and a runner to hold them together. The packages we rely on to achieve this are pykitti and rosbag.

The code is structured into data objects (RawData, OdometryData), factories to generate ROS messages (MonoImageMessages, PointCloudMessages) and writers to insert messages into the target bag (RawWriter, OdometryWriter). Let’s have a look at some of the code for handling raw data and leave the odometry data which follows the same general pattern.

RawData objects

Objects of type RawData provide an abstraction of the KITTI dataset that hides the details of the file structure and removes the necessity for knowledge of the data format defined by the KITTI researchers.


    # setup data path parameters
    kitti = pykitti.raw(
        basedir, date, drive)

    # check valid data path
    if not os.path.exists(kitti.data_path):
        print('Path {} does not exists. Exiting.'.format(kitti.data_path))
        sys.exit(1)

    # check not empty data file
    if len(kitti.timestamps) == 0:
        print('Dataset is empty? Exiting.')
        sys.exit(1)

    ## read the calibration
    kittti_calibration = pykitti.utils.read_calib_file(
        os.path.join(kitti.calib_path, 'calib_cam_to_cam.txt'))

    # create the data object
    self.data = RawData()
    self.data.path = kitti.data_path
    self.data.calibration = kittti_calibration
    self.data.timestamps = kitti.timestamps
    self.data.start = (
        datetime.utcnow() - datetime(1970, 1, 1)).total_seconds()

ImageMessage objects

Objects of type MonoImageMessage use OpenCV to process image data and create CameraInfo and Image objects belonging to sensor_msgs namespace. The messages are published as pairs sharing the same unique timestamp.

CameraInfo messages contain the camera calibration information and are populated using the information obtained from the KITTI calibration files.

        # camera calibration info
        self._info_message = CameraInfo()
        self._info_message.header.frame_id = 'image_' + camera_id
        self._info_message.width, self._info_message.height = tuple(self.calibration['S_rect_{}'.format(camera_id)].tolist())
        self._info_message.distortion_model = 'plumb_bob'
        self._info_message.K = self.calibration['K_{}'.format(camera_id)]
        self._info_message.R = self.calibration['R_rect_{}'.format(camera_id)]
        self._info_message.D = self.calibration['D_{}'.format(camera_id)]
        self._info_message.P = self.calibration['P_rect_{}'.format(camera_id)]

RawWriter objects

Objects of type RawWriter access data using instances of RawData, convert data content to ROS messages and then write tem to the target bag file. The instance of RawData is created internally based on the supplied arguments.

    # create a rosbag for writing
    target_bag = rosbag.Bag(
        "kitti_raw_{}_{}.bag".format(args.date,args.drive), 'w',
        compression=rosbag.Compression.NONE)

    # create the write object
    writer = RawWriter()
    writer.init(args.basedir, args.date, args.drive)
    writer.to_rosbag(target_bag)