Skip to content
Snippets Groups Projects

EVReflex

Dense time-to-impact estimation using events and depth.

Description

This has been accepted to the IROS 2021 conference. The paper (currently pending publication) can be found on ArXiV.

A brief introduction can be found in the following video.

Processing is offline and data must be run through two sub-networks sequentially,

  • Optical flow estimation
  • Time-to-impact (TTI) estimation

Installation

This repository is built with python 3 (>=3.7) on pytorch lightning.

  1. Clone this repository.

    git clone --recursive https://gitlab.eps.surrey.ac.uk/cw0071/EVReflex.git
    cd EVReflex
  2. Install the requirements.

    I recommend first using a miniconda environment and then following the most recent pytorch install instructions.

    # NOTE: Customise for your system/development environment/preferences
    conda install pytorch torchvision cudatoolkit=11.1 -c pytorch -c nvidia
    pip install -r requirements.txt
    pip install -e evreflex

Usage

Remember to activate your conda environment if you have one.

Train:

./train_evreflex.py TRAIN_DIR VAL_DIR OUTPUT_DIR

where TRAIN_DIR and VAL_DIR can be LMDB directories, or text files containing a list of LMDB directories.

OUTPUT_DIR is a tensorboard directory, so to view the output:

tensorboard --bind_all --load_fast=true --logdir OUTPUT_DIR

Dataset

Source files

Format Links Size
LMDBs (preview) download (5.1GB)
LMDBs (full) download (364GB)
Source ROSbags (preview) download (6.6GB)
Source ROSbags (full) download (455GB)

Command-line instructions

wget https://cvssp.org/data/evreflex/lmdb_full.tar
tar --one-top-level -xvf lmdb_full.tar
rm lmdb_full.tar

Using other datasets is possible, but you may need to adapt the data loader. Other datasets will need to include RGB/greyscale and optical flow, as well as an event stream and depth. You may also wish to convert it to LMDB.

Conversion from rosbag:

./evreflex/tools/rosbags_to_danger_lmdb.py PATH_TO_BAG

NOTE: The version of pickle used for the LMDB datasets affects how they are encoded/decoded. The python version in your ROS distribution (for converting rosbags to LMDBs) must be compatible with the python version you would run pytorch with. The downloadable LMDBs were created with python version 3.8.5 (ROS noetic).

Citation

Celyn Walters and Simon Hadfield. "EVReflex: Dense Time-to-Impact Prediction for Event-based Obstacle Avoidance". In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, Sept. 2021.

@inproceedings{Walters2021,
	author = {Walters, Celyn and Hadfield, Simon},
	title = {{EVReflex: Dense Time-to-Impact Prediction for Event-based Obstacle Avoidance}},
	booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
	publisher = {IEEE}
	year = {2021}
	arxivId = {2109.00405},
	url = {http://arxiv.org/abs/2109.00405},
}

Contact

Open an issue or contact me directly.