Mid-Level Fusion for Object Detection with Deep Neural Networks in Autonomous Driving Applications

Lehrstuhl für Fahrzeugtechnik
Masterarbeit /
experimentell / theoretisch /  

Successfully mastering the autonomous driving task depends highly on an accurate representation and understanding of the environment. To achieve such a detailed knowledge of the surrounding, current object detection algorithms use not just camera but also lidar or radar data. However, utilizing the advantages of different sensor modalities without the extensive use of resources is a challenging task, which should be tackled by the development of a mid-level fusion approach for camera, lidar and radar data.

The objective of this thesis is the development of a mid-level fusion approach to combine multiple different sensor modalities. The fusion network should improve the overall detection quality without the overhead of operating multiple independent pipelines for each sensor modality. The goal is the development of an improved object detection trough sensor fusion as well as the provision of a combined feature space for additional perception applications.

The first step of this project consists of a literature research on the current stat of the art in object detection and sensor fusion. In the second step, a pipeline should be implemented to feed the outputs of multiple different sensor modalities to the fusion module. In the next step, a mid-level fusion network should be developed which improves the detection quality and provides a combined feature space for additional perception applications. Finally, the results of the work should be compared to the current state of the art and validated on a real-world autonomous vehicle.


Work packages

  • Literature research on mid-level fusion
  • Fusion of different sensor modalities in a combined feature space
  • Development of a mid-level fusion network for object detection
  • Comparison of the results to the current state of the art
  • Validation and deduction of an outlook on future improvements


  • Programming experience in Python
  • Involved working attitude
  • Ideally experience with Docker
  • Ideally experience in machine learning


The thesis can be written in German or English language.

FTM Studienarbeit, FTM AV, FTM AV Perception, FTM Fent, FTM Informatik, FTM IVS
Möglicher Beginn
Felix Fent, M. Sc.
Raum: MW3508
Tel.: +49 89 289 15347