International Audio Laboratories Erlangen

Reconstruction of Motion Sequences from Low-Dimensional Sensor and Control Data (REKOBA)

Logo_DFG Teaser_REKOBA Logo_MPII Logo_UniSaarland Logo_UniBonn

In the REKOBA project, we developed real-time methods for data-driven reconstruction of motion sequences from low-dimensional sensor data. The project was funded by the German Research Foundation. On this website, we summarize the project's main outcomes and provide links to project-related publications.

Project Description

Reconstruction of Motion Sequences from Low-Dimensional Sensor and Control Data

Teaser_REKOBA_example2

The acquisition, representation, and characterization of complex motion sequences play a fundamental role in areas such as medicine, sports science, and computer graphics. Previously available sensors for motion detection (especially optical systems) cause high acquisition and operating costs and require much time for measurement preparations, which excludes the use of such technologies for mass applications. In contrast, novel sensor systems (e.g., based on acceleration and inertial sensors) are cost-effective alternatives for recording movement data due to their mass production. Furthermore, these sensors allow relatively easy handling, especially if only a few sensors are needed, such as in shoes, belt loops, or bracelets to be attached. However, the resulting sensor data is often noisy and contains errors. In order to make such systems accessible for mass use in areas such as rehabilitation and sports, one requires robust methods for reconstructing motion sequences from low-dimensional and noisy sensor data. In the REKOBA project, we systematically researched methods and implemented real-time systems for the data-driven reconstruction of motion sequences from low-dimensional sensor data. Among other things, the following fundamental questions were systematically addressed: What and how much sensor data (e.g., measuring acceleration and changes in orientation of body parts) is required to reconstruct motions? Which partial data are necessary for the characterization of certain motion classes? How exactly can certain motion sequences be reconstructed with a given sensor system?

Projektbeschreibung

Rekonstruktion von Bewegungsabläufen aus niedrigdimensionalen Sensor- und Kontrolldaten

Teaser_REKOBA_example2

Die Erfassung, Repräsentation und Charakterisierung von komplexen Bewegungsabläufen spielt eine grundlegende Rolle in Gebieten wie der Medizin, den Sportwissenschaften und der Computergraphik. Bislang verfügbare Sensorik zur Bewegungserfassung (insbesondere optische Systeme) verursachen hohe Anschaffungs- und Betriebskosten und erfordern einen hohen zeitlichen Aufwand für die Messvorbereitungen, was den Einsatz solcher Techniken für Massenanwendungen ausschließt. Demgegenüber könnten neuartige Sensorsysteme (z. B. basierend auf Beschleunigungsund Drehsensoren) aufgrund ihrer Massenproduktion äußerst kostengünstige Alternativen zur Erfassung von Bewegungsdaten darstellen, die zudem eine relativ einfach Handhabung erlauben, insbesondere wenn nur wenige Sensoren gebraucht werden, die etwa in Schuhen, Gürtelschlaufen oder Armbändern angebracht werden. Allerdings sind die resultierenden Sensordaten oft verrauscht und fehlerbehaftet. Um solche Systeme für den Masseneinsatz in Bereichen wie Rehabilitation und Breitensport zugänglich zu machen, werden daher robuste Verfahren zur Rekonstruktion von Bewegungsabläufen aus niedrigdimensionalen und verrauschten Sensordaten benötigt. Ziel des REKOBA-Projekts war die systematische Erforschung von Methoden und die Implementation eines Echtzeitsystems zur datengestützten Rekonstruktion von Bewegungsabläufen aus niedrigdimensionalen Sensordaten. Unter anderem wurden folgende grundlegende Fragestellungen systematisch angegangen: Welche und wie viele Sensordaten, die etwa Beschleunigungen und Orientierungsänderungen von Körperteilen messen, sind zur Rekonstruktion von Bewegungen nötig? Welche partiellen Daten sind für Charakterisierungen gewisser Bewegungsklassen notwendig? Wie genau können gewisse Bewegungsabläufe bei vorgegebener Sensorik rekonstruiert werden?

Projected-Related Publications

The following publications reflect the main scientific contributions of the work carried out in the REKOBA project.

  1. Jan Baumann, Björn Krüger, Arno Zinke, and Andreas Weber
    Data-Driven Completion of Motion Capture Data
    In Workshop on Virtual Reality Interaction and Physical Simulation (VRIPHYS), 2011. PDF DOI 
    @inproceedings{BaumannKZW11_MotionCompletation_VRIPHYS,
author     = {Jan Baumann and Bj{\"{o}}rn Kr{\"{u}}ger and Arno Zinke and Andreas Weber},
title      = {Data-Driven Completion of Motion Capture Data},
booktitle  = {Workshop on Virtual Reality Interaction and Physical Simulation ({VRIPHYS})},
year       = {2011},
address    = {Lyon, France},
publisher    = {Eurographics Association},
doi        = {10.2312/PE/vriphys/vriphys11/111-118},
url-pdf   = {2011_BaumannKZW_MotionCompletation_VRIPHYS.pdf},
}
  2. Thomas Helten, Andreas Baak, Meinard Müller, and Christian Theobalt
    Real-time Body Tracking with One Depth Camera and Inertial Sensors
    In Proceedings of the International Conference on Computer Vision (ICCV), 2013. PDF 
    @inproceedings{HeltenBMT13_TrackingDepthInertial_ICCV,
author       = {Thomas Helten and Andreas Baak and Meinard M{\"u}ller and Christian Theobalt},
title        = {Real-time Body Tracking with One Depth Camera and Inertial Sensors},
booktitle    = {Proceedings of the International Conference on Computer Vision ({ICCV})},
year         = {2013},
address      = {Sydney, Australia},
url-pdf      = {2013_HeltenBMT_TrackingDepthInertial_ICCV.pdf}
}
  3. Thomas Helten, Andreas Baak, Meinard Müller, and Christian Theobalt
    Full-Body Human Motion Capture from Monocular Depth Images
    In Marcin Grzegorzek and Christian Theobalt and Reinhard Koch and Andreas Kolb (ed.): LNCS 8200, Time-of-Flight Imaging Algorithms, Sensors and Applications, Springer-Verlag Berlin Heidelberg: 188–206, 2013. PDF 
    @incollection{HeltenBMT13_FullbodyTrackingFromDepth_LNCS,
author      = {Thomas Helten and Andreas Baak and Meinard M{\"u}ller and Christian Theobalt},
title       = {Full-Body Human Motion Capture from Monocular Depth Images},
editor      = {Marcin Grzegorzek and Christian Theobalt and Reinhard Koch and Andreas Kolb},
booktitle   = {LNCS 8200, Time-of-Flight Imaging Algorithms, Sensors and Applications},
publisher   = {Springer-Verlag Berlin Heidelberg},
pages       = {188--206},
year        = {2013},
url-pdf   = {2013_HeltenBMT_FullbodyTrackingFromDepth_LNCS.pdf}
}
  4. Thomas Helten, Heike Brock, Meinard Müller, and Hans-Peter Seidel
    Classification of trampoline jumps using inertial sensors
    Sports Engineering, 14: 155–164, 2011. PDF 
    @article {HeltenBMS11_ClassificationTrampolineJumps_SE,
author = {Thomas Helten and Heike Brock and Meinard M{\"u}ller and Hans-Peter Seidel},
affiliation = {MPI Informatik and Saarland University, Campus E1 4, 66123 Saarbr{\"u}cken, Germany},
title = {Classification of trampoline jumps using inertial sensors},
journal = {Sports Engineering},
publisher = {Springer London},
issn = {1369-7072},
pages = {155--164},
volume = {14},
issue = {2},
year = {2011},
url-pdf   = {2011_HeltenBMS_ClassificationTrampolineJumps_SE.pdf}
}
  5. Thomas Helten, Meinard Müller, Jochen Tautges, Andreas Weber, and Hans-Peter Seidel
    Towards Cross-modal Comparison of Human Motion Data
    In Proceedings of the Annual Symposium of the German Association for Pattern Recognition (DAGM): 61–70, 2011. PDF 
    @inproceedings{HeltenMTWS11_CrossModalMotion_DAGM,
author    = {Thomas Helten and Meinard M{\"u}ller and Jochen Tautges and Andreas Weber and Hans-Peter Seidel},
title     = {Towards Cross-modal Comparison of Human Motion Data},
booktitle = {Proceedings of the Annual Symposium of the German Association for Pattern Recognition ({DAGM})},
address = {Frankfurt, Germany},
publisher = {Springer},
series    = {Lecture Notes in Computer Science},
volume    = {6835},
year      = {2011},
pages     = {61--70},
url-pdf   = {2011_HeltenMTWS_TowardsCross-modalComparison_DAGM.pdf},
}
  6. Björn Krüger, Jochen Tautges, Andreas Weber, and Arno Zinke
    Fast Local and Global Similarity Searches in Large Motion Capture Databases
    In Symposium on Computer Animation (SCA): 1–10, 2010. PDF 
    @inproceedings{KrugerTWZ10_SimSearchMocap_SCA,
author       = {Bj{\"{o}}rn Kr{\"{u}}ger and Jochen Tautges and Andreas Weber and Arno Zinke},
title        = {Fast Local and Global Similarity Searches in Large Motion Capture Databases},
booktitle    = {Symposium on Computer Animation ({SCA})},
pages        = {1--10},
publisher    = {Eurographics Association},
year         = {2010},
url-pdf   = {2010_KrugerTWZ_SimSearchMocap_SCA.pdf}
}
  7. Gerard Pons-Moll, Andreas Baak, Thomas Helten, Meinard Müller, Hans-Peter Seidel, and Bodo Rosenhahn
    Multisensor-Fusion for 3D Full-Body Human Motion Capture
    In IEEE Conference on Computer Vision and Pattern Recognition (CVPR): 663–670, 2010. PDF 
    @inproceedings{PonsBaHeMuSeRo10_MultisensorFusion_CVPR,
author     = {Gerard Pons-Moll and Andreas Baak and Thomas Helten and Meinard M{\"u}ller and Hans-Peter Seidel and Bodo Rosenhahn},
title      = {Multisensor-Fusion for 3D Full-Body Human Motion Capture},
booktitle  = {IEEE Conference on Computer Vision and Pattern Recognition ({CVPR})},
year       = {2010},
address    = {San Francisco, California, USA},
month      = jun,
pages      = {663--670 },
url-pdf   = {2010_PonsBaakHeltenMuellerSeidelRosenhahn_MultisensorFusionMocap_CVPR.pdf}
}
  8. Jochen Tautges, Arno Zinke, Björn Krüger, Jan Baumann, Andreas Weber, Thomas Helten, Meinard Müller, Hand-Peter Seidel, and Bernd Eberhardt
    Motion reconstruction using sparse accelerometer data
    ACM Transactions on Graphics, 30(3), 2011. PDF 
    @article{TautgesZKBWHMSE11_MotionReconstruction_TOG,
author    = {Jochen Tautges and Arno Zinke and Bj{\"o}rn Kr{\"u}ger and Jan Baumann and Andreas Weber and Thomas Helten and Meinard M{\"u}ller and Hand-Peter Seidel and Bernd Eberhardt},
title     = {Motion reconstruction using sparse accelerometer data},
journal   = {ACM Transactions on Graphics},
volume    = {30},
number    = {3},
year      = {2011},
url-pdf   = {2010_TautgesEtAl_MotionReconstruction_ACM-TOG.pdf},
}

Projected-Related Ph.D. Theses

  1. Jochen Tautges
    Reconstruction of Human Motions Based on Low-Dimensional Control Signals
    PhD Thesis, Rheinische Friedrich-Wilhelms-Universität Bonn, 2012. PDF Details 
    @phdthesis{Tautges12_MotionReconstruction_PhD,
author      = {Jochen Tautges},
year        = {2012},
title       = {Reconstruction of Human Motions Based on Low-Dimensional Control Signals},
school      = {Rheinische Friedrich-Wilhelms-Universit{\"a}t Bonn},
url-details = {https://bonndoc.ulb.uni-bonn.de/xmlui/handle/20.500.11811/5362},
url-pdf = {2012_Tautges_MotionReconstruction_PhD-Thesis.pdf}
}
  2. Thomas Helten
    Processing and Tracking Human Motions Using Optical, Inertial, and Depth Sensors
    PhD Thesis, Universität des Saarlandes, 2013. PDF Details 
    @phdthesis{Helten13_HumanTracking_PhD,
author      = {Thomas Helten},
year        = {2013},
title       = {Processing and Tracking Human Motions Using Optical, Inertial, and Depth Sensors},
school      = {Universität des Saarlandes},
url-details = {https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/26607},
url-pdf = {2013_HeltenThomas_HumanMotionProcessing_PhD-Thesis.pdf}
}