A Real-Time Beat Tracking System with Zero Latency and Enhanced Controllability
Peter Meier, Ching-Yu Chiu, Meinard Müller
This website is related to the following publication:
- Peter Meier, Ching-Yu Chiu, and Meinard Müller
A Real-Time Beat Tracking System with Zero Latency and Enhanced Controllability
Transactions of the International Society for Music Information Retrieval (TISMIR), 7(1): 213–227, 2024. Demo DOI@article{MeierCM24_RealTimePLP_TISMIR, author = {Peter Meier and Ching-Yu Chiu and Meinard M{\"u}ller}, title = {{A} Real-Time Beat Tracking System with Zero Latency and Enhanced Controllability}, journal = {Transactions of the International Society for Music Information Retrieval ({TISMIR})}, year = {2024}, volume = {7}, number = {1}, pages = {213--227}, doi = {10.5334/tismir.189}, url-demo = {https://audiolabs-erlangen.de/resources/MIR/2024-TISMIR-RealTimePLP} }
Abstract
Identifying beat positions in music recordings, a central task in Music Information Retrieval (MIR), is commonly referred to as beat tracking. Typically, this involves computing an activation function to reveal beat information and then conducting post-processing to derive final beat positions. Existing methods often operate offline, requiring access to the entire music track for processing. In this article, we introduce a real-time beat tracking system based on the predominant local pulse (PLP) concept, originally designed for offline use. Our main contribution is the successful transformation of the PLP-based algorithm into a real-time procedure. Unlike traditional offline methods providing static beat positions, our real-time approach dynamically captures changes in local pulse characteristics with each frame of an audio stream. This yields additional insights, including beat context, beat stability, and beat lookahead for predicting beats in advance. In this way, our system not only demonstrates high controllability for real-time applications but also can operate at zero latency. Additionally, we present experiments comparing our real-time beat tracking system with other models and evaluating the accuracy of our lookahead feature. Finally, we showcase two real-world applications for interactive music making and educational music gaming that creatively leverage our system's output. In summary, our real-time beat tracking system offers a lightweight algorithm that is particularly well-suited for interactive music software development.
Audio Examples for Real-Time PLP Control Signals
In this example, we use a real-time PLP system to modulate the cutoff frequency of a low-pass filter, automatically shaping the sound of instruments such as bass or piano in a rhythmically interesting way. In another example, we utilize our system to modulate the position of a wavetable synthesizer in real-time.
Real-Time PLP Animation with Control Signals
In this example, we see the real-time PLP system in action, which is used to extract control signals for adjusting parameters in a Digital Audio Workstation. The audio excerpt is featuring: (a) Audio input and control signals, (b) beta-Conf, (c) gamma-Conf, (d) alpha-LFO, and (e) gamma-LFO. The black cursor indicates the current time position. The right side of the figure displays the inner state of the PLP buffer at that time position.
Acknowledgments
This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Grant No. 500643750 (MU 2686/15-1) and the National Science and Technology Council (NSTC), Taiwan, under Grant No. NSTC 112-2917-I-564-011. The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) and Fraunhofer Institute for Integrated Circuits IIS.