Evaluation
QExE — Quality and Experience Evaluation Tool
A software toolkit for conducting subjective evaluations of audiovisual quality and experience in interactive VR environments. Built on a 3D game engine with control software, QExE integrates direct and indirect scaling quality evaluation methods, additional questionnaires, behavioral and interactivity data collection, and support for multiple audio rendering plug-ins (including Ambisonics and object-based audio VSTs). Test item creation and data logging are automated and saved to subject-specific sub-directories.
The tool is available here.
- T. Robotham, O.S. Rummukainen, D. Rebmann, A. Raake, E.A.P. Habets, QExE: A Quality and Experience Evaluation Tool for Audiovisual VR Perception, Behavior, and Cognition Research, Frontiers in Virtual Reality, 2026.
PESQ — Perceptual Evaluation of Speech Quality (pesqc2)
A Python wrapper for the ITU-T P.862 PESQ algorithm that includes the P.862 Corrigendum 2 (03/2018) correction, which addresses an under-prediction of subjective scores (by approximately 0.8 MOS on average). Supports both wideband (16 kHz) and narrowband (8 kHz) modes, with batch processing via multiprocessing.
The Python implementation is available here and can be installed via pip install pesqc2.
- T. Robotham et al., Navigating PESQ: Up-to-Date Versions and Open Implementations, arXiv:2505.19760.
Signal-Based Direct-to-Reverberation Ratio (DRR)
Estimates the direct-to-reverberation ratio (DRR) from input/output signals alone, without access to the room impulse response as described in [3]. Particularly useful for evaluating nonlinear or time-varying dereverberation algorithms.
A MATLAB implementation built upon VoiceBox can be downloaded here.
- P.A. Naylor, N.D. Gaubitch and E.A.P. Habets, Signal-based performance evaluation of dereverberation algorithms, Journal of Electrical and Computer Engineering, 2010.