The Musical Gestures Toolbox for Python (musicalgestures) is a collection of tools for visualising and analysing motion in video recordings, along with the accompanying sound. It was developed for research on music-related body motion, but it works on any video or audio file.
pip install musicalgesturesYou also need FFmpeg on your system; everything else installs automatically. The installation guide covers optional extras such as posture estimation.
import musicalgestures as mg
v = mg.MgVideo(mg.examples.dance) # or your own file: mg.MgVideo('dance.mp4')
v.motiongrams().show()This draws motiongrams: images that trace where motion happens in the frame over time, like a spectrogram for the body. Analysis methods return result objects, and .show() displays them.
You can also try the toolbox in the browser, with no installation:
- Documentation site — installation, user guide, and API reference
- Wiki — examples and discussion of the methods
- Contributing — how to report issues and submit changes
Parts of the toolbox that work on their own, without the rest of the pipeline:
- the FFmpeg convenience bindings in
musicalgestures._utils, for probing, converting, trimming and concatenating any media file MgVideoand the result objects, which load a file once and hand every analysis the same frames, with.show()on the result- the motiongram and videogram renderers, which draw from any frame source
- the pose backends (MediaPipe, RTMPose, YOLO) behind one interface, so a study can switch estimator without changing its analysis
- audio and video alignment (
xcorr_lag, sync marks) and the timecode helpers, which put two recordings on one clock - importers for ELAN annotations and Pupil Labs eye-tracking exports, which turn events into series on the video's clock
musicalgestures is file-based and non-real-time. It does not track expressive qualities as a stream (PyEyesWeb does), does not compute joint angles, body-segment models or 3D animation (Kinetics Toolkit does, with C3D and events), and does not lay data out as BIDS. Motion series from markers or sensors go to micromotion, sound of places to ambiscape, and collections of music to musiscape, as the table below says.
As the toolbox has grown, we split it into four related packages, each released separately on PyPI. They are complementary but focus on different things:
| you have | use | it gives you |
|---|---|---|
| a video file, with or without sound | musicalgestures (this one) | motiongrams, videograms, motion analysis from video |
| a motion time series from markers or sensors | micromotion | quantity of motion (0.2–5 Hz), posture, balance |
| environmental audio recordings — mono, stereo, binaural or ambisonic | ambiscape | level, spectrum, space, time, sources |
| long music recordings (concerts or in environments) | musiscape | many tracks compared at a glance |
The toolboxes are designed to call relevant functions between them using a single implementation, so results don't depend on which package you called.
If you use this toolbox in your research, please cite this article:
Laczkó, B., & Jensenius, A. R. (2021). Reflections on the Development of the Musical Gestures Toolbox for Python. Proceedings of the Nordic Sound and Music Computing Conference, Copenhagen.
If you want to cite the toolbox itself, use the Zenodo CONCEPT DOI, which always resolves to the newest version:
Jensenius, A. R., Laczkó, B., Poutaraud, J., Widmer, M., Furmyr, F., Guo, J., Clim, A., Upham, F., & von Arnim, H. A. (2026). Musical Gestures Toolbox for Python [Computer software]. Zenodo https://doi.org/10.5281/zenodo.21965729
Where the exact behaviour matters, cite the version you ran.
This toolbox builds on the Musical Gestures Toolbox for Matlab, which again builds on the Musical Gestures Toolbox for Max. Many researchers and research assistants have helped its development (both directly and indirectly) over the years; see the contributor list on Zenodo for details.
The software is developed at the fourMs lab, RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion, University of Oslo.
This toolbox is released under the GNU General Public License 3.0.