AI-Driven Motion Capture Introduces an Unseen Dance Experience
Summary
In February 2024, star ballet dancer Friedemann Vogel premiered his solo performance Die Seele am Faden at the Kleist Forum in Frankfurt (Oder). Inspired by Heinrich von Kleist’s famous marionette essay, the piece stages a choreographic dialogue between a human performer and his puppet-like double: a 3D avatar that mirrors and extends Vogel’s movements on stage.
ToTheFullest was invited to make this digital double possible. In close collaboration with digital artist and director Timo Kreitz, we designed and implemented an AI-based motion capture and 3D scanning setup that could translate Vogel’s precise ballet vocabulary into an expressive, stage-ready avatar. The result is a performance in which human nuance and digital exactitude coexist.
Our Work
The artistic concept called for a technically ambitious illusion: a virtual marionette that moves with the same control, elegance, and timing as one of the world’s leading ballet dancers. At the same time, the production needed a lean, reliable setup that could be executed within the constraints of a live theatre environment.
Our role was to turn this vision into a robust pipeline. We engineered an AI-driven motion capture workflow using a compact, camera-based setup that captured Vogel’s movements from multiple angles. Instead of relying on traditional marker suits or complex rigs, we used computer vision to extract his motion frame by frame and translate it into clean, animation-ready data. This allowed us to preserve the subtlety of his technique – from the articulation of his feet to the dynamics of his turns.
In parallel, we conducted a high-fidelity 3D scan of Vogel using a LiDAR-based system (a laser-driven 3D scanning method). This process enabled us to create a detailed digital replica of his physique that could stand up to close viewing and theatrical lighting. Combining this body data with the captured motion, we delivered a fully rigged 3D avatar that met both the artistic and technical standards of the creative team.
Crucially, all motion capture was recorded in advance and precisely aligned with the choreography and musical structure of the piece. Because Vogel’s timing is exceptionally consistent, the digital marionette could be projected onto semi-transparent gauze in near-perfect sync with his live performance. The audience encounters not just a visual effect, but a genuine pas de deux between human dancer and algorithmically animated double.
For ToTheFullest, this project demonstrates how thoughtful strategy and emerging technology can open up new storytelling spaces in the performing arts. By focusing on a minimal but highly precise setup, we helped transform an abstract idea about puppetry and control into a concrete, immersive stage experience, while keeping roles and responsibilities in the production clearly defined: artistic concept and choreography by the creative team, motion capture and 3D scanning pipeline by ToTheFullest.
About Friedemann Vogel
Friedemann Vogel is an internationally acclaimed principal dancer, performing on renowned stages around the world and known for his musicality, precision, and emotional depth. With Die Seele am Faden, he explores the boundary between human and digital presence on stage – inviting audiences to reflect on control, autonomy, and the future of performance.
AI-Driven Motion Capture Introduces an Unseen Dance Experience
Summary
In February 2024, star ballet dancer Friedemann Vogel premiered his solo performance Die Seele am Faden at the Kleist Forum in Frankfurt (Oder). Inspired by Heinrich von Kleist’s famous marionette essay, the piece stages a choreographic dialogue between a human performer and his puppet-like double: a 3D avatar that mirrors and extends Vogel’s movements on stage.
ToTheFullest was invited to make this digital double possible. In close collaboration with digital artist and director Timo Kreitz, we designed and implemented an AI-based motion capture and 3D scanning setup that could translate Vogel’s precise ballet vocabulary into an expressive, stage-ready avatar. The result is a performance in which human nuance and digital exactitude coexist.
Our Work
The artistic concept called for a technically ambitious illusion: a virtual marionette that moves with the same control, elegance, and timing as one of the world’s leading ballet dancers. At the same time, the production needed a lean, reliable setup that could be executed within the constraints of a live theatre environment.
Our role was to turn this vision into a robust pipeline. We engineered an AI-driven motion capture workflow using a compact, camera-based setup that captured Vogel’s movements from multiple angles. Instead of relying on traditional marker suits or complex rigs, we used computer vision to extract his motion frame by frame and translate it into clean, animation-ready data. This allowed us to preserve the subtlety of his technique – from the articulation of his feet to the dynamics of his turns.
In parallel, we conducted a high-fidelity 3D scan of Vogel using a LiDAR-based system (a laser-driven 3D scanning method). This process enabled us to create a detailed digital replica of his physique that could stand up to close viewing and theatrical lighting. Combining this body data with the captured motion, we delivered a fully rigged 3D avatar that met both the artistic and technical standards of the creative team.
Crucially, all motion capture was recorded in advance and precisely aligned with the choreography and musical structure of the piece. Because Vogel’s timing is exceptionally consistent, the digital marionette could be projected onto semi-transparent gauze in near-perfect sync with his live performance. The audience encounters not just a visual effect, but a genuine pas de deux between human dancer and algorithmically animated double.
For ToTheFullest, this project demonstrates how thoughtful strategy and emerging technology can open up new storytelling spaces in the performing arts. By focusing on a minimal but highly precise setup, we helped transform an abstract idea about puppetry and control into a concrete, immersive stage experience, while keeping roles and responsibilities in the production clearly defined: artistic concept and choreography by the creative team, motion capture and 3D scanning pipeline by ToTheFullest.
About Friedemann Vogel
Friedemann Vogel is an internationally acclaimed principal dancer, performing on renowned stages around the world and known for his musicality, precision, and emotional depth. With Die Seele am Faden, he explores the boundary between human and digital presence on stage – inviting audiences to reflect on control, autonomy, and the future of performance.
AI-Driven Motion Capture Introduces an Unseen Dance Experience
Summary
In February 2024, star ballet dancer Friedemann Vogel premiered his solo performance Die Seele am Faden at the Kleist Forum in Frankfurt (Oder). Inspired by Heinrich von Kleist’s famous marionette essay, the piece stages a choreographic dialogue between a human performer and his puppet-like double: a 3D avatar that mirrors and extends Vogel’s movements on stage.
ToTheFullest was invited to make this digital double possible. In close collaboration with digital artist and director Timo Kreitz, we designed and implemented an AI-based motion capture and 3D scanning setup that could translate Vogel’s precise ballet vocabulary into an expressive, stage-ready avatar. The result is a performance in which human nuance and digital exactitude coexist.
Our Work
The artistic concept called for a technically ambitious illusion: a virtual marionette that moves with the same control, elegance, and timing as one of the world’s leading ballet dancers. At the same time, the production needed a lean, reliable setup that could be executed within the constraints of a live theatre environment.
Our role was to turn this vision into a robust pipeline. We engineered an AI-driven motion capture workflow using a compact, camera-based setup that captured Vogel’s movements from multiple angles. Instead of relying on traditional marker suits or complex rigs, we used computer vision to extract his motion frame by frame and translate it into clean, animation-ready data. This allowed us to preserve the subtlety of his technique – from the articulation of his feet to the dynamics of his turns.
In parallel, we conducted a high-fidelity 3D scan of Vogel using a LiDAR-based system (a laser-driven 3D scanning method). This process enabled us to create a detailed digital replica of his physique that could stand up to close viewing and theatrical lighting. Combining this body data with the captured motion, we delivered a fully rigged 3D avatar that met both the artistic and technical standards of the creative team.
Crucially, all motion capture was recorded in advance and precisely aligned with the choreography and musical structure of the piece. Because Vogel’s timing is exceptionally consistent, the digital marionette could be projected onto semi-transparent gauze in near-perfect sync with his live performance. The audience encounters not just a visual effect, but a genuine pas de deux between human dancer and algorithmically animated double.
For ToTheFullest, this project demonstrates how thoughtful strategy and emerging technology can open up new storytelling spaces in the performing arts. By focusing on a minimal but highly precise setup, we helped transform an abstract idea about puppetry and control into a concrete, immersive stage experience, while keeping roles and responsibilities in the production clearly defined: artistic concept and choreography by the creative team, motion capture and 3D scanning pipeline by ToTheFullest.
About Friedemann Vogel
Friedemann Vogel is an internationally acclaimed principal dancer, performing on renowned stages around the world and known for his musicality, precision, and emotional depth. With Die Seele am Faden, he explores the boundary between human and digital presence on stage – inviting audiences to reflect on control, autonomy, and the future of performance.
