Capturing Motion
Capturing movement so it can play back later can take many forms. From the first zoopraxiscope (https://en.wikipedia.org/wiki/Zoopraxiscope) to the latest, state of the art, full body motion capture of the Avengers.
Motion capture also took different form in Video game and real-time 3D. One of most known is for sure Prince of Persia. It used rotoscoping: capture videos with a camera then draw by hand each individual frame on an Apple II.
https://www.forbes.com/sites/sethporges/2017/12/19/how-the-original-prince-of-persia-changed-video-game-animation/
Rotoscoping is a pretty old way to get believable drawn movement from real life characters. It’s been used for decades by Disney Studios.
Technology evolves very quickly and 30 years after the groundbreaking Out of this world (Another World for European folks) in 1991 it’s now possible to capture 3D movements from a device you carry in your pocket. Impressive leap.
Fast forward to 2022. 3D on the web is getting widespread, movement recognition algorithms are more and more robust and cameras with good quality photo sensors are here.
A few weeks ago, this article from 80.lv caught my eyes:
I decided I wanted to check it out and see results it can produce. Good point: it’s using Babylonjs !
Test 1: A parkour video with moving camera and superhuman movements
As expected, the root motion is totally lost but the poses are respected for most frames.
If it was possible to import a simplified mocap set and lock bones, it would be possible to improve the motion, so it corresponds to what the game motion constraints.
Test 2: A simple walk video found on Youtube
Video is not perfectly vertical, so is the capture. A very important step is to get a good video quality. Bad video contrasts will add shaking. Bad mounted camera will break your root orientation.
Test 3: Direct capture from a MBPro front camera
Lighting and camera position were not great. My foot is hidden behind some furniture at the end of video.
Test 4: 100 Different Ways to Walk by Kevin Parry
Good contrast, good framing and you get what you deserve. Free use of Plask.ai is limited to 5 minutes of video and time to process it roughly 6 times the length of video.
Result has been exported as glb and imported as-is in Babylonjs. Easy.
So to sum up — Plask is a simple, free, easy-to-use tool that can be a perfect solution for prototyping CG motion based on video capture of human performance. It is not perfect, but the more effort you put into feeding it great video data (stable, good framing, good contrast, etc.) the better the results will be. All around this is a great tool to check out and the animation produced can easily be cleaned up and refined in any DCC tool…oh…and it’s incredible to see it built on top of Babylon.js.
