Mind freakin’ blown.
Scientists at MIT have found a way to extract audio from silent videos, where the only information presented is the visual component.
Graduate student Abe Davis explains:
“When sound hits an object, it causes the object to vibrate. The motion of this vibration creates a very subtle visual signal that’s usually invisible to the naked eye. People didn’t realize that this information was there.
We’re recovering sounds from objects. That gives us a lot of information about the sound that’s going on around the object, but it also gives us a lot of information about the object itself, because different objects are going to respond to sound in different ways.
Sound moves at a wide range of frequencies, so the better the camera (meaning the higher speeds it can capture), the better the sound one can achieve from the vibrations. The scientists used cameras that can record up to 6,000 frames per second, which is about 100x more powerful than smartphone cameras but well below the commercial high speed cameras that are able to achieve 100,000 fps.
It’d be pretty amazing to see if they could take older silent films, such as Nosferatu and The Cabinet Of Dr. Caligari, and recover the audio from those.
There’s an amazing video of the process below and you can get greater details about this research here.