We taught a laptop to find the laws of physics from a video (and spot things it can't see)

Video2Law is a native Rust desktop app that turns observed motion into a mathematical model. Its central idea is to film something moving and let the computer work out the physics. This reverses the usual classroom approach, where students receive a formula first and substitute measurements. It matters because simple observations can reveal laws without manually supplying the textbook equation.

For a swinging object, the app tracks its position in each frame and smooths noise. It tests possible terms, such as sin θ, cos θ, and speed, then uses sparse regression to retain the terms that actually explain the motion. SVD also searches for a nearly unchanged combination of measured quantities, revealing energy. The app measures how quickly friction drains that energy.

It can then solve the discovered equation with RK4 and predict what happens next. It can also run a ghost pendulum, explore conditions such as the Moon, and search for hidden motion in Neptune Mode. So far, testing has used simulated videos, not real phone footage.

The visible pendulum does not move exactly like an isolated pendulum when a second pendulum is secretly connected to it. Its motion contains additional wobble or patterns caused by the hidden system. Video2Law uses those deviations as clues rather than treating them as ordinary noise. This matters because the app can infer part of a physical system that the camera never directly sees.

The app first tracks the visible pendulum through the frames. In Neptune Mode, it uses SVD and an FFT to search for extra patterns in that motion. SVD helps identify important combinations in the data, while an FFT helps expose recurring motion patterns. Patterns beyond the expected pendulum behavior suggest that another object is influencing the visible one.

The app then works out where the hidden pendulum must be and draws it. In the reported tests, it placed the hidden pendulum within half a centimetre, even though the right half of the video was hidden from the app. The result is a reconstruction based on motion evidence, not a direct view.

In the creators’ tests, Neptune Mode located the hidden pendulum within half a centimetre of its actual position. That is striking because the app did not receive a complete view of the experiment. The right half of the video was hidden, so the second pendulum remained off-screen from the app’s perspective.

The system relied on the visible pendulum’s motion instead. A hidden pendulum changes the visible one’s wobble, creating extra patterns beyond ordinary pendulum movement. Video2Law used SVD and an FFT to identify those patterns. From them, it inferred that an unseen object was influencing the visible pendulum and calculated where that object should be.

The result is an impressive test of indirect reconstruction, but the article describes it as a test on simulated videos. The stated accuracy therefore applies to those tests, not necessarily to real camera footage. Real phone videos may introduce noise and tracking difficulties, which the creators identify as a next step rather than a completed result.

Once Video2Law has discovered the pendulum’s equation, it can use that model rather than merely describe the recorded motion. This matters because a learned equation becomes a tool for prediction and experimentation. The app can show how the motion should continue and let users explore altered conditions without rebuilding the physical setup.

It solves the equation step by step with RK4, a numerical method for producing approximate future states. The app displays a “ghost” pendulum that runs ahead of the real one. Users can also ask what would happen if the pendulum were on the Moon. The article presents this as changing the simulated environment after learning the motion.

The prediction depends on the quality of the discovered model. The creators tested their system with physics-engine videos, where they knew the correct answer and could catch bugs. They have not yet tested real phone footage. That real-world test is the next step, so the article does not claim that every prediction already works reliably outside simulation.

Neptune Mode imitates the reasoning behind Neptune’s discovery in the 1840s. Uranus was not moving exactly as Newton’s laws predicted. Astronomers treated that mismatch as evidence that another body was pulling on it. They calculated where the hidden planet had to be, and Neptune was found there. The app applies the same pattern of indirect inference to a pendulum.

In the app’s experiment, the camera sees one pendulum while a second is connected but hidden off-screen. The visible pendulum’s unexpected wobble contains the clue. Video2Law searches for extra patterns using SVD and an FFT. Those patterns indicate that something unseen is affecting the visible motion, allowing the app to calculate and draw the hidden pendulum.

The analogy is about method, not scale. Both cases begin with motion that does not fit the expected model, then use mathematics to infer an unseen influence. In the app’s simulated tests, the hidden pendulum was placed within half a centimetre while only half the video was visible.

The team created test videos with a physics engine so the underlying truth was known. That gave them a reliable benchmark for checking whether Video2Law recovered the right motion, energy, and hidden-object behavior. Comparing results with known answers also helped them find most of their bugs. The approach made development faster and more controlled.

Simulated footage is especially useful for a one-day hackathon project because the team can control the setup and measurements. They can create the motion they intend to study and compare the app’s output against it. This is how they tested Neptune Mode’s reported placement accuracy and other parts of the system.

However, simulation is not the same as recording a real experiment. The article states that the system has not yet been tested on real phone footage. It therefore leaves open how well tracking, noise smoothing, and equation discovery will work with ordinary camera recordings. Testing real phone videos is the next step.

An equation of motion is a mathematical rule describing how an object’s position, speed, or angle changes over time. A conserved quantity is a combination of measurements that stays constant, or nearly constant, during ideal motion. Energy is a familiar example. Finding these from observations matters because the computer can build a physical model without being handed the final formula.

Video2Law tracks the object, smooths measurement noise, and creates candidate terms such as sin θ, cos θ, and speed. Sparse regression compares those terms and keeps only the small set that explains the observed motion. For energy, SVD examines the data matrix and finds a combination that changes very little over time. That combination reveals the system’s conserved quantity.

These tools turn recorded motion into testable mathematics. The discovered equation can be solved with RK4 to predict the future, while energy loss can be measured when friction is present. The team’s simulated videos supplied known answers for checking the results, but real phone footage remains untested.