How AI Is Changing Fitness — From Wearables to Real-Time Form Judging

For most of the last decade, 'AI in fitness' meant a wearable estimating your heart rate and calories. Useful, but passive — it measured what already happened. The more interesting shift over the last few years is active: computer vision models that watch your body move, in real time, from an ordinary camera, and judge the movement itself.

What pose estimation actually does

Pose estimation models map the human body to a skeleton of key points — shoulders, elbows, hips, knees, ankles — dozens of times per second, from a normal 2D video feed. From that skeleton, software can calculate joint angles, depth, tempo, and symmetry: exactly the inputs a human coach would use to judge a rep, just measured with far more precision and consistency than an eye can manage.

Why on-device matters

A meaningful design choice in modern pose-estimation is running the model directly in the browser or on-device, rather than uploading video to a server. It's faster (no round-trip latency), cheaper to run at scale, and — importantly for user trust — means your actual video never has to leave your device for the scoring to happen. REPPED's rep-counting and form-scoring runs entirely client-side for exactly this reason; only the final numeric result is ever sent anywhere.

What a joint angle can and cannot tell you

It is worth being precise about what this technology measures, because the marketing around it rarely is. A pose model returns coordinates. Everything else — depth, tempo, symmetry, whether a repetition counted — is arithmetic performed on those coordinates by code somebody wrote. The model is not judging your form. It is reporting where your knee is, and a separate set of rules decides what that means.

That distinction matters because the rules are where the opinions live. Deciding that a squat counts below 110 degrees is a judgement call, not a measurement. A different system could reasonably pick 100 or 120 and be equally defensible. What the camera gives you is consistency: whatever threshold is chosen gets applied identically to every rep, by every person, on every device, which is more than can be said for a human counting in a gym.

The honest summary is that computer vision does not remove subjectivity from judging exercise. It moves the subjectivity out of the moment and into the rules, where it can be written down, argued with and applied the same way to everybody.

Two-dimensional video has real blind spots

A single ordinary camera sees a flat projection of a three-dimensional body, and some information simply is not recoverable from it. Rotation toward or away from the lens compresses limbs in ways that look identical to those limbs being shorter. A joint hidden behind another part of your body has to be estimated rather than seen.

This is the actual reason systems ask you to stand side-on with your whole body in frame. It is not a preference. It is the arrangement in which the angles that matter — elbow, hip, knee — lie roughly in the plane the camera can measure, so they can be read rather than inferred.

It also explains a category of complaint that sounds like a bug and is not. If your reps stop counting when you drift toward the edge of the frame, nothing has broken; the joints the scoring depends on have left the picture. A system that kept confidently counting in that situation would be worse, not better, because it would be making the numbers up.

What happens when the judge cannot see a body

The easiest way to see what a pose model will and will not accept is to hand it something that is almost a person. To take screenshots for this site without putting anyone on camera, a battle was played against REPBOT with a flat grey mannequin doing squats in place of a player. To a human eye it is obviously squatting. To the model it is not a body at all: it finds no shoulders, hips or knees it trusts, so it measures no angles, credits no repetitions, and the battle screen spends thirty seconds asking the figure to get back in frame.

The result screen is the other half of the rule. With almost no frames in which a body was visible, the score is not just low, it is capped, and the screen says so. A judge that guessed generously when it could not see would be easy to cheat by stepping half out of shot. A judge that refuses to guess is occasionally annoying and never fooled that way.

The privacy argument is a technical one

There is a version of on-device processing that is a marketing claim and a version that is an architectural fact, and they are easy to confuse. The distinguishing question is simple: does the video have to leave the device for the product to work at all? If the scoring happens in a data centre, the answer is yes, and every promise about deletion is a promise about what someone chooses to do with a file they already have.

If the model runs in the browser, the answer is no, and the promise is structural instead. There is no upload to delete because there was never an upload. What travels is a number — a rep count, a score — which is not reconstructable into a picture of anyone's living room.

For a product where the whole premise is pointing a camera at yourself, often at home, often as a teenager, this is not a small detail. It is most of the reason the thing is defensible at all.

Where this is heading

• Real-time form correction during a workout, not after it
• Objective scoring that removes subjective 'that counts, right?' arguments
• Fair matchmaking based on measured performance, not self-reported fitness level
• Competitive formats that weren't possible when judging required a human referee for every rep

The honest limitations

Camera-based pose estimation isn't a medical device and isn't perfect in poor lighting, unusual camera angles, or with the body partially out of frame — which is why fair systems build in visibility checks and graceful fallbacks rather than pretending to be flawless. REPPED, for instance, flags a rep when a player isn't sufficiently visible rather than guessing. Read our full explanation of how the AI judges every rep for the specifics.