Product / Research and clinical decision support
Facial movement, measured in millimetres.
Advanced Face Therapy turns an ordinary phone camera into an objective measure of how a face moves, and turns facial exercises into games people actually want to do. One platform for therapy, measurement and the clinicians and researchers who need to see the difference.
Research and clinical decision-support platform. Not a diagnostic or treatment device. Validated on synthetic data so far; we are seeking clinical and research partners.
All figures come from synthetic validation, not from patient data.
What it does
Today, facial function is mostly judged by eye.
A clinician watches a smile and gives it a grade. It is quick, but it is subjective and hard to compare from one visit to the next. Advanced Face Therapy gives you numbers instead: how far each part of the face moved, how fast, how evenly left and right, and how that changes over weeks.
How it works
From camera to clinical timeline in four steps
Capture
A phone camera follows 468 points on the face in real time. Patients do guided exercises, or a clinician runs a standard 12-movement exam.
Map
Those camera points are mapped onto a 3D head model, so flat tracking becomes distances you can read in millimetres.
Measure
Each face is compared with its own neutral. The pipeline reports movement per region, left and right symmetry, timing, smoothness, blinks, fatigue and tremor-band activity.
Review
Clinicians and researchers see sessions, trends and study views in a web dashboard, with reliability statistics next to every number.
Live tracking, real therapy
The camera tracks the face. The face plays the game.
Using nothing but a phone camera, the app follows 468 points on a real face in real time. Those same points drive the therapy games: the patient smiles, the meter fills, and holding the smile completes the exercise.
Illustrative, measured from the tracked points on this example.
- 1Start at restThe game calibrates to this person’s own neutral face.
- 2Smile to fill the meterLeft and right sides are tracked separately.
- 3Hold the smile to finishTalking does not count, so the exercise stays honest.
Camera to 3D
From 68 points to 468, on a head you can measure
A camera can follow a face but knows nothing about its shape in 3D. A 3D model knows the shape but not where the camera is looking. Our mapping joins the two, placing all 468 tracked points on a parametric 3D head. That is what makes millimetres possible. We have open-sourced the mapping on GitHub.
Measurement
More than a smile score
A single recording gives you timing and speed as well as size, with each side of the face traced separately. Onset latency, rise time, smoothness, blinks, fatigue over sustained effort and tremor-band activity are all measured from the same capture.
Because each person is compared with their own neutral face, differences in face shape do not get mistaken for differences in movement.
Clinician dashboard
Sessions, timelines and study views
Every session shows symmetry, excursion and expressivity at a glance, with quality warnings when a recording is not good enough to trust. Timelines show each measurement with its own error band, so you can see whether a change is real or within normal variation.
The study view brings test-retest reliability and the smallest detectable change into the same screen.
Verification
Tested against the right answer
With real patients you never know the true answer. So we generate virtual patients with a known impairment, then check that the measurements recover it. A simulated 50% one-sided weakness is detected at over 100 times the measurement noise floor, and the engine passes 21 of 21 ground-truth checks. It has caught silent errors that ordinary testing would have missed. This is synthetic validation; real-patient validation is the next step.
The analysis web app
From a recording to a report in seconds
The clinician web app turns recordings into reports, and it is also where you build and test the 3D heads used for simulation.
Quick analysis
Drop in one file, get a full report
Drop one recording CSV and get a full report in seconds: symmetry, smile excursion in millimetres, expressivity and quality warnings. Nothing is stored in the patient library, so it is a safe way to try a recording or check a capture.
Session report
Measurements and a visual report
Every session lists its measurements, then adds a visual report with charts and animations. Print it as a PDF, or download everything for the session as a single zip.
GNM Head Studio
Generate a realistic 3D head
Shape a head with sliders: a sex and ancestry blend, a “character” scale that sets how distinctive the face is, and a seed that picks the person. Fine-tune the top PCA components, then download a Unity bundle or bake the head straight into the Unity project.
It can also build the head from a front photo or from a patient’s own recording, shown below.
From a photo
A head from a single front photo
Head Studio detects 468 landmarks on a frontal, neutral photo, including their relative depth, and fits the head’s identity to them against a prior for the person’s sex and ancestry, so the result stays within the range of real faces. The photo is then projected onto the face as a texture. Depth beyond the landmarks comes from the model, so a side view is a sensible estimate rather than a scan.
From patient data
A head from the patient’s own recording
Load a recording from the app and Head Studio fits the identity from the patient’s resting face, matching their tracked landmarks to within a few millimetres. The result is a virtual patient with that person’s face shape, ready to export to Unity.
Patients, timelines and the study report (test-retest reliability, group discrimination and a tidy.csv export) live in the same app, as shown in the clinician views above.
Simulation in Unity
A virtual patient that plays expressions
The GNM head, imported into Unity, runs through a set of expressions. The 468 MediaPipe landmarks are sampled from the deforming mesh every frame through our open-source correspondence. They are the same points the phone tracks, so the game logic and the analysis can be tested without a device or a patient.
- Smile
- Open smile
- Jaw open
- Pucker
- Eyes closed
- Weak left smile
The six points the smile detector reads
- Mouth corners
- Inner lips
- Outer eye corners
Machine learning
Models that earn their place
Machine learning is an assistive layer on top of measures a clinician can read. It is not a diagnosis.
Two feature routes
Geometric features (clinically meaningful distances and symmetry measures in millimetres) are compared with raw landmark PCA. On top of each we test a temporal convolutional network (TCN), a GRU and gradient boosting.
Interpretable at the centre
Geometric features stay readable for clinicians, which is why they sit at the centre. Models are judged against them, not instead of them.
Ready for the device
Trained models export to ONNX with verified numerical parity (maximum difference under 1e-6), so they can run on the phone.
Synthetic benchmark
| Features and model | Clip accuracy | Session accuracy | AUC | Severity MAE (lower is better) |
|---|---|---|---|---|
| Geometric + TCNBest | 0.993 | 1.000 | 1.000 | 0.053 |
| Geometric + GRU | 0.951 | 1.000 | 0.999 | 0.063 |
| Raw PCA + TCN | 0.882 | 0.917 | 0.982 | 0.103 |
| Geometric + gradient boosting | 0.868 | 0.875 | 0.976 | 0.059 |
These are results on a synthetic benchmark. They are not clinical validation, and they say nothing yet about performance on patients.
We test our own assumptions
An ablation showed the GRU struggled because one final state had to carry every timescale. A pooled head fixed it, lifting session accuracy from 0.500 to 0.875, while adding derivative channels made things worse.
A change is only reported if it is real
The minimal detectable change (MDC95) is computed per metric, so a change is reported only when it exceeds measurement error. In the simulated weakness test the commissure symmetry effect was 113x its MDC95.
The platform
Everything in one system
AR therapy app
Facial exercises become games. Smile-driven games and a voice game that adapts to the user and to room noise. Left and right smiles are tracked separately, a talking guard stops speech counting as exercise, and every user is calibrated. Sessions, training plans, progress and cloud save are built in.
Virtual patient simulator
A realistic parametric 3D head built on Google's open GNM model, with 70+ expressions, several identities, animated transitions and device-realistic tracking noise. Develop and test face-tracking features without a phone or a volunteer.
Camera to 3D bridge
The mapping of 468 tracked points onto the 3D head that turns camera tracking into millimetres. Our team has open-sourced it on GitHub so others can inspect and build on it.
Clinical measurement pipeline
Person-normalised measures, per-region movement in millimetres, symmetry, onset latency, rise time, speed, smoothness, blink detection, fatigue over sustained effort and tremor-band analysis.
Analysis panels
Panels for facial palsy including synkinesis, reduced expressivity (hypomimia) as seen in Parkinson's disease, myasthenia-type fatigability, stroke follow-up, pre and post surgical comparison, and general expressivity.
Synthetic ground-truth engine
Generates virtual patients with known impairment, so each measurement can be checked against the right answer. It passes 21 of 21 ground-truth checks and has caught silent errors that ordinary testing would miss.
GNM Head Studio
Head Studio builds heads from photos and patient data, or from parametric sliders, and sends them straight to Unity.
Machine learning, kept honest
Temporal models are benchmarked against interpretable geometric measures. The deployable model exports to ONNX at about 340 KB for on-device use. ML augments auditable measures; it does not replace them.
Statistics built in
Reliability (ICC, smallest detectable change), agreement analysis and sample-size planning for studies, in the same place as the data.
Who it is for
Built for people who work with faces
Clinical researchers
Standardised exams, millimetre outputs, reliability statistics and sample-size planning for movement-disorder, stroke and facial palsy studies.
Clinicians and therapists
A clear record of how a face moves over time, with timelines that show whether a change is larger than the measurement error.
Patients in therapy
Facial exercises as games, at home, with feedback on each side of the face and progress that is easy to see.
Teams building face-based products
A simulator, a ground-truth engine and an open mapping, so you can build and test face-tracking features properly.
Why it is different
Measurement you can question
Measured against each person
Every face is compared with its own neutral, not with an average face.
Interpretable first
Geometric measures you can audit sit at the centre. Machine learning is benchmarked against them.
Tested against known answers
Virtual patients with known impairment let us check the measurements, and have exposed errors normal tests missed.
Honest about uncertainty
Timelines show measurement error next to each change, and the study view reports when a metric is not reliable enough to use.
Therapy and measurement together
The same tracking powers the games patients play and the data clinicians review.
Privacy by design
Face identity is kept separate from movement data, and raw landmarks can be discarded after analysis.
Download Technical Report
AR Facial Tracking and Adaptive Smile Detection for a Therapeutic Exercise Application in Parkinson's-Related Hypomimia
A 24-page technical report on how the platform is built and analysed. It covers:
- System design
- Personalised, calibration-based smile detection
- Facial-data capture and the automated analysis pipeline
Tell us a little about your interest and we will send the report.
Cover
Contents
Executive summary
Movement over time, page 12
Expression signature, page 13
Muscle group map, page 15
The report has 24 pages. Only selected pages are shown here.
Partner with us
We are looking for research and clinical partners.
If you are building face-related solutions, or researching Parkinson's disease (facial hypomimia), facial palsy, stroke recovery or similar conditions, we would like to hear from you. The platform is ready for pilot studies. What it needs next is real patient data and the people who know how to interpret it.
Responsible use: Advanced Face Therapy is a research and clinical decision-support platform. It measures facial movement objectively to support clinicians and researchers. It does not diagnose, screen for or treat any condition, and it is not a CE-marked or FDA-cleared medical device. Validation to date has been on synthetic data; clinical validation on patients has not yet been carried out.