PROMO
PROMO
Preference-Conditioned Multi-Objective Reinforcement Learning for Quadrupedal Robots
What should the robot prioritise — not how should it move?
1 University of Manchester, UK | 2 BAE Systems, UK | 3 University of Warwick, UK
Overview
Quadrupedal locomotion balances command tracking, stability and energy efficiency. PROMO makes that balance an explicit runtime input: an operator changes priorities while the same policy keeps controlling the robot.
Learned control
-
Train → fixed trade-off. One scalar reward decides tracking vs stability vs efficiency, once.
-
New mission priority? Retrain · redesign reward · switch controller.
-
Discrete behaviours, and a deployment cost that grows with every new trade-off.
Priorities as an input
-
Train → runtime trade-off. A preference vector w over tracking, stability and efficiency is a policy input.
-
New mission priority? Move the preference.
-
A continuum of behaviours from one deployed actor.
Preference control
Watch priorities change mid-run, compare four recorded preferences, then examine how the same policy produces different gaits.
Live switch
Priorities change while the robot keeps walking. The overlay shows commands and preferences; the policy stays fixed.
Jump to a preference change using the chapters above.
Compare
Choose an anchor to watch a recorded hardware trial. The measurements show both the benefit and the cost of each choice, relative to balanced priorities.
Balanced priorities: a compromise between command following, body regulation and effort.
Matched-command hardware trials, n = 5 per anchor; lower is better for all three metrics. Dragging between anchors explores the weights; recordings and measurements are available only at the four anchors.
Emerging different gaits
No gait is prescribed. The policy discovers the motion that serves each priority: larger steps for tracking, active body regulation for stability, and compact steps for efficiency.
Same deployed policy; only the preference changes. Two-second loops.
Try itDrive the policy in your browser
The trained policy runs locally in your browser. Set the preference, drive with the keyboard, kick the robot, add payload or change friction and watch the gait, torque and power respond.
About 5 MB download, then everything runs on your device. The Go1 policy is executed in MuJoCo rather than the training simulator, so behaviour can differ from the paper.
Method
Three semantic objectives describe what the operator wants to prioritise. Fixed locomotion priors guide how the robot moves; training teaches one actor to respond to the preference.
Objective designSemantic objectives, fixed priors
How to move, versus what should matter. Gait-conditioned controllers expose physical descriptors — gait timing, stance geometry, foot clearance — and so say how to move. A semantic preference says what to prioritise, and leaves the gait to the policy.
But locomotion rewards also carry posture, contact-shaping and smoothness terms. Making each one a MORL objective would turn reward-engineering decisions into deployment controls and inflate the Pareto space.
PROMO factorises instead: three semantic objectives form the operator interface, while embodiment-level terms stay fixed priors that never enter w.
ArchitectureTraining and deployment
PROMO extends TAR with a preference-conditioned actor and one critic value head per objective. Training balances tracking, stability and efficiency while keeping locomotion priors fixed; an action-diversity regulariser encourages the actor to respond to the preference.
Evaluation
Simulation tests the preference space densely. Hardware trials test four anchors across three command regimes, measuring improvements alongside their trade-offs.
SimulationA continuous range of choices
Across 100 tested preferences, increasing an objective’s weight generally improves its realised return. Tracking and efficiency conflict most strongly: better command following requires more actuation. The policy learns a range of compromises, beyond the four hardware anchors.
Protocol: 100 sampled preference vectors, each evaluated in 1024 parallel environments for 20 seconds under domain randomisation and external disturbances.
Stability and efficiency: trade-offs and preference responses
Stability


Efficiency


HardwareThe priorities transfer to the real robot
One simulation-trained policy, deployed zero-shot on a Unitree Go2. Each objective-focused preference improves its corresponding metric across matched-command, fast and slow regimes. There are five trials per preference in each regime.
Average improvements across the three regimes
Mean percentage reductions across matched-command, fast and slow regimes, relative to balanced; five trials per condition.
Better tracking costs more energy and actuation; efficiency reduces effort but slows response. Stability benefits depend on the motion regime. The explorer above shows these trade-offs for matched commands; the table below reports the same matched-command measurements, including the costs.
| Preference | Position MAE (m) ↓ | Peak attitude (rad) ↓ | Specific energy (J/m) ↓ |
|---|---|---|---|
| Balanced | 0.534baseline | 0.196baseline | 124.6baseline |
| Tracking | 0.405−24.2% | 0.281+43.4% | 169.6+36.1% |
| Stability | 0.538+0.7% | 0.158−19.4% | 132.4+6.3% |
| Efficiency | 0.624+16.9% | 0.187−4.6% | 102.3−17.9% |
Best observed reductions relative to balanced were 38.7% in position error (slow regime), 59.0% in peak body-attitude deviation (fast), and 30.4% in specific energy (fast). These are individual-regime results, rather than the averages above.
Hardware comparison across all three command regimes
Robustness
On a deliberately slippery surface, the same policy follows commands, expresses different priorities and recovers from disturbances.
The robot was intentionally placed on a hard plastic sheet coated with soap and water to simulate a surface with very low friction. Across all four preference anchors, it followed commands through acceleration and deceleration, responded to external disturbances and recovered without falling. The same policy was used throughout.
Command following on the soap-and-water-coated plastic sheet, with the balanced preference.
Changing preferences
Different priorities remain usable even with poor traction. Watch command following and recovery under each preference; the policy and slippery surface stay the same.
The overlay shows commands and preference weights in each ten-second clip.
External disturbances
External disturbances applied to the robot, shown from impact through recovery, using the balanced preference.
Additional recoveries · nine clips on low friction
Additional pushes, kicks and turning manoeuvres.
Full session · 3:25, no falls
The unedited session: four preferences, acceleration, deceleration and repeated disturbances on the slippery sheet, with no falls.
Full annotated session, 3:25. Press play to watch.
Limitations
The evidence has clear boundaries, and each one limits how far the claims extend:
- Preference coverage. On hardware, four anchor preferences with five trials each in three command regimes. Dense, continuous preference control is shown in simulation only; between the anchors, hardware behaviour is interpolated rather than measured.
- Scalarisation. Linear weighting reaches only the supported (convex) part of the Pareto front. A trade-off that lies in a concave region cannot be selected by any preference weight.
- Disturbances. Push forces were not instrumented. The recovery clips show that the policy recovers, not how much it can withstand, and they do not compare preferences under equal pushes.
- Scope. One robot and three objectives fixed at training time. Adding an objective, or changing what an existing one rewards, means training a new policy.
Citation
@article{mousa2026promo,
title = {PROMO: Preference-Conditioned Multi-Objective Reinforcement Learning
for Quadrupedal Robots},
author = {Mousa, Amr and Rachman, Rifny and Karavis, Neil and
Caprio, Michele and Allmendinger, Richard},
journal = {Submitted to IEEE Transactions on Robotics},
year = {2026},
url = {https://amrmousa.com/promo/}
}
Acknowledgements
The authors gratefully acknowledge the RAICO1 team for providing access to the robotic platform and for their support during hardware deployment. We thank Matthew Hale for his assistance with the experimental testing, Amy Johnson and Darren Cunningham for their operational support, and Professor Wei Pan for valuable technical discussions and insightful feedback that contributed to the development of this work.
This work builds on IsaacLab, RSL-RL and our earlier TARLoco framework.
One robot. Different mission priorities.
Code and the full video release are being prepared. In the meantime, see our earlier work on robust quadrupedal locomotion.
🐾 TARLoco project page