Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure
Robotics Institute, Carnegie Mellon University
Key Takeaways
Steer a learned policy with a sentence: ReStruct performs inference-time policy steering from a preference stated in plain language. Object-centric requests and temporal orderings alike are compiled into a temporal-logic (LTLf) automaton, with no retraining and no expert-written guidance.
Steering you can verify: the state machine shows which transitions a preference removes and the prototype action priors show which actions remain, so the preference is checked at both the symbolic and the control level.
New behavior from existing experience: ReStruct composes a target behavior by injecting the preference into demonstrations of related tasks, improving data efficiency and cross-task adaptation.
Abstract
A central challenge in deploying learned robot policies is inference-time behavior steering: redirecting a policy at test time to satisfy user preferences not anticipated during training, without retraining. Existing methods fail in two modes: end-to-end methods require fine-tuning or expert-level guidance, while neuro-symbolic methods rely on predefined symbols whose edits can result in logically reasonable but physically infeasible plans. To address this challenge, we propose ReStruct, which builds upon a neural automaton policy that decomposes a visuomotor policy into a high-level state-machine skeleton capturing task structure and a low-level continuous controller represented as a residual policy. Specifically, ReStruct adopts the automaton to represent the preference and incorporates it into the skeleton through a synchronous product, thereby reconfiguring the task structure. With the controller kept frozen, the action priors provided by the skeleton are updated accordingly to enable physically-aware control under a modified task structure. Extensive experiments from simulation and real-world show that ReStruct steers a wide range of preferences, from object-centric specifications to temporal-logic constraints, and after steering surpasses existing methods, exceeding VLA models in both task success and preference-following by up to 25%.
Preliminary: Neural Automaton PolicyENAP · RSS 2026Project page ↗
A neuro-symbolic policy learned from demonstrations: a probabilistic Mealy machine (PMM) over visual symbols tracks the task phase qt and gives a small control network πψ an action prior to refine. We build on a state machine because it is the most general of the usual task representations: PDDL domains and LTLf formulas can both be written as one, so a preference in either form merges into the policy’s own structure.
Two ways steering fails
End-to-end policies (VLAs, diffusion policies) and neuro-symbolic policies fail in opposite ways; ReStruct avoids both.
Base policy. Trained on demonstrations that put the can into the trash bin and the mug onto the shelf; neither φ1 Put the can into the shelf. nor φ2 Put the can to the higher place. appears in them.
End-to-end policy. VLAs and diffusion policies take the preference through fine-tuning, the prompt, or a guidance cost added to sampling. With no explicit task structure to edit, each new preference needs retraining or expert-written, roughly differentiable guidance.
Neuro-symbolic policy. A planner edits a plan over predefined operators, e.g. dropping throw(?b) for pick(can) → place(shelf), and networks execute it. The edit never reaches the controller, so the plan for φ1 is logically valid but physically infeasible (a vertical-clearance violation), and φ2 cannot be expressed: “higher place” is not in the vocabulary.
ReStruct. The preference becomes an automaton that edits the policy’s own learned state machine, and each edge’s action priors are rebuilt from demonstrations that satisfy it. The symbols come from the demonstrations and the controller stays frozen, so the edited plan stays executable.
Three steps, one frozen controller
ReStruct builds on ENAP, a neuro-symbolic policy that pairs a learned state machine with a small action network. Steering is a standard construction from formal methods: the preference becomes an LTLf formula and its automaton, whose synchronous product with the state machine keeps only the transitions the formula allows. Only the state machine and the action lists on its edges change; the controller stays frozen.
Overview. The paper’s pipeline, and below it the same pipeline on StackCube: one sentence, and the policy only ever takes the green-cube branch.
Base policy. ENAP clusters camera frames into symbols and learns a state machine ℳ1 over them; both stacking orders are branches of it.
1 · Compile. A VLM grounds “red cube” in the policy’s own symbol c1 and writes □¬c1; Spot turns it into the automaton A.
2 · Reconfigure. ℳ2 = ℳ1 ⊗ A keeps only the transitions A allows: the c1 edges are cut and q8 disappears.
3 · Replay. Demonstrations walk ℳ2; accepted ones refill its action lists, rejected ones are discarded.
Steered policy. ℳ2 drives the same network weights, and the robot now always grasps the green cube.
Steer it yourself
Pick a task and a preference, or type your own.
Learned symbols
Preference
Base policy
Recorded ManiSkill rollouts of the trained policy, replayed slowed down. Symbol captions are VLM-generated from sample frames of each cluster; the policy itself only uses the cluster index.
Preference kept, task intact
Every method is trained on the same demonstrations and steered in the way its architecture allows: vision-language-action models (π0.5, OpenVLA) by prompt injection, Diffusion Policy by a soft Lagrangian constraint, and Transformer and GMM policies by reject filtering.
Averaged over five preferences, ReStruct succeeds with the preference kept (SRP) in 70.9% of ManiSkill trials, against 33.7% for π0.5, 19.1% for OpenVLA and 33.8% for Diffusion Policy, and it also has the highest task success (73.4%). On the real Kinova Gen3 robot its SRP is 92.5%, against 23.1% for π0.5 steered through its prompt. Reject filtering keeps Transformer and GMM compliant only by discarding rollouts, which lowers their task success.
Full tables
| Method | StackCube | PegInsert | SeqPushT | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Base SR | φ₁ SR | φ₁ SRP | φ₂ SR | φ₂ SRP | Base SR | φ₁ SR | φ₁ SRP | Base SR | φ₁ SR | φ₁ SRP | φ₂ SR | φ₂ SRP | |
| Transformer | 89.1 | 29.7 | 29.7 | 59.4 | 59.4 | 90.6 | 60.9 | 60.9 | 37.5 | 21.9 | 21.9 | 15.6 | 15.6 |
| GMM | 87.5 | 42.2 | 42.2 | 45.3 | 45.3 | 85.9 | 53.1 | 53.1 | 53.1 | 34.4 | 34.4 | 21.9 | 21.9 |
| Diffusion Policy | 93.8 | 87.5 | 68.8 | 81.2 | 42.2 | 57.8 | 42.2 | 20.3 | 40.6 | 28.1 | 18.8 | 31.2 | 18.8 |
| OpenVLA | 81.2 | 67.2 | 21.9 | 57.8 | 28.1 | 68.8 | 54.7 | 29.7 | 18.8 | 20.3 | 12.5 | 7.8 | 3.1 |
| π0.5 | 92.2 | 85.9 | 37.5 | 73.4 | 48.4 | 85.9 | 75.0 | 50.0 | 28.1 | 31.2 | 26.6 | 12.5 | 6.2 |
| ReStruct | 95.3 | 98.4 | 98.4 | 76.6 | 76.6 | 98.4 | 95.3 | 95.3 | 46.9 | 53.1 | 50.0 | 43.8 | 34.4 |
| Method | CanSorting | PassTeaPot | ConeCover | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Base SR | φ₁ SR | φ₁ SRP | φ₂ SR | φ₂ SRP | Base SR | φ₁ SR | φ₁ SRP | Base SR | φ₁ SR | φ₁ SRP | φ₂ SR | φ₂ SRP | |
| π0.5 | 100.0 | 78.1 | 43.8 | 71.9 | 31.2 | 100.0 | 81.2 | 18.8 | 43.8 | 43.8 | 12.5 | 34.4 | 9.4 |
| ReStruct | 93.3 | 100.0 | 100.0 | 100.0 | 84.4 | 100.0 | 100.0 | 100.0 | 87.5 | 100.0 | 100.0 | 78.1 | 78.1 |
| Method | Params | Causal φ₁ | Causal φ₂ | Unconstrained φ₁ | Unconstrained φ₂ | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| (M) | 3/5 | 4/5 | 5/5 | 3/5 | 4/5 | 5/5 | 3/5 | 4/5 | 5/5 | 3/5 | 4/5 | 5/5 | |
| HULC | 47 | 12.5 | 10.9 | 1.6 | 7.8 | 3.1 | 0.0 | 6.3 | 0.0 | 0.0 | 3.1 | 3.1 | 3.1 |
| LCD | 68 | 15.6 | 7.8 | 7.8 | 18.8 | 12.5 | 12.5 | 14.1 | 10.9 | 3.1 | 15.6 | 15.6 | 9.4 |
| MDT | 440 | 31.3 | 31.3 | 3.1 | 100.0 | 100.0 | 35.9 | 14.1 | 10.9 | 10.9 | 15.6 | 15.6 | 15.6 |
| Flower | 947 | 96.9 | 57.8 | 57.8 | 92.2 | 42.2 | 39.1 | 64.1 | 51.6 | 45.3 | 73.4 | 70.3 | 60.9 |
| Flower* | 947 | 100.0 | 87.5 | 87.5 | 100.0 | 54.7 | 51.6 | 78.1 | 78.1 | 54.7 | 93.8 | 93.8 | 76.6 |
| ReStruct | 571 | 98.4 | 93.8 | 92.2 | 100.0 | 81.3 | 76.6 | 90.6 | 84.4 | 81.3 | 100.0 | 93.8 | 93.8 |
Reusing related experience
One state machine spans all demonstrations, so a preference can route the policy through pieces of different ones.
Red can, green bowl
CanSorting demonstrations share a pick phase, then branch by bowl. The steered rollout picks the red can as the red-can demonstrations do, then takes the green-bowl branch that only green-can demonstrations used. Hover a video to play it.
One order from rollouts in many orders
On CALVIN, a requested order collapses the skill graph extracted from FLOWER rollouts into a single chain, without fine-tuning. Averaged over four requested orders, ReStruct (571M parameters) completes all five skills in 86.0% of trials, against 50.8% for the FLOWER vision-language-action policy (947M) and 67.6% for FLOWER fine-tuned on the requested order.
Questions
What is ReStruct?
ReStruct is a zero-shot, inference-time policy steering method for robot manipulation. It turns a preference stated in plain language into an automaton and merges it into the state machine of a neural automaton policy (ENAP), so the trained visuomotor policy follows the preference without retraining.
How is it different from steering a VLA or a diffusion policy?
Vision-language-action models are steered through their prompt, and diffusion policies through guidance or constraints added to sampling. Neither exposes a task structure to edit; in our experiments both keep task success but often fail to keep the preference. ReStruct edits the learned task structure directly and rebuilds each transition’s action prior from demonstrations that satisfy the preference, while the low-level controller stays frozen.
Does it need retraining or new demonstrations?
No. The network weights are unchanged and no new data is collected: the existing demonstrations that the reconfigured state machine accepts are replayed to refill its action lists.
Does ReStruct guarantee that the preference is satisfied?
At the level of the learned state machine, yes, by construction: the steered machine ℳ2 = ℳ1 ⊗ A, the synchronous product of the policy’s state machine with the preference automaton, keeps only the transitions the LTLf formula allows, and replay keeps only demonstrations whose symbol sequences the automaton accepts. Whether the physical execution satisfies the preference also depends on how well the learned symbols and the VLM-written formula capture it, so that part is measured in experiments (SRP in the tables), not proven.
Which preferences can it follow, and where are its limits?
Preferences that can be written as LTLf formulas over the policy’s learned symbols, which a VLM can refine (for example, splitting “pick” into “pick red” and “pick blue”): object-centric requests, avoidance, and orderings of subtasks. The preferred behavior must have been demonstrated, either whole or as pieces that can be recombined; eliciting new skills is not yet explored. Results also depend on the formula the VLM writes, and ReStruct applies to policies that are, or are distilled into, ENAP.
How does it relate to ENAP?
ENAP (RSS 2026) learns the state machine and the residual controller from demonstrations; ReStruct is the inference-time steering layer on top of it. On CALVIN, ReStruct steers an ENAP policy distilled from the FLOWER vision-language-action model.
BibTeX
Accepted to the Conference on Robot Learning (CoRL) 2026 · OpenReview · arXiv:2606.26588
@article{pan2026inference,
title={Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure},
author={Pan, Yiyuan and Hu, Hanjiang and Li, Shangtao and Luo, Xusheng and Liu, Changliu},
journal={arXiv preprint arXiv:2606.26588},
year={2026}
}