Guide · Policy deployment

How to run SmolVLA on an SO-101 arm

Published 28 Oct 2026 · Checked against the LeRobot docs (SmolVLA, Imitation Learning on Real-World Robots, Real-Time Chunking) on 28 Oct 2026

Short answer. You don't run the downloaded SmolVLA as-is. lerobot/smolvla_base is a base model, and the LeRobot docs say "fine-tuning on your own data is required for optimal performance in your setup". Record about 50 teleoperated episodes of your task with lerobot-record, fine-tune from lerobot/smolvla_base with lerobot-train (the docs estimate ~4 hours for 20k steps on one A100), then run the result on the arm with lerobot-rollout. Use the same robot.id as at calibration and the same task sentence you recorded with.

What is SmolVLA?

SmolVLA is Hugging Face's small vision-language-action (VLA) model. It takes camera images, the arm's joint state and a language instruction, and outputs a chunk of actions (Shukor et al., arXiv:2506.01844). The model card lists those outputs as continuous actions. The paper describes the main model as 450 million parameters, about 100 million of them in the action expert. It is pretrained on 481 community datasets totalling 22.9k episodes, all from SO-100 arms. The authors note it was "not pretrained on any datasets recorded for the SO101". That is one reason your own SO-101 data matters.

Before you start

• A calibrated follower and leader arm. Calibration maps joint positions into the values the policy sees. If you skip it or change it later, the policy quietly drives to the wrong place. See How to calibrate an SO-101 arm in LeRobot.
• Fixed cameras. The LeRobot tutorial says to "keep the cameras fixed" across recordings (Imitation Learning on Real-World Robots).
• A Hugging Face login with a write token. Datasets and checkpoints are pushed to the Hub by default.
• A GPU for training. Inference is lighter. The paper says SmolVLA is designed to be "deployed on consumer-grade GPUs or even CPUs".

Install LeRobot, then the SmolVLA extras from the repo (SmolVLA docs):

pip install -e ".[smolvla]"

Step 1: How much data do you need?

The SmolVLA docs recommend "~50 episodes of your task as a starting point". The reference dataset used 50 episodes across 5 cube positions, 10 per position. The team adds: "We tried similar dataset with 25 episodes, and it was not enough leading to a bad performance." The general tutorial gives the same rule. Record at least 50 episodes, 10 per location, and "you should be able to do the task yourself by only looking at the camera images" (il_robots).

lerobot-record \
    --robot.type=so101_follower \
    --robot.port=/dev/ttyACM0 \
    --robot.id=my_follower_arm \
    --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
    --teleop.type=so101_leader \
    --teleop.port=/dev/ttyACM1 \
    --teleop.id=my_leader_arm \
    --dataset.repo_id=${HF_USER}/so101_pick_lego \
    --dataset.num_episodes=50 \
    --dataset.single_task="Grasp a lego block and put it in the bin."

During recording, → (or n) ends an episode early, ← (or r) re-records it, and Esc (or q) stops and uploads. Each episode defaults to 60 seconds with a 60-second reset (il_robots).

Step 2: Fine-tune from smolvla_base

From the SmolVLA docs, with your dataset swapped in:

lerobot-train \
  --policy.path=lerobot/smolvla_base \
  --dataset.repo_id=${HF_USER}/so101_pick_lego \
  --batch_size=64 \
  --steps=20000 \
  --output_dir=outputs/train/my_smolvla \
  --job_name=my_smolvla_training \
  --policy.device=cuda \
  --wandb.enable=true

The SmolVLA docs' estimate is "~4 hrs on a single A100" for 20k steps. If the batch doesn't fit, start smaller and work up. Without a local GPU, those docs point to a Colab notebook; the Imitation Learning tutorial also covers Hugging Face Jobs (--job.target=a10g-small, billed per second; see hf jobs hardware for prices).

Step 3: Run it on the arm

lerobot-rollout loads the policy and drives the follower (SmolVLA docs):

lerobot-rollout \
  --strategy.type=base \
  --robot.type=so101_follower \
  --robot.port=/dev/ttyACM0 \
  --robot.id=my_follower_arm \
  --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
  --task="Grasp a lego block and put it in the bin." \
  --policy.path=${HF_USER}/my_smolvla

Two things the docs make explicit, and one this guide adds:

1. Same id. The robot id is what LeRobot uses to find the calibration file. The tutorial says to "use the same id when teleoperating, recording, and evaluating".
2. Same sentence. The SmolVLA docs say to use "the same task description you used in your dataset recording".
3. Same cameras. Use the same camera names and setup as in the dataset. The docs' rollout comment is only "# <- Use your cameras"; matching the recording setup is this guide's advice, because a renamed or moved camera is a different observation than the one the policy trained on.

--strategy.type=base runs with no recording. Use sentry to record evaluations continuously, or episodic for episodes with resets in between (il_robots).

Why does the arm pause between moves?

SmolVLA predicts actions in chunks; the paper's model outputs chunks of 50. If the next chunk is computed only after the last one finishes, the arm waits. LeRobot offers Real-Time Chunking for this. It starts the next chunk early and blends its start into the motion already under way (RTC docs). Add:

  --inference.type=rtc \
  --inference.rtc.execution_horizon=10 \
  --inference.rtc.max_guidance_weight=10.0

The RTC docs give 8–12 as typical execution horizons and call 10.0 "an optimal value" of guidance for 10-step flow-matching policies like SmolVLA. The paper's own asynchronous inference stack, a separate mechanism, cut average task time on the SO-100 cube pick-and-place from 13.75 s to 9.7 s with similar success rates (arXiv:2506.01844, Fig. 5).

What success rate should you expect?

The only SO-101 number in the paper is a single-task Lego pick-and-place. SmolVLA scored 90% in-distribution and 50% with the brick in new positions, against 70% and 40% for ACT (arXiv:2506.01844, Table 4). Those are the authors' results on their setup. Your arm, lighting and data will give a different number. Out-of-distribution placements are where it fails, so record variation you expect at test time.

Common reasons it misbehaves

• Recalibrated after recording. The joint values no longer mean what they meant in the dataset. Re-record or restore the calibration file.
• Too few or too uniform episodes. 25 episodes was "not enough" in the SmolVLA team's own test.
• Different task text or camera names from the dataset.
• Moved camera. The tutorial's advice is fixed cameras, and to add variation slowly.

The same install-step notes are listed on the blog. On the homepage, this is the train-and-run step in the loop AVIOS is for. Calibration comes first: How to calibrate an SO-101 arm in LeRobot.

FAQ

Sources

• LeRobot docs, SmolVLA: https://huggingface.co/docs/lerobot/smolvla
• LeRobot docs, Imitation Learning on Real-World Robots: https://huggingface.co/docs/lerobot/il_robots
• LeRobot docs, Real-Time Chunking (RTC): https://huggingface.co/docs/lerobot/rtc
• lerobot/smolvla_base model card: https://huggingface.co/lerobot/smolvla_base
• Shukor M. et al. SmolVLA: A vision-language-action model for affordable and efficient robotics. arXiv:2506.01844: https://arxiv.org/abs/2506.01844