RobotControlStack/robot-control-stack: A lean, ROS-free sim-to-real framework for training and deploying Vision-Language-Action (VLA) models and RL agents. Native MuJoCo Gymnasium wrappers with synchronous execution for Franka, UR5e, xArm, SO101 and YAM. · GitHub

A lean, ROS-free Sim-to-Real framework for training and deploying Vision-Language-Action (VLA) models and Reinforcement Learning (RL) agents.

Robot Control Stack (RCS) is a flexible, native Gymnasium wrapper-based robot control interface designed specifically for modern robot learning and Vision-Language-Action (VLA) models.

It completely unifies MuJoCo simulation and real-world physical robot control into a single, seamless API. Currently, RCS natively supports five robots out-of-the-box: Franka FR3/Panda, xArm7, UR5e, SO101, and I2RT YAM.

Traditional robotics middleware (like ROS/ROS2) and complex motion planning pipelines (like MoveIt or standard ros2_control) are built for asynchronous, distributed systems. This often becomes a massive bottleneck when attempting to train modern, synchronous machine learning models.

RCS is built differently:

• Zero ROS Overhead: No complex message-passing, middleware, or network configuration required. Run natively in Python with a lightweight C++ backend.
• Frictionless Sim-to-Real: Train your Reinforcement Learning or VLA policies in our MuJoCo Gymnasium wrapper, and deploy the exact same code directly to physical hardware.
• Synchronous Execution: Optimized specifically for the highly parallelized, synchronous data collection required by modern ML workflows.
• Ready-to-Use Apps: Ships with pre-built applications for data collection via teleoperation and remote model inference via vlagents. See the teleoperation guide, and inference guide.

RCS utilizes a highly modular, wrapper-based architecture, allowing you to easily stack capabilities (cameras, grippers, action spaces) as needed.

Flexibly compose your Gymnasium environment to fit your exact training needs. For common environment compositions, factory functions such as rcs.envs.creators.SimEnvCreator are provided.

from time import sleep

import gymnasium as gym
import numpy as np
from rcs._core.sim import SimConfig
from rcs.camera.sim import SimCameraSet
from rcs.envs.base import (
    CameraSetWrapper,
    ControlMode,
    CoverWrapper,
    GripperWrapper,
    RelativeActionSpace,
    RelativeTo,
    RobotWrapper,
    SimEnv,
)
from rcs.envs.configs import EmptyWorldFR3
from rcs.envs.sim import GripperWrapperSim, RobotSimWrapper

import rcs
from rcs import sim

if __name__ == "__main__":
    # default configs
    scene = EmptyWorldFR3()
    cfg = scene.prefixed_cfg(scene.config())
    fr3 = scene.lead_robot_name(cfg)

    robot_cfg = cfg.robot_cfgs[fr3]
    gripper_cfg = cfg.gripper_cfgs[fr3]  # type: ignore
    camera_cfgs = cfg.camera_cfgs
    sim_cfg = SimConfig(
        realtime=True,
        async_control=True,
        frequency=1,  # in Hz (1 sec delay)
    )
    mjmodel = scene.create_model(cfg)
    kinematic_model_path, attachment_site = scene.kinematics_cfg(cfg)[fr3]

    simulation = sim.Sim(mjmodel, sim_cfg)
    ik = rcs.common.Pin(
        kinematic_model_path,
        attachment_site,
    )

    # base env
    robot = rcs.sim.SimRobot(simulation, ik, robot_cfg)
    env: gym.Env = SimEnv(simulation)
    env = RobotWrapper(env, robot, ControlMode.CARTESIAN_TQuat)

    # gripper
    gripper = sim.SimGripper(simulation, gripper_cfg)
    env = GripperWrapper(env, gripper)

    env = RobotSimWrapper(env)
    env = GripperWrapperSim(env)

    # camera
    camera_set = SimCameraSet(simulation, camera_cfgs, physical_units=True, render_on_demand=True)  # type: ignore
    env = CameraSetWrapper(env, camera_set, include_depth=True)  # type: ignore

    # relative actions bounded by 10cm translation and 10 degree rotation
    env = RelativeActionSpace(env, max_mov=(0.1, np.deg2rad(10)), relative_to=RelativeTo.LAST_STEP)
    env = CoverWrapper(env)

    env.get_wrapper_attr("sim").open_gui()
    # wait for gui to open
    sleep(1)
    env.reset()

    # access low level robot api to get current cartesian position
    print(env.get_wrapper_attr("robot").get_cartesian_position())

    for _ in range(10):
        # move 1cm in x direction (forward) and close gripper
        act = {"tquat": [0.01, 0, 0, 0, 0, 0, 1], "gripper": [0]}
        obs, reward, terminated, truncated, info = env.step(act)
        print(obs)

Note: This and other examples can be found in the examples/ folder.

• Platform support: The core package rcs-core (MuJoCo simulation + Python API) is supported on Linux and macOS (Apple Silicon / arm64). The hardware extensions are Linux only.
• For Python >3.11: The rcs_realsense extension won't work due to the pyrealsense2 version RCS utilizes.
• For Python >3.12: The ompl python module is currently not available on PyPI. If OMPL is not used, it is safe to remove this dependency in pyproject.toml.

pip install rcs-core

Make sure that common build tools (i.e., build-essential), python headers and a C++ compiler like gcc or clang are installed on your system/conda/docker.

RCS works best in Python 3.11, and all extensions have been tested to work in 3.11.

# clone repository
git clone https://github.com/RobotControlStack/robot-control-stack.git
cd robot-control-stack

# setup environment
conda create -n rcs python=3.11
conda activate rcs
conda install -c conda-forge urdfdom urdfdom_headers glfw

# or sudo apt install $(cat debian_deps.txt)
pip install 'pip>=25.1'
pip install --group build_deps

# install rcs
pip install -ve . --no-build-isolation

RCS resolves its asset directory from the RCS_PREFIX environment variable. When it is unset, RCS defaults to ~/.rcs.

On import, RCS checks whether that path exists. If it does not, it downloads the matching asset archive from GitHub into that location automatically.

export RCS_PREFIX=/path/to/rcs-assets

RCS supports various hardware extensions to seamlessly connect your policies to the real world (e.g., FR3, xArm7, YAM, RealSense). These are located in the extensions directory.

Note: Hardware extensions are supported on Linux only. On macOS you can use the core rcs-core package for simulation, but the hardware extensions are not supported.

To install a specific robot extension (example for Franka FR3):

sudo apt install $(cat extensions/rcs_fr3/debian_deps.txt)
pip install rcs-fr3

# or install it locally
pip install -ve extensions/rcs_fr3

For a full list of extensions and detailed documentation, visit robotcontrolstack.org/extensions.

• License error or group argument not found during installation? Make sure you are using a pip version >=25.1 and setuptools version >=45.
• Dependency error during installation? Make sure you are using Python 3.11. RCS extensions currently do not support 3.12+ due to OMPL and RealSense dependencies.
• Simulation is running too slow? Check that you have enable on-demand rendering: SimCameraSet(..., render_on_demand=True) to render camera frames only once per step. Resolution and number of cameras in the scene has a large impact on simulation speed. Make sure to use a decent GPU when rendering is enabled.

For full documentation, including advanced installation, modular usage, and API references, please visit: 👉 robotcontrolstack.org

Useful quick-reference pages:

• RCS Conventions for quaternion order, frames, Euler angles, and gripper semantics
• Sim Scene Configuration for SimEnvCreatorConfig, scene frames, and example setup patterns
• Apps for the teleoperation and inference example entry points
• libfranka Version Info for the currently pinned rcs_fr3 and rcs_panda libfranka versions and local-install guidance

We welcome contributions from the robotics and ML community! For contribution guidelines, please check out robotcontrolstack.org/contributing.

If you find RCS useful for your academic work please consider citing it:

@inproceedings{juelg2026robotcontrolstack,
  title={{Robot Control Stack}: {A} Lean Ecosystem for Robot Learning at Scale}, 
  author={Tobias J{\"u}lg and Pierre Krack and Seongjin Bien and Yannik Blei and Khaled Gamal and Ken Nakahara and Johannes Hechtl and Roberto Calandra and Wolfram Burgard and Florian Walter},
  year={2026},
  booktitle={Proc.~of the IEEE Int.~Conf.~on Robotics \& Automation (ICRA)},
  note={Accepted for publication.}
}

For more scientific information and supplementary videos, visit the paper website.

The RCS source code is licensed under AGPL-3.0. A small subset of redistributed third-party robot and sensor assets under assets/ keeps its original upstream license; the applicable notices are collected in THIRD_PARTY_ASSET_LICENSES.md.