Lightweight code-generation agent for X2 robot experiments.

The project connects a natural-language task to a small, sandboxed primitive API, then executes the resulting robot actions through AimDK-style ROS2 messages and Gazebo ros2_control.

• Builds a prompt from a user task and current robot state.
• Calls an OpenAI-compatible LLM to generate Python primitive code.
• Validates the generated code with a restrictive sandbox.
• Executes primitives such as get_tcp_pose, move_tcp, open_gripper, and close_gripper.
• Converts TCP targets to X2 arm joint targets with IK.
• Publishes AimDK-style ROS2 messages and bridges them to Gazebo JointTrajectory commands.
• Saves each trial as JSON and optionally records a Gazebo video.

src/x2_agent_lab/agent/       LLM task loop and primitive registry
src/x2_agent_lab/executor/    generated-code validator and sandbox
src/x2_agent_lab/llm/         OpenAI-compatible client and prompts
src/x2_agent_lab/robots/x2/   X2 runtime, IK, and AimDK ROS2 adapter
src/x2_agent_lab/sim/         Gazebo session and AimDK-to-Gazebo bridge
scripts/                      runnable entrypoints
configs/                      local robot/LLM/safety config stubs
sim/gazebo/                   Gazebo world and controller config

This repository expects the runtime machine to provide:

• Ubuntu with ROS2 Humble
• Gazebo Classic
• X2 AimDK SDK installed locally
• Python 3.10 system ROS environment
• An OpenAI-compatible API key

The local SDK/vendor copies, generated URDF files, logs, videos, and result JSON files are intentionally ignored by git.

The run script can reuse the same key as a Codex terminal if AXONHUB_API_KEY is already exported.

Alternatively, copy the example environment file and set your own values:

cp .env.example .env.local

X2_AGENT_LLM_BASE_URL=https://ai.zxcoding.top/v1
X2_AGENT_LLM_API_KEY=your-real-api-key
X2_AGENT_LLM_MODEL=gpt-5.5

cd /home/xingshu/workspaces/fys/x2-agent-lab

./scripts/run_x2_agent_gazebo.sh \
  "让右臂明显地前后摆动两次，动作幅度要大一些，保持身体和左臂不动。" \
  x2_agent_swing_arm

Outputs are written under:

results/gazebo/x2_agent_swing_arm.json
results/gazebo/x2_agent_swing_arm.mp4

The JSON contains the prompt, raw LLM response, extracted Python code, primitive calls, runtime calls, environment snapshot, and success/failure diagnostics.

See command.txt for the short command used during local testing.

The LLM does not get direct ROS, file, network, or subprocess access. It can only call whitelisted primitives. Real robot execution should remain behind explicit runtime gates and additional safety checks.