Overall architecture

Lumi is being built as a mobile manipulation system, not a single isolated robot arm. The long-term cleanup loop connects perception, task planning, navigation, manipulation, storage, verification, and recovery.

The system is still in supervised development. Lumi does not yet autonomously clean a full home, and object-to-bin cleanup is still being assembled layer by layer.

  1. 01 SEE

    Detect floor objects and estimate where they are in the room.

  2. 02 PLAN

    Choose what to attempt, where it should go, and how to recover if it fails.

  3. 03 MOVE

    Navigate to a safe pickup or bin standoff pose.

  4. 04 PICK

    Position the arm, grasp, lift, and check whether anything was captured.

  5. 05 STORE

    Bring the held object to a bin pose and coordinate placement.

  6. 06 VERIFY

    Confirm the grasp or release before updating task state.

  7. 07 REPEAT OR RETURN HOME

    Select the next object or end the run safely.

This is Lumi's intended long-term loop. Nav2, SLAM, robust floor pickup, and full object-to-bin cleanup are not yet complete.

Hardware foundation

The V1 hardware foundation is organized around onboard compute, real-time control, indoor mobility, sensing, manipulation, power, and safety.

Lumi prototype architecture diagram showing battery, BMS, buck converters, STM32, Jetson, sensors, motor driver, motors, encoders, arm, cameras, lidar, and USB hub connections.
Current Lumi prototype architecture. Purple lines represent power distribution, black lines represent compute and USB connections, orange lines represent sensing and encoder feedback, and green lines represent motor output paths.

Compute and control

  • Jetson Orin Nano
  • STM32G431

Jetson handles higher-level perception, planning, navigation, recording, and learned-policy inference. STM32 keeps time-sensitive motor control, encoder reading, IMU interfaces, watchdog behavior, and low-level safety close to the hardware.

Mobile base

  • Four-wheel differential-drive chassis for V1
  • Wheel encoders
  • Motor drivers

The base is being prepared for slow, supervised indoor motion before autonomous navigation is trusted.

Sensing

  • BNO085/BNO086 IMU
  • RPLIDAR
  • Intel RealSense D435
  • Wrist camera

The sensor stack supports odometry validation, mapping, object detection, and close-range manipulation feedback.

Manipulation and safety

  • Front-mounted SO-ARM follower arm
  • Leader/follower teleoperation
  • Battery
  • Emergency stop

Teleoperation supports supervised testing and demonstration collection. Battery, motor drivers, watchdogs, and emergency stop behavior remain part of the safety foundation.

Software foundation

ROS 2 is becoming the central integration layer for Lumi's robot state, transforms, diagnostics, launch files, and modular capability testing.

The Jetson runs higher-level perception, planning, navigation, recording, and learned-policy inference. The STM32 retains time-sensitive motor control, encoder reading, IMU interfaces, watchdog behavior, and low-level safety.

Direct diagnostic tools remain available during hardware development so the team can validate motors, sensors, and serial interfaces before routing everything through the full autonomy stack.

Operator tools / web panel
Mission executive
ROS 2 integration layer
Perception Camera / lidar
Navigation Odometry / SLAM / Nav2
Manipulation Teleoperation / learned policy
Chassis bridge and safety interfaces
STM32 real-time control
Motors, encoders, IMU, battery, emergency stop

Operator tools / web panel -> mission executive -> ROS 2 integration layer -> perception, navigation, and manipulation -> chassis bridge and safety interfaces -> STM32 real-time control -> motors, encoders, IMU, battery, emergency stop.

Wheel/IMU calibration, ROS 2 integration, URDF/Xacro, lidar integration, and navigation validation are still ongoing.

The autonomy stack

Planning the cleanup loop

Lumi needs to decide which object to address next, whether it is reachable, where to place it, and how to recover from failed attempts.

Read how Lumi plans a cleanup task ->

Preparing the navigation stack

Navigation turns a selected object or bin into a safe destination, using odometry, IMU data, lidar mapping, localization, and Nav2.

Read how Lumi will move through a room ->

Teaching Lumi to pick up from the floor

Manipulation combines teleoperation, camera observations, imitation learning, grasp verification, placement, and recovery behavior.

Read the pickup-learning roadmap ->

Storage is where these three layers meet: the planner chooses the bin, navigation brings Lumi there, and the arm places and verifies the object.

Current status and next milestones

What is taking shape

  • V1 chassis, battery, motor drivers, and emergency-stop foundation
  • STM32 motor-control, encoder, IMU, watchdog, and battery-monitoring interfaces
  • Jetson-based operator tools, recording path, and ROS 2 integration work
  • Leader/follower arm teleoperation for structured demonstration collection
  • Early tabletop pickup and local perception experiments

What comes next

  • Continue wheel encoder and IMU calibration.
  • Validate ROS 2 chassis bridge, TF tree, URDF/Xacro, lidar, EKF, mapping, and Nav2 in supervised stages.
  • Collect structured floor-pickup demonstrations.
  • Train and evaluate a first ACT-style manipulation baseline.
  • Connect planning, navigation, pickup, storage, verification, and recovery into a supervised cleanup loop.

The goal is useful home cleanup behavior, but the current work is still foundation building and validation.