What planning owns
Planning is neither motor control nor grasping. It decides what should happen next and keeps track of what has already happened.
Planning decides what should happen next. Navigation decides how the base should move. Manipulation decides how the arm should interact with the object.
For storage, planning chooses the bin, stores task state, and decides what happens after success or failure.
Lumi's object registry
Object record
Object status values
- pending
- selected
- navigating
- attempting pickup
- held
- moving to bin
- stored
- failed
- needs review
First planning algorithm
The first planner should be transparent and deterministic, so each target choice can be inspected during supervised testing.
- Build an object list from local perception.
- Estimate object location, confidence, class, and reachability.
- Filter objects that are unsafe, uncertain, outside reach, or blocked.
- Score remaining targets.
- Select the next target.
- Execute the task sequence.
- Update the object registry after success, failure, or interruption.
Initial target score
Adds weight for
- priority
- confidence
- reachability
- likelihood of successful pickup
Subtracts weight for
- navigation travel cost
- obstacle risk
- repeated failed attempts
Target choice
A utility-score heuristic chooses the next useful object to attempt.
Object ordering
A greedy strategy orders objects before more complex multi-object routing is needed.
Execution and recovery
A behavior tree or hierarchical state machine coordinates retries, pauses, and operator-review states.
Cleanup task state machine
- Survey room
- Choose object
- Navigate to standoff pose
- Verify target is still visible
- Attempt pickup
- Verify grasp
- Navigate to bin
- Place object
- Verify release
- Update task list
- Select next object or return home
Object not visible
Rescan or mark for review.
Navigation failed
Try a different standoff pose or defer the object.
Pickup failed
Retry with a bounded number of attempts, then mark failed.
Grasp verification failed
Do not navigate to the bin. Recover or retry before continuing.
Bin unavailable
Pause the task and request operator review.
Local and cloud responsibilities
Local robot responsibilities
Local perception, safety checks, navigation, and arm-control loops must handle time-sensitive decisions.
Possible cloud responsibilities
Cloud AI may later help with high-level scene interpretation or ambiguous cleanup requests, but it must not directly control real-time motion.
Planning roadmap
Phase 1
Single-object cleanup with fixed bin location and transparent rules.
Phase 2
Multiple-object ordering, retries, and object-state tracking.
Phase 3
Room-level cleanup plans, user preferences, and learned prioritization.
Phase 4
Natural-language requests such as 'clear the toys' translated into safe, verified task steps.