Vision-Guided Pick-and-Place with the xArm6

The workshop is structured as six sequential phases, each ending with checkpoints you can verify before moving on. The workshop has two milestones: by the end of Phase 4 you drive the robot from your own code through a static, pre-planned sequence (milestone one), and by the end of Phase 5 you close the loop with live perception so the robot detects, picks, and places blocks on its own (milestone two).

What you’ll build

You will configure an xArm6 robotic arm fitted with a two-finger gripper and a wrist-mounted Intel RealSense depth camera. A shape-detection vision service finds blocks, and the motion service plans collision-free picks that place each block in the bin. By the end, you will have a Python script you run from your laptop and a deployable module that drives the full detect-pick-place loop.

Required hardware

• uFactory xArm6: the six-axis robotic arm that picks and places the blocks.
• Intel RealSense D435: the wrist-mounted depth camera that detects block positions by shape.
• uFactory finger gripper: the end-effector that grasps the blocks.
• System76 Meerkat: the on-robot mini-PC that runs the Viam machine server.

Phases

Phases 1 through 5 are the core workshop. Phase 6 is optional.

1. Platform mental model (~20 min)
2. Configure resources and explore the app (~15 min)
3. Static positions and obstacles (~20 min)
4. Control the robot from Python (~15 min, milestone one)
5. Perception-guided picking (~22 min, milestone two)
6. Inline module (~20 min, optional)

When you finish, the wrap-up reviews what you built and points to next steps.

Companion code

All supporting files for this workshop live in the viam-devrel/pick-and-place repository. You do not need it to start, but the prerequisites below reference its scripts/ project, and later phases pull from it.

• config/ holds a machine config fragment and an obstacles template. Use them to check your work after you configure resources by hand.
• scripts/ holds the starter script for Phase 4 and the reference solution for Phase 5.

Prerequisites

This is a self-serve workshop, so confirm each of the following before you start:

• Python 3.10 or newer. Install it with uv (recommended) or from python.org.
• The Viam Python SDK. The companion scripts/ project already declares viam-sdk, so uv run installs it for you in Phase 4. See the Python SDK docs for reference. Pip works too if you prefer it.
• A working terminal on the machine you will run the Phase 4 and Phase 5 scripts from, typically your personal computer rather than the robot’s Meerkat.
• A Viam account with an online machine. Log in at app.viam.com, create a machine, and confirm the green Live indicator before you begin. “Online” means viam-server is running on the machine and reachable from the Viam app, so the app can deliver config and you can reach it from your own computer; the green Live indicator is how you confirm that connection.

Validate your environment

Before starting Phase 4, confirm your environment is ready:

python3 --version   # 3.10 or newer
uv run --with viam-sdk python -c "import viam; print(viam.__version__)"   # prints a version

If either command fails, revisit the checklist above before continuing.

Where to start

• Hardware ready (viam-server running): start at Phase 1.
• Provisioning your own hardware: complete the hardware setup, then return here for Phase 1.