Perception — Perceiving the parts in the bin using a sensor like a depth camera that returns RGB-D images. Then an algorithm is used to determine type and pose of the objects within. In some modern systems, you can use the raw RGB-D data to select a grasp instead of relying on the intermediate pose data. Alternatively, some camera systems simply return the pose of parts in the bin.

Grasp Selection & Trajectory Planning — Pick a perceived object in the bin based on instructions from the supervisory logic. The manipulator chooses a valid grasp, picks the part, and plans a collision-free path to bring the objects to a designated placement area.

High-Level Supervisory Logic — A task scheduler that coordinates the different components. It tells the camera when to collect new data, instructs the planning component on what objects to pick and where to place them, and determines when to move the robot, and when to wait for further instruction.

Robot Control and Execution — The trajectory and any end effector instructions are sent to the robot or a robot simulator. This often requires translation into a compatible interface layer such as ROS or a robot-specific language, like UR Script. The robot executes the trajectory and gripper commands, and sensors in the environment return the observed behavior.