Brick-Composer: Using MLLMs for Assembly with Diverse Bricks
Researchers have developed a new framework called Brick-Composer to enable multimodal large language models (MLLMs) to perform brick assembly tasks. Current state-of-the-art MLLMs struggle with precise brick selection and pose estimation, achieving less than 1% success rate in assembly. Brick-Composer utilizes human design demonstrations, world feedback, and synthetic experience to significantly improve these capabilities, raising step-level assembly success to around 15% and enabling a Qwen-3-8B model to compose up to 42% of assembly steps. AI
IMPACT Enables MLLMs to acquire physical construction skills, potentially leading to more capable AI agents for real-world object assembly.