The researchers additionally expanded the system’s talents by including texture and shade choices. For instance, utilizing an look immediate like “Electrical guitar in metallic purple,” LegoGPT can generate a guitar mannequin, with bricks assigned a purple shade.
Testing with robots and people
To show their designs labored in actual life, the researchers had robots assemble the AI-created Lego fashions. They used a dual-robot arm system with drive sensors to select up and place bricks in keeping with the AI-generated directions.
Human testers additionally constructed a few of the designs by hand, displaying that the AI creates genuinely buildable fashions. “Our experiments present that LegoGPT produces secure, numerous, and aesthetically pleasing Lego designs that align intently with the enter textual content prompts,” the workforce famous in its paper.
When examined in opposition to different AI programs for 3D creation, LegoGPT stands out by means of its deal with structural integrity. The workforce examined in opposition to a number of alternate options, together with LLaMA-Mesh and different 3D-generation fashions, and located its strategy produced the very best proportion of secure constructions.
A video of two robotic arms constructing a LegoGPT creation, supplied by the researchers.
Nonetheless, there are some limitations. The present model of LegoGPT solely works inside a 20×20×20 constructing house and makes use of a mere eight customary brick varieties. “Our technique at present helps a hard and fast set of generally used Lego bricks,” the workforce acknowledged. “In future work, we plan to increase the brick library to incorporate a broader vary of dimensions and brick varieties, comparable to slopes and tiles.”
The researchers additionally hope to scale up their coaching dataset to incorporate extra objects than the 21 classes at present accessible. In the meantime, others can actually construct on their work—the researchers launched their dataset, code, and fashions on their project website and GitHub.
