TidyBot - A Cleaning Robot Configured via a LLM
Ai
I’m fascinated by the merger of robotics and LLMs so I was very excited to see the TidyBot project from Stanford, Princeton, Google, Columbia, and Nueva School.
In this work, we investigate personalization of household cleanup with robots that can tidy up rooms by picking up objects and putting them away. A key challenge is determining the proper place to put each object, as people’s preferences can vary greatly depending on personal taste or cultural background. For instance, one person may prefer storing shirts in the drawer, while another may prefer them on the shelf. We aim to build systems that can learn such preferences from just a handful of examples via prior interactions with a particular person. We show that robots can combine language-based planning and perception with the few-shot summarization capabilities of large language models (LLMs) to infer generalized user preferences that are broadly applicable to future interactions.
Find a minute to go watch their videos as seeing this robot flip toys in to a bin has strong WALL-E vibes and you can’t help but love it, but I think this is a exactly the king of thing that will produce real results from the current AI boom. No, LLMs are not artificial intelligence, but they are wonderful tools for translating data in to human language (and vice-versa). Having a robot that cleans up a room is already possible, but it was a dead end unless you can configure it. If LLMs give everyone the ability to configure a robot by just talking to it, that’s a massive leap for robot assistants.
TidyBot - Personalized Robot Assistance with Large Language Models
Related Posts
AI Hype Comes Back Down to Earth
AI’s hype has been out of control for a long time now. Truly. It’s a shame too, because there’s interesting and useful functionality here. This isn’t the crypto boom which only birthed NFTs and accelerated ransomware, there are real benefits here, but none of this was even close to “AGI” (or even real “AI”) so there was a segment of the technology industry that sat and waited for days like the last few to come.
Read moreHacking the STLToday Paywall v2025
Welcome back to our continuing series of breaking new STLToday paywall versions. Want to hear something crazy? We’ve been doing this for over 10 years now! Or to put it another way, STLToday.com, the website for the St. Louis Post Dispatch, has been using a paywall for over a decade, and they haven’t been able to make one that isn’t cracked in a matter of minutes.
Read moreMy Current AI Workflow
Sometimes, after rebutting someone’s extreme exuberance over AI or cracking a joke, I feel like people may assume that I don’t like AI or I never use it. That’s not the case at all! I do use it and I integrate it (or advocate for its use) in my CTO role at Ten2 and when consulting but I do take care to use it in situations that make sense, such as text generation, and not places where it doesn’t make sense, such as decision-making.
Read more