TidyBot - A Cleaning Robot Configured via a LLM
Ai
I’m fascinated by the merger of robotics and LLMs so I was very excited to see the TidyBot project from Stanford, Princeton, Google, Columbia, and Nueva School.
In this work, we investigate personalization of household cleanup with robots that can tidy up rooms by picking up objects and putting them away. A key challenge is determining the proper place to put each object, as people’s preferences can vary greatly depending on personal taste or cultural background. For instance, one person may prefer storing shirts in the drawer, while another may prefer them on the shelf. We aim to build systems that can learn such preferences from just a handful of examples via prior interactions with a particular person. We show that robots can combine language-based planning and perception with the few-shot summarization capabilities of large language models (LLMs) to infer generalized user preferences that are broadly applicable to future interactions.
Find a minute to go watch their videos as seeing this robot flip toys in to a bin has strong WALL-E vibes and you can’t help but love it, but I think this is a exactly the king of thing that will produce real results from the current AI boom. No, LLMs are not artificial intelligence, but they are wonderful tools for translating data in to human language (and vice-versa). Having a robot that cleans up a room is already possible, but it was a dead end unless you can configure it. If LLMs give everyone the ability to configure a robot by just talking to it, that’s a massive leap for robot assistants.
TidyBot - Personalized Robot Assistance with Large Language Models
Related Posts
My Current AI Workflow
Sometimes, after rebutting someone’s extreme exuberance over AI or cracking a joke, I feel like people may assume that I don’t like AI or I never use it. That’s not the case at all! I do use it and I integrate it (or advocate for its use) in my CTO role at Ten2 and when consulting but I do take care to use it in situations that make sense, such as text generation, and not places where it doesn’t make sense, such as decision-making.
Read moreLinkedIn is Going Great
🚨🔥AI will obviously solve everything because hype is never ever wrong. Do you want to know a thing that will help you?! I have an AI thing that does stuff like, I don’t know, let’s say sales or marketing…whatever. It’s amazing though and it won’t be replaced in a matter of days by a new model.
Read moreSoftware Development is Dead, Long Live Developers!
“If you don’t learn to code, in ten years it will be like being illiterate!” That was what someone exclaimed on a panel discussion I was on in 2013. It was a talk about bringing technology and entertainment together held in Beverly Hills and hosted by a coding bootcamp company. Two of the people on the panel were from a different bootcamp company, and then there was me, an actual technologist working in entertainment.
Read more