When the privatechef sent by Shift arrived at my apartmentto cooka three-course meal, he was already ready to record. He walked through my front door wearing a baseball hat with a blocky, white camera attached to the brim.
I signed up for a free lunch in exchange for the whole experience being recorded and uploaded astraining data. As chef Ali unpacked his supplies and started on a gazpacho, the hat cam recorded his hand movements, with the intention that this data could help build abetter robot—one with culinary expertise.
It’s all part of the growing boom ofegocentric data collection—recording human hand movements from a first-person point of view.Shiftmade waves earlier this year by providing free cleaning services in New York City under a similar arrangement: Let Shift record tasks in your home and, in exchange, you get a gratis service. The company expanded in San Francisco by offering free home-cooked meals. If, in the not-so-distant future, I’m able to buy a …
Shift is a division of the larger German startupMicroagi, which focuses on deploying robots in real life. “The goal of Shift is twofold. First, allow people to join the AI economy,” says Microagi CEO Bercan Kilic. “The other part is to handhold the entire society and governments together, from today until the abundance.”
In the short term, Kilic sees this as a way to build a marketplace where contractors can make some cash recording videos and contributing to the improvement ofAI models. Looking at the horizon, he believes impending tech innovations may usher in a new, utopian-esque age ofwidespread wealth.
This year, I’ve spent dozens of hours with aniPhone strapped to my head, attempting to understand this growing avenue of the gig economy. I’ve recorded my hand movements forDoorDashwhile scrambling eggs for breakfast and recorded my fingers tying shoelaces, over and over again, for another data collection startup. Shift offers a similar gig work setup for people who want to record and sell their own egocentric data. While my payouts have been ski…
The uniform for Shift chefs blends elements of a restaurant worker’s outfit with cyborg qualities. Ali, the chef, was wearing a white shirt and pants with a brown apron wrapped around his body. The camera attached to his hat was roughly the size of aGoPro, with a white wire running through the top and down his back. The wire was attached to a smartphone, where the recordings from this free lunch were stored. It jiggled a bit as he cooked.
I’ve never had a private chef before in my cramped apartment, so I felt a bit weird just standing by in the kitchen as he prepared a gazpacho appetizer and started chopping trumpet mushrooms. The finesse and dexterity displayed (and recorded) with his knife skills go far beyond what my rudimentary chops could help train a robot to cook. My sense of awkwardness only increased as I remembered that everything was being recorded.Ishould have scrubbed…
The apartment smelled amazing as Ali got started on the main course, baked salmon with a creamy zucchini sauce, and I began to loosen up. Similar to how I respond when people use Meta’sAI glasses to record in public, I initially froze under the weight of being perceived so directly by technology. But as I chomped on the crispy salmon skin at my dinner table, I considered these recordings from a broader perspective. My kitchen would be just a sing…
Ali finished the meal with a beautifully-piped tiramisu and immediately got started cleaning up the kitchen. What good would a humanoid robot be if it could cook you some Michelin-star-level meal but left you with a pile of crusty pans soaking in the sink? Collecting cleaning data can be just as critical as collecting cooking data.
Kilic says Shift plans to branch out and eventually add other free services, or at least offer them at discounted rates, in exchange for recording the experience. While the boom oflarge language modelswas powered by scraping the internet for text data, no comparable, vast repository of egocentric video data exists for training robots. That’s why the inside of your home, alongside how to navigate it while completing tasks, is so valuable that a co…
He’s optimistic that we’ll soon be living with humanoids. “By this time next year we will have quite good home robots that are at reasonable prices,” he says. “It won’t do all the tasks, but it will do most tasks quite well. It will be likevacuum robotsin 2020.”
I’m not worried about these bots gaining consciousness, but I am anxious about the ramifications ofphysically embodying AI modelsfurther and giving them tools. While a Roomba may smear dog poop all over the carpet when it messes up while you’re at work, a nascent, knife-wielding chef-bot could endanger the dog, or even you, if it glitched out.
I’m also skeptical that these robots, even if they eventually live up to the hype, will truly usher in an age of abundance. Yes, the Industrial Revolution changed aspects of the world for the better, but wealth inequality isworse than ever, and homelessnessremains near record highsin the US. A revolution in robotics has the potential to improve baseline living conditions, without eliminating poverty.
Wealthy early adopters who are willing to spend thousands next year on home bots will have my apartment’s kitchen as well as chef Ali to thank, in small part, for their humanoid-crafted meals. Until then, I have an abundance of leftover tiramisu in the fridge to enjoy.