Revolutionizing Robotics: The Data Challenge at Encord’s San Leandro Warehouse
The frontier of physical AI is a bustling warehouse in San Leandro, California, where innovative startups are tackling the hurdles of training intelligent machines. At the center of this endeavor is Encord, a company dedicated to building data tooling necessary for training AI models. Here, Andrew Ceja, one of the company’s robotic trainers known as pilots, is engaged in a unique task: painstakingly pulling blocks from a precarious Jenga tower while wearing a sophisticated headset. This isn’t just any headset; it also tracks brain activity, offering insights that could redefine how robots learn.
Meeting the Data Demand in Robotics
Encord is one of a select few startups betting that the next major limitation in humanoid and warehouse robotics won’t come from the models themselves but from the availability of real-world physical training data. Many companies have realized that instead of just managing existing data, they must also create new training sets. “The data simply does not exist,” says Vineeth Velmurugan, Encord’s head of robot learning.
The innovative brain wave headset Ceja is using was developed by Zander Labs, a German neuroscience startup focused on understanding mental states such as error, intent, and surprise. This collaboration aims to develop an initial dataset tagged with brain-wave information to enhance AI training models. The results of this trial run will determine if this method leads to better robot performance.
The Bottleneck of Real-World Data
Vineeth Velmurugan, a veteran in robotic development from organizations like OpenAI, highlights the growing challenge: As robotics companies adopt end-to-end learning approaches, the necessity to produce original training data is ever more crucial. Velmurugan explains that self-driving car developers manage to collect physical-world data, but this process is difficult to scale and lacks the fidelity of real-world interactions.
New Data Modalities on the Horizon
To navigate this challenge, companies are looking at two primary sources of data: “egocentric” video from workers wearing cameras and data from remotely operated robots. Encord excels in both these areas, drawing egocentric data from various factories worldwide while also experimenting in its San Leandro facility. Here, they are not just collecting data but are also testing new methodologies, such as measuring brain waves and developing training datasets focused on specific tasks.
During a recent visit from TechCrunch, Ceja and fellow pilot Sofia Infante demonstrated the use of paired robotic arms for tasks like pouring coffee and stacking poker chips. The warehouse also houses a variety of items—from fake flowers to plastic vegetables—used for creating training scenarios for robots aimed at household tasks.
Challenge of Precision Manipulation
Infante engaged robotic arms to perform the intricate task of plugging and unplugging Ethernet cables, a job critical for the automation of data centers. However, her hands-on experience underscores the current limitations; robotic pincers, while effective, lack the dexterity and range of motion that human hands provide.
Another exciting data modality under development at Encord involves sensors strapped to the forearms to detect muscle electrical signals. This innovation can create a 3D model of hand movements, enriching the data contextualness beyond what is typically captured by video. Velmurugan argues that annotations describing actions—like “right hand tightens bolt”—enhance the value of the dataset significantly, making it more beneficial for training specialized models.
Navigating Economic Challenges
Despite these advancements, there are financial realities to consider. While generating physical training data is more costly than simply scraping text from the internet, it’s a necessary investment that sets the stage for developing sophisticated AI. Velmurugan admits that building these data sets can be pricey, costing roughly 20 times more than traditional methods but yielding much greater returns in effectiveness.
Insights from the Front Lines
With their insights from multiple projects and data techniques across the industry, Encord keeps its workforce engaged. Both Infante and Ceja come with valuable backgrounds from other data annotation companies. Ceja, who previously worked in waste management, describes his current role as rewarding and engaging, noting that every day presents a new puzzle to solve in the world of robotic training.
The Future of Robotics Data Generation
Encord’s efforts reflect a shift in how the robotics industry approaches the data challenge. As the need for high-quality, specialized training data grows, companies like Encord are stepping up to not just fill gaps but to create a new framework for machine learning. With the partnership of technology and human ingenuity, the future of physical AI is brighter than ever.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
