Salt-Peanuts #2
Both current AI and the current response to climate change are dominated by words. I'm interested in actions.
From words to action
A few lines in Faust have inspired me
Scene 3 (The Study) begins with “In the beginning was the Word” and traverses through, senses, and power and ends with "In the beginning was the Deed!."
My world shifted on November 30, 2022 when I first experimented with ChatGPT. It was far more than I expected from a language model.
Yet, that passage from Faust came to mind immediately.
I asked: “When will we realize intelligence is about solving problems through actions? When will we realize that human intelligence is embodied intelligence?
Learn Through Play
I think one of the reasons young humans learn so much will with so little training is play. Play is fun and play lets you focus attention. Perhaps “Attention is all you need” is an overstatement, but it is a rich idea.
When I accidentally learned that Nvidia was using Minecraft to study autonomous agents, and
Remembered how that world class Go masters were learning from Alpha Zeros’ game play
I decided to watch many presentations at Nvidia’s GTC 2023 conference. I was blown away by the implications for climate change. One of them was from Sergey Levine an Associate professor in the Department of Electrical Engineering and Computer Sciences at UC Berkeley.
Sergey Levine’s GTC presentation knocked my socks off
Since I’m typically barefoot, that is no mean feat.
He asks: “Why do we need machine learning?”
He asks: “We can ask an even more basic question why do we need brains?”
He says: “We can ask a fellow who knows quite a lot about brains — neuroscientist Daniel Wolpert — and he has this to say on the topic:”
“We have a brain for one reason and one reason only and that's to produce adaptable and complex movements. Movement is the only way we have affecting the world around us. I believe that to understand movement is to understand the whole brain.”
Most of Levine’s presentation at GTC is technical and may not be accessible to some readers, but I recommend the opening section to everyone. It is about what AI needs to do, not about how it works.
Towards General-Purpose Models for All Robots
When I think of a robot, I think of a great deal of special purpose hardware and software. That is not crazy, but it is a 20th Century view, not a 21st Century view. The foundation models like Claude 2, BERT, and GPT4 get their core power by digesting terabytes of data.
Can massive datasets serve a similar function for Robots?
Can that provide a foundation for transferring learning across a community of robots?
Evidently the answer is yes. Last week DeepMind announced Open X-Embodiment, new set of resources for general-purpose robotics learning across different robot types, or embodiments.
The work was done in collaboration with 33 research labs and involved 22 robot types. The key paper is available GitHub. This video is a quick introduction to the research and results.
The paper shows training a single model on data from multiple embodiments leads to significantly better performance across many robots than those trained on data from individual embodiments.
500 skills, 150,000 tasks, 1,000,000+ workflows
Learning about the world, from the world
Learning about the world by hoovering up much of the internet has produced some startling results. Yet, the indirecion can be misleading and perhaps worse. Robots with rich facilities for observation (i.e. better senses than humans) can act on those observations as well as provide unprecedented information.
As robot development is accelerated through Open X-Embodiment, progress toward autonomous, general purpose robots, as described in this presentation by Levine, will present opportunities in the many tasks that will be required to slow down and mitigate climate change.
We will need to make a transition from
governing the relatively contained world of language models to
governing a world with robots that can, potentially, learn in real time from the behavior of other robots
Autonomy is not something that will be limited to robots. Especially in military systems, components are moving toward more and more autonomy. This video presents issues that need to be dealt with today. Can current discussions between Senators+WH and the big AI companies prepare us to face those issues? I don’t think so.
Ideas and progress are generated by intersections
Many humans are not healthy. The intersection of medical, public health, and economics disciplines is required for good progress.
Most education is based on ideas from 100s or 1000s of years ago. Often the goal has not been knowledge, but a desire to maintain order and control. The intersection of 21 Century understanding of human bodies, minds, and emotions with educational systems is a foundation for better progress.
All across the planet, we produce amazing things. We do it in a way that is destroying the planet. We need a social/technical science of production and supply chains. It will intersect with diverse general purpose technologies as well as new tightly focused technology.
To what extend do new tools offer solutions and to what extend do they compound the problems?
Supply chains
Country home and grounds of one of Italy’s historic rice growers taken in the fall of 2022 near Bologna. It is now a museum with an imprssive display of the tools
Thanks to a friend in Milan, I can share music to accompany this image. In my About page I say this blog is about contending forces. Please try to join the beauty of the image with the horrors —poverty, leaches, rotting feet, and misogyny — facing the women whose work in the rice fields that made it possible.
The Atlas of AI—Kate Crawford Interview
I had been concerned about the atoms involved in 21st Century Compute and AI when I heard of Kate’s book. It fleshed out those concerns, provided facts, and included the impact on people and culture. Dylan and Jess of the Radical AI Podcast interviewed her about the book on Youtube. Here are a few selected quotes from the transcript.
“…the idea behind this book was really to explore how artificial intelligence is made and i mean that in its widest possible sense essentially thinking about the economic political cultural and historical forces that shape what we call artificial intelligence.”
“…in the five plus years of writing this book that i really had to dive deeper into ai at a much bigger scope and ultimately explore the materiality of how artificial intelligence is made and the reason i like this metaphor of an atlas is that i think atlases are very unusual books they they help us look at things at different scales…”
“…you find that it's neither artificial or intelligent it's it's both embodied and material it's made from these natural resources human labor logistics and histories and classifications…”