![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Computation VS Understanding, OR, Why Don't We Have Dependable Autonomous Cars?
Another lecture of mine, this one more technological in nature...
Tesla is far from the only company to be trying to make autonomous vehicles. This has been a major focus of the DOD and their associated contractors for years (the DARPA Challenge, etc.). The company I've worked with for over two decades has designed and built more than one autonomous system, and I'm very familiar with their capabilities -- and limitations -- and how they've changed over the years.
In some ways, immense strides have been made in making autonomous vehicles. We've taken cues from nature and physics and computer science to construct systems that can be very quick, very accurate, and have far wider situational awareness than a human being.
"Situational Awareness" is the ability of a person or system to recognize all the relevant elements of their location and position, ranging from inert physical obstacles to potential threats or hostile actors.
Human beings primarily use sight, secondarily sound, for this purpose, with the other senses having tertiary input; Sight gives us detailed understanding, in sufficient lighting, in a roughly 50-60 degree field of view, more if we move the eyes and head, but without turning completely around we can't really exceed 180 degrees and even that's a periodic scan, not a constant awareness. Sound is generally omnidirectional, but we're not terribly good at getting the direction of a particular sound -- and the contours of the local area can make it even more difficult. We may occasionally get a cue from smell, vibration, heat senses, and so on, but that's much less common.
An autonomous vehicle, properly designed, can maintain a 360-degree scan of the environment at a much higher rate than a human being could manage. It can use lower-light vision, or infrared, or LIDAR, millimeter wave radar, or other means for detecting objects. This can be coupled with automatic object detection and extraction, and with physics of motion and knowledge of the vehicle's operating capabilities, to allow an extremely fast and accurate control of the vehicle. Autonomous vehicles can respond to a change in the environment in milliseconds, when a human may require well over a second depending on exactly what the change is. A group of autonomous vehicles following each other ("platooning") can follow at distances utterly unsafe for human drivers, and never crash when the front vehicle suddenly begins braking.
Despite these obvious advantages, no autonomous vehicle yet can equal the ability of a human being to address all the challenges of uncontrolled driving -- going from one place to another and addressing all threats to the vehicle, not merely those of other vehicles movement but of unsecured cargo, unstable wheel behavior, sudden appearance of children after a small animal or ball, etc.
This is, to some people, surprising, especially since we KNOW that human beings actually SUCK at attentiveness -- and get worse, the longer they go without incident, sometimes literally falling asleep while driving. Humans' reaction time is pathetic compared to a machine, and while the raw "bandwidth" -- amount of data coming into a human being for processing -- is quite large, in reality we dispose of almost ALL of that data before it ever gets to our conscious mind, sorting it out by mostly automatic and unconscious processes so that we can manage to make sense out of it in the sluggish, crawling fractions of a second we have to think in.
The difference is, simply put, UNDERSTANDING. We construct, and update, a MODEL of the world in our heads -- sometimes consciously, often subconsciously -- which provides us with almost innumerable expectations of what we will encounter next. We can do this because we have internalized and extracted a comprehensive description of how the world works -- not merely the physics, but the expectations, the effects of people, the behaviors of animals, the assumptions of our society, and so on and so forth.
A modern computer -- even the current AIs -- DOES NOT do this. You can explicitly program in a physics model (and some of the more impressive video games do this to a fine degree). You can train a computer to recognize specific situations and behaviors. But we cannot, as of now, show a computer how to connect all of the training and averaged values and weighted networks and explicit physics into a comprehensive and sensible, real-time model of the world around the computer.
This isn't surprising, since we're not really quite sure how we do it ourselves. But this is part of the core essence of "understanding", of being able to extract true meaning out of simple data.
A real-life example I like to use of a situation that even the best current autonomous vehicle could not handle, but that a human being -- me, specifically -- could and did, is a driving experience I had many years ago, in which I was following a large pickup truck. This truck was for a gardening and landscaping service, and to advertise this it had a garden-shedlike structure on the truck bed, made primarily of wood. As I rode behind this vehicle, I noticed the shed was vibrating in the postholes that held it to the truck.
While I had no knowledge of the exact way in which the shed had been built, or the methods used to fasten it, or how long it had been in place (judging by wood coloration and weathering, probably many months, perhaps years), my brain immediately said "something about that doesn't look right", and I could immediately envision what might happen to me if that thing came loose. So I slowed down, increasing the distance between us to something over fifty yards -- and then the shed leapt clear off the truck and came smashing down exactly where I'd been following a moment before.
You could PROGRAM an autonomous vehicle to recognize this exact scenario and act upon it -- but that would not give it the ability to recognize ANOTHER situation I encountered, in which I saw a truck's tire ahead of me wobbling in a very peculiar way, and swerved over two lanes just before the tread ripped off.
Our current autonomous systems do not understand the environment, and because of this they can NEVER actually attain parity with human beings in unusual circumstances. In USUAL circumstances, in ones in which the movement of the vehicles themselves is the only concern, or the direct issue of a human being stepping into the road, these they can, and when designed properly should, be able to outperform human beings. But the NUMBER of "unusual circumstances" is very nearly unbounded, and because of this the approach of "program it to deal with that specific problem" will do very, very little to reduce the number of times an autonomous vehicle makes the wrong choice -- or fails to choose, as it was unable to understand there WAS a problem.
This is, of course, separate from issues of sensory selection, resolution, and so on. The recent test in which a Tesla crashed itself into a painting of a road illustrates the problem of relying on pure vision without understanding. Such a painting *COULD* fool a human being, sometimes, but there's numerous cues that might cause the human to slow down before he or she even understood what they were seeing.
A machine, however, can be given better senses; for instance, just using dual-mode fusion -- visible light AND thermal infrared -- would have made such a collision essentially impossible, because the infrared pattern of a wall would not, in any way, resemble that of an open road, and certainly wouldn't match with what was seen in the visible spectrum. A direct millimeter-wave radar scan or similar active approach would instantly warn the car of an obstacle, regardless of what was seen -- although THERE you need alternative sensors so that you don't treat a spray of water as a solid object.
And another -- crucial, and even more difficult question -- is how you give a machine the understanding of RESPONSIBILITY -- something at least emulating a conscience and an awareness that its choices can't be made purely for the sake of the car it pilots.
Tesla is far from the only company to be trying to make autonomous vehicles. This has been a major focus of the DOD and their associated contractors for years (the DARPA Challenge, etc.). The company I've worked with for over two decades has designed and built more than one autonomous system, and I'm very familiar with their capabilities -- and limitations -- and how they've changed over the years.
In some ways, immense strides have been made in making autonomous vehicles. We've taken cues from nature and physics and computer science to construct systems that can be very quick, very accurate, and have far wider situational awareness than a human being.
"Situational Awareness" is the ability of a person or system to recognize all the relevant elements of their location and position, ranging from inert physical obstacles to potential threats or hostile actors.
Human beings primarily use sight, secondarily sound, for this purpose, with the other senses having tertiary input; Sight gives us detailed understanding, in sufficient lighting, in a roughly 50-60 degree field of view, more if we move the eyes and head, but without turning completely around we can't really exceed 180 degrees and even that's a periodic scan, not a constant awareness. Sound is generally omnidirectional, but we're not terribly good at getting the direction of a particular sound -- and the contours of the local area can make it even more difficult. We may occasionally get a cue from smell, vibration, heat senses, and so on, but that's much less common.
An autonomous vehicle, properly designed, can maintain a 360-degree scan of the environment at a much higher rate than a human being could manage. It can use lower-light vision, or infrared, or LIDAR, millimeter wave radar, or other means for detecting objects. This can be coupled with automatic object detection and extraction, and with physics of motion and knowledge of the vehicle's operating capabilities, to allow an extremely fast and accurate control of the vehicle. Autonomous vehicles can respond to a change in the environment in milliseconds, when a human may require well over a second depending on exactly what the change is. A group of autonomous vehicles following each other ("platooning") can follow at distances utterly unsafe for human drivers, and never crash when the front vehicle suddenly begins braking.
Despite these obvious advantages, no autonomous vehicle yet can equal the ability of a human being to address all the challenges of uncontrolled driving -- going from one place to another and addressing all threats to the vehicle, not merely those of other vehicles movement but of unsecured cargo, unstable wheel behavior, sudden appearance of children after a small animal or ball, etc.
This is, to some people, surprising, especially since we KNOW that human beings actually SUCK at attentiveness -- and get worse, the longer they go without incident, sometimes literally falling asleep while driving. Humans' reaction time is pathetic compared to a machine, and while the raw "bandwidth" -- amount of data coming into a human being for processing -- is quite large, in reality we dispose of almost ALL of that data before it ever gets to our conscious mind, sorting it out by mostly automatic and unconscious processes so that we can manage to make sense out of it in the sluggish, crawling fractions of a second we have to think in.
The difference is, simply put, UNDERSTANDING. We construct, and update, a MODEL of the world in our heads -- sometimes consciously, often subconsciously -- which provides us with almost innumerable expectations of what we will encounter next. We can do this because we have internalized and extracted a comprehensive description of how the world works -- not merely the physics, but the expectations, the effects of people, the behaviors of animals, the assumptions of our society, and so on and so forth.
A modern computer -- even the current AIs -- DOES NOT do this. You can explicitly program in a physics model (and some of the more impressive video games do this to a fine degree). You can train a computer to recognize specific situations and behaviors. But we cannot, as of now, show a computer how to connect all of the training and averaged values and weighted networks and explicit physics into a comprehensive and sensible, real-time model of the world around the computer.
This isn't surprising, since we're not really quite sure how we do it ourselves. But this is part of the core essence of "understanding", of being able to extract true meaning out of simple data.
A real-life example I like to use of a situation that even the best current autonomous vehicle could not handle, but that a human being -- me, specifically -- could and did, is a driving experience I had many years ago, in which I was following a large pickup truck. This truck was for a gardening and landscaping service, and to advertise this it had a garden-shedlike structure on the truck bed, made primarily of wood. As I rode behind this vehicle, I noticed the shed was vibrating in the postholes that held it to the truck.
While I had no knowledge of the exact way in which the shed had been built, or the methods used to fasten it, or how long it had been in place (judging by wood coloration and weathering, probably many months, perhaps years), my brain immediately said "something about that doesn't look right", and I could immediately envision what might happen to me if that thing came loose. So I slowed down, increasing the distance between us to something over fifty yards -- and then the shed leapt clear off the truck and came smashing down exactly where I'd been following a moment before.
You could PROGRAM an autonomous vehicle to recognize this exact scenario and act upon it -- but that would not give it the ability to recognize ANOTHER situation I encountered, in which I saw a truck's tire ahead of me wobbling in a very peculiar way, and swerved over two lanes just before the tread ripped off.
Our current autonomous systems do not understand the environment, and because of this they can NEVER actually attain parity with human beings in unusual circumstances. In USUAL circumstances, in ones in which the movement of the vehicles themselves is the only concern, or the direct issue of a human being stepping into the road, these they can, and when designed properly should, be able to outperform human beings. But the NUMBER of "unusual circumstances" is very nearly unbounded, and because of this the approach of "program it to deal with that specific problem" will do very, very little to reduce the number of times an autonomous vehicle makes the wrong choice -- or fails to choose, as it was unable to understand there WAS a problem.
This is, of course, separate from issues of sensory selection, resolution, and so on. The recent test in which a Tesla crashed itself into a painting of a road illustrates the problem of relying on pure vision without understanding. Such a painting *COULD* fool a human being, sometimes, but there's numerous cues that might cause the human to slow down before he or she even understood what they were seeing.
A machine, however, can be given better senses; for instance, just using dual-mode fusion -- visible light AND thermal infrared -- would have made such a collision essentially impossible, because the infrared pattern of a wall would not, in any way, resemble that of an open road, and certainly wouldn't match with what was seen in the visible spectrum. A direct millimeter-wave radar scan or similar active approach would instantly warn the car of an obstacle, regardless of what was seen -- although THERE you need alternative sensors so that you don't treat a spray of water as a solid object.
And another -- crucial, and even more difficult question -- is how you give a machine the understanding of RESPONSIBILITY -- something at least emulating a conscience and an awareness that its choices can't be made purely for the sake of the car it pilots.
no subject
Like protesters placing traffic cones on the hoods of Cruise taxis to disable said taxis.
no subject
The problem there is, again, the size of the number of situations you have to deal with.
And if you could give the AI actual understanding, all those problems go away. Unfortunately, we don't quite understand understanding.
no subject
no subject
An actual intelligence can recognize "this simply is wrong, though I have no specific knowledge of this example of wrong, and I'm going to treat it as wrong and take action." A programmed one can't do that.
no subject
no subject
no subject
no subject
Personally, I want the car to give me warnings of things (especially stuff I can't see myself, like in fog), and to take action if things go very wrong (like stopping if a car in front of me stops suddenly).
no subject
And what you're describing is more additional safety from automated systems, a semi-autonomous mode in which you're still doing the actual thinking. This is a common mode of operation for drones and other robotic systems, but it is, of course, not actual autonomous functioning.
no subject
no subject
no subject
I think for me, I would change that something like better on reasonable balance, rather than all. That is, there might be reasonable tradeoffs, where it's better in some situations and worse in others.
no subject
no subject
I also don't want a false sense safety. Over a decade ago I got to ride in a Google self-driving car for city driving. It felt like it knew what it was doing. So I was not at all surprised to hear that testers were doing things like watching movies, doing their nails, or even napping.
I think there have been stories of drivers bullying self-driving cars. Like, oh, it's fine to cut in front, it will know to slow down. So if they are in the mix with human drivers, I very much want confidence that will be able to handle such aggressive behavior.
no subject
I want it good enough that if I see what happened I could say "Ouch, tough luck, Robo, but I couldn't have avoided that myself either."
no subject
Still, if you're at the 98th percentile, by definition at least half of drivers are at the 50th or lower....
no subject
no subject
no subject