The billion dollar man steering Toyota into the age of self-driving cars

Dr Gill Pratt is Toyota's billion dollar man. The quietly spoken, unassuming academic is tasked with mapping out a future for the Japanese car giant in the age of auto industry revolution. His job is to not only to steer Toyota into the age of self-driving vehicles, but to hopefully secure it a role in a world where the market will no longer be simply selling mobile metal, but rather selling mobility as a service.

“A lot of the autonomous technology is happening today because of start-ups like Uber and the onset of the mobility as a service market. It’s estimated the size of this market – no matter who provides it – is between five and 10 times as large as the market for personally-owned vehicles,” he says.

Crash statistics are also widely reported in the autonomous debate: over 80 per cent of collisions are attributed to human error. When it comes to safety Pratt’s ambitions are also driven by a higher moral motivation. When he was young a close friend was killed in a car crash. “That burned in my mind that something needed to be done about this. But some people still don’t get it,” he says.

Pratt's Toyota Research Institute (TRI) was started in January 2016 and now boasts 200 highly qualified researchers, with plans to reach 300 in the next year. In addition, it works with at least one university in every state in the United States, along with a global network in Europe and Asia.

Volvo’s Irish sales rise 24.5% due to popularity of XC90 SUV

---

Kia, Jeep and Mercedes criticised over passenger safety

---

What to do on a flooded road

---

The problem it is trying to solve, says Pratt, is that while Toyota “has been better than any manufacturer at practising continuous improvement, the problem occurs when suddenly there is a new optimum, like autonomy or mobility as a service, or alternative fuel sources”.

Explore

“The trouble with continuous improvement is that you then have to take a leap. And the purpose of TRI is to explore spaces so we can tell where and when to take the leap.”

Pratt is well versed in working towards great leaps. From MIT, his primary interest is in the field of robotics and intelligent systems, particularly enhancing human/machine collaboration. Before joining Toyota, he served as a programme manager in the Defense Sciences Office at the highly regarded US Defense Advanced Research Projects Agency (Darpa).

"From my experience at Darpa and other institutions is that very often when you take a leap what happens is failure. It's natural. Failure feels terrible. And that's natural, but then you try again. And you fail again. You see where this is going... If you are lucky and you have the courage to fail again and again and again, you take a leap and you suddenly realise it has worked. TRI's role is to take these leaps for Toyota," says Pratt.

Pratt succinctly sums up the issues facing autonomous technology as perception, planning and prediction. For perception, you need the hardware, like cameras, radar, lidar and the like. Much of the work on camera technology and recognition software is being done in Europe. But this is just the first hurdle.

“The hard part of this is that it’s not about physics, this is about the human mind. It’s about figuring our human behaviour and can we predict what humans are going to do. It turns out recognition of a car’s surroundings is hard, predicting human behaviour, though, is really hard. And that’s because everybody’s different. And so you have not only to collect a lot of data, you also have to understand the variability that’s there and of course people are very complex as well. So that’s another challenging area we are working on.

“Then there is planning. What should we do in our car to work with the predictions we make based on the perceptions we get from recognition of the environment? Planning means you have to follow the many rules of the road in place, and there are very good software systems capable of putting the rules of the road into our AI systems.

Progress

"But it looks like there is also learning to be done there too because if you take the driving manual and you follow those rules precisely, you'll often get into situations where your car will get stuck and not make any progress at all. So how do you figure that out? And learning has a lot to do with it too. Driving in Rome is different than driving in Tokyo or Brussels or the US."

Car firms talk about the five stages of autonomy, with zero having no autonomy and five representing the fully autonomous vehicle with no human interaction. Today there are several cars on sale with level two partial autonomy, where speed and braking is autonomous but steering is assisted rather than completely controlled by the car and the driver is always ultimately in charge. Level three is the next big leap, conditional automation as with the Audi, followed by level four – highly autonomous vehicles where on many routes a human driver isn't required to take control at any time.

The issue for Toyota is that there is a real race under way to get autonomous cars – albeit with limited ability – on to the market. Audi is ready to fit level three self-driving technology – billed as “Traffic Jam Pilot” – to its cars once the legislators allow it. Volvo, promises to have its own self-driving system as an option on its cars by 2021. Others aren’t far behind.

Toyota promises what it calls Highway Teammate by 2020 (limited autonomous driving on motorways) and Urban Teammate (limited urban autonomous driving) early in the next decade.

In terms of timeframe, Pratt sees us being able to reach level three and four by the middle of the next decade. As for the arrival of vehicles without any human controls at all, Pratt says that is too far away to predict. “I would challenge anyone in the automated driving field to give a rational basis for when level 5 will be available.”

He says that some driving scenarios are extremely difficult to figure out and “they require our reasoning, and so far no one knows how to build an AI system that has human reasoning the way we can use it. And so [at TRI] think that some situations for level 5 are extremely difficult and we don’t know how to solve.”

Situational awareness

So for the foreseeable future drivers will have to be ready to take back control in certain situations. “The problem is that car firms are not agreed how much notice a driver must have. It’s not defined,” Pratt explains.

“Part of our concern, is that whatever number you use, however many seconds you say is required for a driver to reacquaint with situational awareness and safely go back to driving, it must be that the car can safely see and predict that far into the future. And, of course, the longer that time is the more difficult it is for the ‘crystal ball’ in the car to predict. On the highway the car travels a long distance in a given length of time and so how far in the distance does it need to see in order to allow the driver that many seconds before taking back control. In the city so many things can change in that time, how do you predict what’s going to happen?”

As to public resistance to travelling in such vehicles – particularly the ageing society – Pratt says: “I would refer you to the history of elevators. It used to be that there was a man in the elevator who operated it and gradually the automatic elevator replaced them. I do believe that there will be fear. But my suspicion is that people will adapt very quickly and the ageing society will adapt quickest of all because for them the freedom you get, first of all from a car that keeps you safe if your reaction times are not so good as when you were younger, will be regarded as wonderful.”

Pratt may have the power to spend big, but his billion dollar budget will be minuscule compared to the socio-economic impact of the changes that will come if and when his aspirations become reality. As he says, we are living through a revolution and those who get the innovation right could rewrite the rulebook and not just on the road.