An interesting Twitter thread sprung up recently discussing the merits of Tesla’s AutoPilot partially automated driving system relative to competitors. This came in the wake of a preliminary report from the National Transportation Safety Board that was examining a March 2018 crash that killed an Apple engineer in a Tesla Model X. One of the commenters raised the concept of which automated driving systems are most developed, citing how far back Tesla ranked in Navigant Research’s Automated Driving Leaderboard report.

How to Interpret Leaderboard Rankings

While it is true that Tesla had the lowest score in this year’s ranking, it’s important to understand both what is being ranked and how most developed may relate to best developed for automated driving.

This Navigant Research Leaderboard is intended to provide a snapshot of where analysts believe companies rank in their ability to successfully commercialize automated driving technology. To do that, the Leaderboard goes well beyond just the core automation technology, with individual subscores assigned for 10 different criteria including vision, go-to market strategies, partnerships, manufacturing capability, product quality, and financial strength. While Tesla has always ranked high on vision, it often lags in many other areas, leading to its low overall score.

Scoring for Vision

The vision score is based in part on where a company sees its business going in the coming decade as well as what it hopes to achieve. In Tesla’s case, its score was boosted by its belief that automated driving technology should be deployed as far and wide as possible as soon as possible.

That is a fantastic idea. In an ideal world, it would save many of the tens of thousands of lives lost annually in American traffic accidents and the more than one million lost globally. Unfortunately, we don’t live in an ideal world. Putting insufficiently developed technology in the hands of untrained consumers on public roads can be a recipe for disaster.

The Gap between Most and Best

AutoPilot may indeed be the most developed system on the market today in terms of the manufacturer’s willingness to stretch its capabilities and extract all it can from the available sensor suite. That does not necessarily make it the best developed system in terms of what should be in production based on safety requirements and consumers’ current abilities and understanding of the technology.

Humans Like Consistency

Early in my engineering career, working on anti-lock braking systems, I learned the value of ensuring that the technology performed with as much consistency and predictability as possible. For all our flaws, humans are remarkably adaptable if we understand the conditions. Most people tend to drive the same vehicle every day for years. Even if a vehicle has limitations or quirks, as long as they are consistent, drivers will adjust how they use the vehicle. If a vehicle has a longer stopping distance, drivers will brake sooner. If the steering response is slower, a larger angle will be used.

The problem comes when the system responds differently every time you use it, as many voice control systems have done. That’s when people stop using it or get caught unawares when they do. That likely contributed to Walter Huang ignoring warnings to hold the steering wheel in his Tesla. There are other systems on the market, that while not as capable as they could be in certain conditions, are indeed better developed for the real world.

Acknowledging Room for Improvement

As we move to higher levels of automated driving in the next several years, we need to encourage manufacturers to acknowledge what their products can’t do yet, while working to make available functions the best they can be.

If we’ve learned anything from the era of reality television and user-generated online video, it’s that a surprising number of people will risk great harm by misusing themselves or technology to get some online attention. Whether it’s blowing up a microwave, eating laundry detergent pods, or misusing driver assist features on a car, too many are willing to abandon common sense in search of the dopamine hit that comes with seeing the number of views ratchet higher. I shake my head in bewilderment when I hear of someone swallowing a detergent pod, but at least they are not putting others in harm’s way.

Vehicle Travel Should Be Serious

More concerning is seeing videos of people using today’s vehicle partial automation systems, like Tesla AutoPilot, beyond the scope of its capabilities or trying to figure out how to trick it into functioning as a more highly automated system. I have no issues with hardware hacking of stationary devices, or vehicle systems not related to driving. Repurposing hardware you have purchased to provide added functionality can be fun, educational, and allows you to extract more value from it.

But modifying or tricking a vehicle’s guidance system puts innocent bystanders at risk, with potentially disastrous consequences. People who override driver assistance systems or pay little attention to the vehicle’s operation could negatively affect the adoption of automated vehicles.

Consumers Shouldn’t Overestimate Vehicle Autopilot

Tesla AutoPilot and similar systems from General Motors, Volvo, Mercedes-Benz, Nissan, and others are not automated driving systems. Except for GM’s SuperCruise, none of these systems are reliably able to hold a vehicle in lane to the degree of hands-off functionality. All of the driving systems, including GM’s, require the driver to remain engaged with eyes on the road and ready to take over.

Overconfident Users Are Misusing Existing Automated Capabilities

While Tesla CEO Elon Musk often talks about software updates that will give AutoPilot full self-driving capability, that day has not arrived and may never be here with the current generation of hardware. Despite the well-known flaws and limitations of AutoPilot, Tesla owners continue to ignore warnings from the system and the company, using the system in ways or in places where it should be disabled. One owner that has posted dozens of videos to YouTube recently tried to demonstrate that stuffing oranges between the steering wheel rim and spokes could fool the system into thinking the driver’s hands were on the wheel. Had this been done on a closed track, it might have been an interesting stunt. On a public road, with other vehicles around, this was downright reckless.

An Apple engineer recently died when his Tesla was on AutoPilot mode and ran into a highway barrier in California. While the system clearly failed to hold the vehicle in the lane, this driver had previously complained about the car exhibiting the same bad behavior to Tesla service. Since the accident, several other Tesla owners have replicated the situation while recording video with a hand-held phone, risking further injuries.

A pedestrian was killed by an Uber autonomous test vehicle in another instance of a driver not paying attention as instructed and pushing the technology beyond its limits. Automakers need to continue clarifying the vast differences between the driver assist technologies of today and the driver not needed technologies of tomorrow.

Holding out Hope for Progress

A number of studies have already shown that a majority of people don’t trust automated driving systems. Automation has the potential to provide enormous societal benefits by saving lives and damage to property. However, if the actions of those looking for views erode public trust in the technology even as it improves, those benefits may remain off in the horizon.

For many of us that keep tabs on the automotive industry for a living, the first 2 weeks of January are among the most grueling of the year. The North American International Auto Show in Detroit has kicked off the year for several decades. And in the past 10 years, International CES in Las Vegas has become an increasingly important addition to our schedule as the two events run back to back. The announcements at 2018’s shows illustrated some of the crucial interconnections between the growth of technology and the transportation business.

For automakers, CES has largely been a place where they talk about future technologies and try to shift the media’s perception of them from being old-fashioned metal benders to forward-thinking visionaries. They rarely show actual new products, instead focusing on automated and connected concept vehicles. The Detroit show, like most other auto shows, targets consumers that are buying vehicles in the coming year.

For an industry that is facing the biggest transformation in more than 100 years, this is a crucial time. While many recent auto shows have highlighted new plug-in and hybrid vehicles, there were almost none in Detroit this year. Instead, the biggest announcements came from the Detroit-area manufacturers, and they were all pickup trucks—mostly full-size. Fiat Chrysler unveiled the redesigned 2019 Ram 1500. Chevrolet brought out a new from the ground up Silverado, and Ford launched a diesel version of the F-150 and a midsize Ranger pickup.

Profit in Pickups

Pickups are a segment that is likely to be among the last to gain highly automated driving capabilities, as discussed in Navigant Research’s Market Data: Automated Driving Vehicles forecast and its Leaderboard reports. However, those automation technologies were a major topic of conversation in Las Vegas, particularly in the context of whether manufacturers will build new business models around these costly, complicated, support-intensive vehicles.

That’s why pickups are so important to Detroit. They are the profit engines that keep this industry humming along while indirectly funding R&D efforts that will create the next big things. Part of why Ford is bringing the Ranger back to North America is that the average selling price of an F-150 is now more than $58,000. Pickups and large SUVs generate far more profit per vehicle than any small car and they sell in far larger volumes than any other segment in the American market. Ford is projected to make a full-year 2017 profit of more than $9 billion, largely thanks to sales of nearly 900,000 F-series trucks. Even the third place Fiat Chrysler sold more than 500,000 Ram pickups in 2017.

All three manufacturers are adopting fuel efficiency technologies such as 48 V mild-hybrids, dynamic cylinder deactivation, diesel and active aerodynamics in order to meet fuel economy requirements, as discussed in Navigant Research’s Automotive Fuel Efficiency Strategies report. However, until they all figure out how to make sustainable profits in the new age of mobility, we can rest assured that they will continue pressing ahead with enhancing the customer appeal of these trucks in order to keep the cash flowing to develop the promises made at CES.

Let’s be absolutely clear about something. As I write these words in October 2017, there are exactly zero self-driving vehicles available for consumers to purchase in America. In fact, Elon Musk’s proclamations and pre-sales of non-existent technology aside, it will likely be at least several more years before an individual can buy a self-driving vehicle. With that in mind, the media needs to stop using the term self-driving in the context of any production vehicle.

Misleading Headlines

In recent weeks, Cadillac has conducted a cross-country media preview of the 2018 CT6 sedan with the first production application of its Super Cruise system. From the event launch in Manhattan to its arrival in Los Angeles 2 weeks later, media outlets including NBC’s Today show, USA Today, Business Insider, and Fortune have referred to Super Cruise as self-driving. In doing so, they are doing a disservice to their own credibility, to consumers, to General Motors (GM), and to every engineer working on automated driving technology.

To its credit, GM itself never calls this self-driving or automated technology. Following the 2014 ignition switch recall, GM instituted new safety review procedures on new products and those changes are reflected in the capabilities and limitations of Super Cruise.

What Is the 2018 CT6 Super Cruise?

This is a very capable advanced driver assistance system (ADAS) similar in principle to Tesla Auto Pilot, Volvo Pilot Assist, and Mercedes-Benz Drive Pilot. Navigant Research’s Automated Driving Vehicle Technology report projects that these types of systems—defined as Level 2 partial automation by SAE—will account for nearly 59 million sales annually by 2026.

I personally spent nearly 900 miles with the system over 2 days. Within its operating domain, it works very well. But the key is that operating domain or definition of where the system can work. This is a supervised partially automated assisted driving system. On divided highways where there are no intersections, cyclists, or pedestrians, Super Cruise can handle steering, acceleration, and braking with the driver taking their hands and feet off the steering wheel and pedals.

That doesn’t make it automated. As the Super Cruise branding implies, this is a more advanced adaptive cruise control. The driver must still watch the road and be ready to take over when the system encounters a situation it cannot handle such as a construction zone, lane merge, or faded lane markings.

A face tracking camera similar in principle to the Face ID system on the upcoming Apple iPhone X watches for facial and eye movements to ensure the driver is alert and attentive, something no other current ADAS does. Meanwhile, high definition navigation maps prevent inappropriate use on surface streets.

Misrepresentation Leads to Unrealistic Expectations

By continuing to call Super Cruise self-driving, media creates unjustified expectations of its capabilities with consumers. GM’s design approach should reduce the sort of misadventures we’ve seen from Tesla drivers on YouTube. However, customers that bought into the media hyperbole may be disappointed with the more cautious assisted driving technologies. Even as the technologies become more sophisticated, customers burned by misleading headlines today may remain skeptical and decide to hold off on future purchases, which damages the overall goal of improving safety on the road.

Patience

It’s one thing to take someone like Elon Musk at his word and assume he’s going to deliver what he promises (which he often eventually does, albeit over budget and months to years late). But by inaccurately portraying what a product can and more importantly cannot do, customer interest (and possibly safety) can be compromised. Truly automated vehicles will get here soon enough. Let’s not rush to mislabel.

Over the past several years, there has been an ongoing narrative that a battle has sprung up between Silicon Valley and the auto industry. The tech industry hype machine wants the world to believe that venture capital-backed startups are going to appear with some magic technology that disrupts and destroys the century-old incumbents. The reality is likely to turn out quite differently, with some of the brightest minds in the valley coming up with cool ideas that become a key part of the transportation ecosystem.

Tech Has Saved the Automobile Industry Before

The fact that the auto industry has remained vibrant over the past 50 years can in large part be traced to innovations that have emerged from the San Francisco Bay Area, particularly the silicon microprocessor that gave the region its nickname. At the onset of environmental regulation at the end of the 1960s, most of the functional aspects of cars were mechanically controlled, and these vehicles consumed more fuel and spewed more pollution than they do today.

As engineers struggled to meet the new regulatory requirements, the industry entered what became known to car enthusiasts like myself as the malaise era. Attempts to better control engines through mechanical means like vacuum lines led to many terrible engines with weak output, awful drivability, and barely improved emissions and efficiency.

Silicon Valley saved the auto industry from being suffocated by regulations. As early microprocessors and sensors were applied to engine and transmission management as well as new safety systems like anti-lock brakes, it became clear that computers in the car would be the key to enhanced driving. By the mid-1980s, electronic controls were enabling engineers to extract more power while using less fuel and cleaning up emissions. As fuel economy regulations stopped climbing, car companies offered customers improved performance and capability without making them spend more at the pump.

After earning my degree in mechanical engineering, I spent the next 17 years working on improving vehicles through  more sophisticated software running on a series of cheaper, yet more powerful slivers of silicon. Today’s most sophisticated vehicles utilize anywhere from 50 to 100 onboard computers to manage everything from lights that follow the angle of the steering wheel to automatically maneuvering a truck to connect a trailer.

Looking Forward to More Industry Collaboration

Silicon Valley has been a key enabler of the modern vehicle for decades. As we shift toward a world where most of the driving is done by software instead of people, the tech and auto industries must continue to collaborate more closely. The auto industry has developed an immense base of knowledge in building complex pieces of hardware at high volume and with high degrees of reliability and durability. Those machines come in a huge variety of configurations to meet virtually every possible transportation need.

Meanwhile, the tech industry has an unrivaled set of capabilities in developing software and electronics and driving down costs while improving performance. There are great minds on both sides focused on how to make mobility safer, cheaper, and more universally accessible. The Navigant Research Leaderboard Report: Automated Driving scored automakers and tech companies on their likelihood of success in commercializing this technology.

Almost everyone recognizes that transportation will change in the coming decades. The collaboration between the tech and auto industries has yielded incredible results for nearly half a century. New partnerships are going to form on the way to fully automated driving. There’s no need to spin those relationships into a competition when greater collaboration will likely yield much better results for everyone moving forward.

An old axiom in motorsports goes: “to finish first, first you must finish.” This means you can have the fastest car on the track and qualify on the pole position, but if you don’t have the preparation or team to back you up, the quality of the car is meaningless. In the race to make automated driving a successful commercial reality, hype may get companies all the media attention, but a fully realized strategy combined with the ability to execute are the keys to success. This is why Ford, General Motors (GM), the Renault-Nissan Alliance, and Daimler are the leaders in the latest Navigant Research Leaderboard Report: Automated Driving Systems.

Outside observers would not be faulted for believing that companies in Silicon Valley were about to roll over the entire automotive industry and take over personal mobility in the coming months based on news coverage. However, as many veterans of the technology industry have become painfully aware of, the reality is that building vehicles to safely transport the world’s population is far more difficult than just writing an app and publishing it to an online store.

Horse Before the Cart

Assembling a suite of sensors and writing the basic software to control a vehicle are actually the easy parts. Before that package can become a real product, you need a vehicle. Google developed its automated driving system in 2009 by hiring many of the top brains from Stanford, Carnegie Mellon, and several automakers that had previously created winning vehicles in the DARPA Grand Challenge program between 2004 and 2007. Then Google went to local Toyota and Lexus dealers and bought vehicles one or two at a time. Companies like Cruise Automation and Uber followed similar paths. In order to commercialize a system, they will need to invest billions more to develop and manufacture vehicles or find an automaker partner willing to supply cars.

Uber is reported to have lost more than $3 billion in 2016 without capital investment in vehicles or manufacturing. The world’s major automakers already have the engineering and manufacturing infrastructure in place, and many of them have been working on autonomous technology for far longer than Silicon Valley. Major automakers understand the intricacies of developing, validating, and certifying vehicles for profitable production.

At Navigant Research, we believe the leading automakers are learning what it takes to develop automated vehicles faster than new entrants can learn how to build cars. Companies like Ford, GM, Nissan, and Daimler also understand the regulatory and product liability hurdles faced by bringing automation to the world’s roads. These companies have heavily invested in controlling and understanding the key technologies required to make vehicles and automated driving system work seamlessly.

Just Around the Corner

The leading companies in this field are also rapidly developing their own in-house mobility services so that they can provide consumer access to automated driving systems while retaining control of vehicle manufacturing. This will help to ensure that the vehicles are properly maintained and updated—something that is key to safe and proper use within the early years of deployment.

Technology companies like Waymo and nuTonomy, as well as suppliers like Delphi and ZF, will have an important role to play in the new mobility ecosystem. But for now, automakers lead in the automated driving system race.