When faced with any major transformation, it’s common for misconceptions and misunderstandings to emerge as we begin to understand our new reality and learn how to manage it.
Given that transportation electrification is arguably the most significant industry transformation we will see in a lifetime, it’s not surprising that what we think of as common EV wisdom continues to change -– and change rapidly.
I was always a fan of the Discovery Channel television program Mythbusters where the hosts used science and logic to disprove all sorts of commonly held beliefs. But what about EV mythbusting? Given how important it is that we get it right when it comes to managing the EV revolution, it seems like a good idea to take a closer look at how utilities are currently analyzing EVs, and see how today’s EV assumptions stack up against the latest data science. So let’s dive in. (You can also watch my webinar on Debunking EV Myths here).
EV Myth 1 – DMV Data or Telematics Data is Sufficient for EV Analysis
When it comes to identifying which customers own an electric vehicle, many utilities are turning to DMV data or customer surveys. Others are starting to rely on data from customers who enroll in a telematics program, believing they can extrapolate from those samples to make assumptions about their entire population.
And of course that makes sense. We’ve compiled insights from DMV data for decades, and it’s fairly easy to acquire. And customer surveys have often been the only tool we’ve had to collect information about customer energy habits and household energy use.
There’s nothing wrong with these incumbent methods. The fact is, at this stage of the EV journey, any piece of EV data that we can get is an important one.
But at the same time, it’s a myth that these data sources are sufficient.
Let’s take a closer look at DMV data, for example. A DMV report is typically a summation of vehicle types by state. Sometimes, the list includes a breakdown by model or zip code. DMV data is useful in tracking growth month-over-month or year-over-year. But it provides no value in plotting or forecasting load shape. Because the vehicles aren’t associated with specific addresses, we don’t know what time they are charging, and we can’t determine how the vehicles line up to specific grid assets. Even if the data allows targeted outreach to a particular zip code, it doesn’t provide the granularity that utilities need to make informed, data-driven decisions about the impact of EV proliferation on the grid.
What about customers who opt-in to share their charging data through APIs telematics? Obviously, pulling data from a vehicle generates the most granular high-value data available. However, customers have to opt in to provide it, which means that utilities only receive telematics data from a very small percentage of the EV drivers in their service territory.
The truth is that accurate EV detection requires coverage that is both broad and granular. DMV data is broad. Telematics data is granular. But neither delivers both.
The reality is, that only artificial intelligence and disaggregation provide the complete data picture and depth of information utilities require, including charger type, charging habits, and load profile.
EV Myth 2 – In-House Analysts Can Effectively Use AMI to Detect EVs
Some utilities have turned to their in-house data analysts to run queries to find the EVs in their territory.
In the spirit of the Mythbusters TV show, let’s evaluate this idea by taking a deep dive into the science.
In the most basic terms, the data science known as disaggregation is the breaking apart of the energy load of any residential or business premise into the appliance-level contributions that make up the whole.
For more than a decade, Bidgely has been disaggregating smart meter data and plotting these data points over the course of a day. We are typically looking at 15-minute granularity, which reveals 96 points over the course of a day.
Throughout the day, a premise’s electricity consumption goes up and down, forming a unique shape. Within that shape, each appliance also has a unique fingerprint that we can identify.
When all of the appliance fingerprints are layered one on top of the other, it is hard to identify what appliances are present. But what our disaggregation algorithms do is isolate each layer of the total energy load. We start with always-on consumption and peel that layer away. Then we identify heating and cooling fingerprints, which reveal themselves with their temperature dependencies and cyclical on-off, on-off cycles. Then we strip away high amplitude appliances that pull a lot of power quickly, like hot water heaters, which reveal themselves with unique “pointy” load shapes. Only then can we clearly identify EV signatures, which are both high amplitude and pull power for many hours at a time.
With true disaggregation, we can see very clearly the start and end of every charging session. We are able to determine how many times a week a customer charges a vehicle, and how many kWh they are using each time and in total. We are able to determine what type of charger is in use. And ultimately, we are able to form the type of detailed EV customer persona that is essential to every aspect of successful EV load management.
To get to this level of detection takes extensive ground truth. In Bidgely’s case, we have more than 30 data scientists, 11 years of experience refining our algorithms, and over 20,000 households of EV ground truth.
Generic methodologies won’t suffice. For example, an in-house analyst may identify patterns such as consecutive reads that are above six kW for 4 times in a month. They will identify some EVs within a population, but it will also yield false positives of 75 percent or more, which makes this approach unlikely to provide value.
The reality is, even the most talented in-house teams don’t have the experience or the depth of data to accurately identify EVs from within AMI signatures. There is simply too much complexity involved in the EV data science puzzle.
EV Myth 3 – My Vendor has a Solution with High Accuracy so I’m Set
Many vendors today claim 90% or 95% accuracy, which sounds like a solid track record. But here again, it’s time to turn to the science.
What do we need to be measuring? How do we define what success looks like in a data science classification algorithm? How do we know if a data science algorithm is good? Or bad? Or somewhere in between?
Let’s put this in the context of a television meteorologist. Every day that person makes a prediction as to whether it will rain tomorrow. And TV viewers usually assume he or she will be right most of the time.
But what does it look like if we actually plot the meteorologist’s predictions over time vs. the actual weather? A true positive occurs if the meteorologist predicted rain, and it did in fact rain. A false positive occurs if the meteorologist predicted rain, and no rain fell. A false negative occurs when the meteorologist predicted sunny skies, and instead we got caught in a downpour. And finally, if the forecast is for no rain, and no rain falls, we have a true negative.
We can think about EVs in this same way. Do I predict the customer has an EV? Or do I predict they don’t have an EV? And then do they actually have an EV? Or do they not have an EV?
The simplest measure of accuracy quantifies the percentage of true positives and true negatives over the total number of predictions. The higher the percentage the better.
But there are other important metrics as well.
Precision, for example, quantifies the number of times a meteorologist predicts rain and it rains, or we predict a customer has an EV and there is an EV in the household. And again, we always want to maximize this result.
Then there is recall. Here we want to understand out of all the EVs in my territory, how many am I actually able to detect. Recall is the measure of the number of your true positives over both your true positives and false negatives combined.
And finally, there’s the false positive rate.
Let’s look at this in practice.
EVs are growing in popularity, but currently most communities in the U.S. have an average of 1-2% penetration. This changes how we view success. I can easily create an algorithm that finds no one has an EV. In a territory with two percent EV adoption, I would actually be able to claim 98% accuracy for that algorithm. On the other hand, if we use a generic approach such as the one I described when we were examining Myth 2, we would have quite a few false positives, so the accuracy measure would be absolutely terrible – under 30%.
Taking it a step further, it’s important to play with the back-and-forth between false negatives and false positives. What is better for the outcome? Is it better to minimize false positives? Which means we won’t be telling customers they have an EV if they don’t, but we might also be missing a lot of customers who do have EVs. Alternatively, we might decide it’s really important to maximize the number of detections, by identifying everyone who has an EV, even if we end up suggesting to many people that they have an EV when they don’t, and increasing our false positives. (In most cases, our recommendation is to avoid false positives, because they erode consumer trust in the utility.)
The key takeaway is that to determine EV detection success requires a bigger picture understanding of what goes into an accuracy claim, as well as how measures of precision, recall and false positives figure into the equation that determines how accurate an algorithm is. A simple claim of general accuracy doesn’t equate to EV detection accuracy.
EV Myth Four – What Works for One Use Case Will Work for All
We’ve described above how EV intelligence is critical input when managing EV growth. And that is true across the board, for both customer and grid use cases, including targeted EV program marketing, EV owner engagement, EV program planning, infrastructure planning, EV growth analysis, and EV forecasting.
Given that there are such a wide range of potential EV intelligence applications, the question that we must analyze is whether the same data science approach can be employed for all.
The truth is that while it is tempting to look for a single data silver bullet, different use cases have unique EV intelligence requirements.
For example, in the context of targeted program marketing, a lower level of accuracy might be acceptable, but establishing EV customer engagement requires a much higher level of accuracy. In the first instance, we don’t need to know a great deal about the customer — only that they own an EV. Promotional language that uses more generic phrasing is acceptable – such as to recommend a new charger purchase or invite a customer to join a managed charging program.
By contrast, customer engagement requires personalized messaging, and in order for that personalized messaging to be effective, it must be accurate and highly relevant. For instance, if we tell a customer that, based on their current charging time of use, they could save $X if they were to switch to a time of use rate, or send them an alert that they’ve charged during a peak period, what we say must be true. Similarly, billing estimates have to be highly accurate. False positives, or incorrect hyper-personalized content, create a negative experience for the customer, and an increased likelihood that they will lose faith in your solution.
Identifying the precise level of accuracy required for each unique EV use case is key to successfully employing EV intelligence.
EV Myth 5 – It’s Too Hard to Really Leverage This Data
Maybe you don’t have in-house analysts, or the data team you do have is already overburdened. In either case, it’s tempting to believe that you don’t have the resources to leverage behind-the-meter EV intelligence.
At one point in time, this was likely an accurate notion. In a territory with a million customers, each one is generating ~35,000 AMI data points annually. Once we add the estimation from every single device in the home, suddenly we’re faced with crunching billions of data points. With legacy tools, analysis at this scale would require days or weeks of work.
But today, Bidgely’s UtilityAI Analytics Workbench tool is helping utilities affordably and efficiently distill those billions of data points into actionable EV intelligence, regardless of their in-house analytics capacity. Analysis that historically would have taken multiple highly skilled analysts several weeks to complete manually can now be accomplished within hours even by non-analysts.
If I am a program manager, and I’m really interested in understanding the utility’s EV load shape, a single click applies the filter I need to reveal where my EV owners are located geographically and what their load shape is. It’s also possible to map substations, feeders and transformers to EV customers in order to pinpoint where electric vehicles fall on the network, and the load shape that is occurring on each grid asset.
I invite you to visit our demo portal (contact me at [email protected] to request access) to see for yourself. Teams across the utility – including those with limited technological sophistication – can use Analytics Workbench to distill EV intelligence from AMI data and put it to work to inform and improve the complete range of EV-related use cases. It is easy for analysts and program managers alike to use the tool and its outputs to make data-driven decisions.
EV Myths busted. Now what?
Learning to leverage data science to inform utility operations is a process that takes time. Given the anticipated rapid pace of transportation electrification, now is the time to inform EV programs with behind-the-meter intelligence so that you have sufficient time to design, refine, and scale.
If you are curious about how other utilities are managing their data transformation, check out the case studies, video interviews and more featured on our EV solutions web page and reach out to our team to schedule time to learn more.
And don’t forget to WATCH my webinar where I dispel these EV myths, and provide an overview of UtilityAI Analytics Workbench: EV Intelligence.
