Amperon’s Energy Forecasting Methodology with Elliott Chorn

Introduction

Kerim: Hi, everyone. This is Kerim, Kerim Baran with SolarAcademy. I’m here today with Elliott Chorn of Amperon. Elliott, welcome. It’s good to have you.

Elliott: Hey, Kerim, thanks for having me today. I’m really excited to be here.

Kerim: And this is the second in a series of conversations that we are doing with Amperon. In our first conversation, I had the opportunity to host Sean Kelly, President of Amperon, and we talked about Amperon’s recent fundraising, as well as its general offerings and products for the energy transition market, as it relates to forecasting power on both the supply and demand side.

Today, Elliott and I are planning to dive a little deeper into Amperon’s approach and methodology in forecasting. But before we go there, Elliott, I want to ask a little bit about you, how you ended up at Amperon, maybe a little bit about your professional and, if you want, personal background as well. So let’s start with that, and then we’ll tie it into how it all relates to Amperon.

Elliott Chorn’s background before and after he joined Amperon, and why he joined Amperon

Elliott: Yeah, sure. We’ll start with the important stuff. I’m married, with two beautiful kids. So we got the important stuff out of the way. Professionally. I’m a long-time power market, professional. I’ve spent most of my career using products like Amperon. I spent most of my career buying and selling power in one format or another. I started off my career selling power to large energy users, hospitals, universities, big manufacturing.

I became fascinated with the industry but was really not cut out to be a “salesperson.” So I took the opportunity with the company that I was with to move into our back office. I moved into the back office with them, started running RFPs, digging into quantitative analysis, and learning more and more about the industry.

That company eventually spun off and became a retail energy provider. I was the key employee, doing all of the commodity back office work for them. So I did all the pricing, structuring, hedging, forecasting, etc. for a small retail energy provider.

From there, I spent a few years on a sabbatical, spent a few years out in Colorado, goofing off, having fun, and then came back into the industry. I spent, prior to Amperon, about six years, with a small consulting firm/ hedge fund. So we were doing consulting in the industry, trading around renewable assets, giving risk management guidance and advice to customers as well as running a small hedge fund where we were trading electric power. So again, my background is all around buying and selling power in one format or another, which was a real natural evolution for me to join a company like Amperon.

I met Sean and Abe about six years ago now at a conference. Sean and I having similar backgrounds and as former power traders hit it off. Started lightly flirting about how what it might be like to work together. Eventually, came to the conclusion that it made sense for us to work together, and I joined the company, now about four and a half years ago, as VP of product.

Kerim: Nice. And you’ve been working obviously, on the product for all these years. Can you tell us a little bit about like what appeals to you in Amperon’s approach to the problem you’re solving, which is mainly forecasting, I understand it? And yeah, maybe, a little bit about the market dynamics and how that product market fit appeal to you.

Elliott: Yeah, that’s a great question. I think this is a problem that the industry has faced since day one. Electricity grids have to be perfectly in balance at all points in time. And so, knowing how much power we’re going to need tomorrow or next hour or next month becomes a very, very important question for then, how do you operate your power grid?

So again, I’ve spent most of my career using load forecasts to try to trade effectively, most of the time getting frustrated because load forecasts weren’t accurate enough, mostly my own fault, because load forecasting is very, very difficult.

So when I met Sean and Abe and started hearing about what they were working on, I started to get excited because they were bringing true modern computing power to bear on an age-old industry problem of how much tomorrow, or how much next hour? It’s the quintessential problem for the power industry is how much. So keeping things in balance becomes so important. And so when I heard their initial MVP accuracy metrics, my jaw was like the cartoon character, dropped, the tongue rolls out, kind of the whole thing. It was truly a jaw-dropping performance, and I knew from that how valuable that was. Because of my background, understanding that each fraction of a percentage point and accuracy, how much that would drive value, whether it was increasing the value of my solar farm, increasing my margin on the load that I was serving, or allowing me to be more aggressive with the speculative trading that I might be doing, it really gives an advantage to any participant in the marketplace. So I saw the breadth of opportunity to apply these capabilities too.

Unique core competencies of Amperon: Domain expertise, weather data, data cleaning capabilities

Kerim: Got it. And so I’m assuming this is the uniqueness of the Amperon’s Methodology to forecasting that allows for that extra margin to be created. Is that thanks to the AI/ML models, you guys are deploying in the background? Or are there other elements or drivers for that increased efficiency?

Elliott: Yeah, I really think about it, that three real key drivers to our unique core competencies set us apart from the competition. The first is our domain expertise. Like, I said earlier, Sean is a former power trader, who spent 20 years in the industry. When, on day one, you have someone with that level of expertise diving into the market, not building this just as a pure thought exercise, but understanding the use case of that, that brings you an edge right there. So one is our domain expertise, our understanding of power markets and how they work.

The second is then the data cleaning, so pulling in data, managing that data, and making sure that it’s clean and ready to be run into those artificial intelligence models. In addition to that the quantity of weather data with which we operate is really staggering. We have more weather data in our system than we have any other data. Load data, meter data – weather data is, the be all end all, starting point for us. So our ability to ingest all of the historical data for weather as well as forecast data, not just looking at temperature, but looking at temperature, dew point, wind speed, irradiance data and looking at these highly diverse points spread across the United States. Unlike, historically, due to computing power, we’ve been constrained to only looking at a few weather points, trying to be representative of the dynamic weather systems that we’re facing.

Kerim: Yeah.

Elliott: So when you take those three things and you combine them and pour them into our AI/ML modeling, that’s when the magic starts to happen. So without those three pillars that we build upon, being, domain expertise, cleaning of the data, and weather data, it wouldn’t matter how fancy of machine learning models we had, or how great of data scientists we had, if we didn’t have that data and that domain expertise to manage and use that data.

The nuances of various regional energy markets and how Amperon serves those markets

Kerim: Yeah. So this is really interesting because since our first conversation, I’ve been a little bit more intrigued about the various markets and the RTO markets, and the dereg, deregulated markets as well as the regulated market. So when you take all of Amperon’s approach, methodology and the way you run your AI/ML models, does it apply across the board to all markets, whether there’s RTO or not? Or if it does or doesn’t, how do various types of entities become customers? Which customer finds what kind of value when you are…

Elliott: The too long, didn’t read, answer to that is, electricity is the same across all markets. Right? So if we can get the data cleaned, and into that format that our models are comfortable with, then we can do this across any market anywhere in the world, pretty much instantly.

The challenges I was alluding to, and this is where those other pieces of our core competencies come in, our domain expertise and our comfort with managing and cleaning these large data sets, is that each market has all sorts of nuances to it. Whether we’re talking about it at the wholesale level or you’re talking about RTOs, regional transmission organizations or ISOs, independent system operators, or you’re talking about areas like the southeast of the United States, the southeast reliability coordinator, where there is no overarching organization, there’s a bunch of utilities working bilaterally, each of those entities, whether you’re a participant in an RTO market, an ISO market, a non- RTO, SERC or the Western Electricity imbalanced market, any of those markets, the challenge that we face as a company is getting that data cleaned and into the format that our models are going to be happy with. 

And that’s where, again, a big part of our core competency, our value-add, comes in is we’re able to do that. We’re very comfortable and confident, doing that which then powers those AI/ML models. That then, as a result, we’re not feeding aberrant data into it. We’re able to leverage that to use this cutting-edge technology to then produce our industry-leading MAPEs.

Kerim: Industry leading – what did you call that?

Elliott: MAPE – mean absolute percentage error. MAPE is a very common forecast error, methodology –

Kerim: Yeah.

Elliott: – we commonly refer to. It allows you to compare different forecasts and have them placed on the same scale to evaluate for accuracy.

Kerim: Got it. 

Long-term vs short-term energy forecasts

Can you tell us a little bit about those forecasts? I’ve heard you use the term, long-term versus short-term forecast before, in the context of Amperon’s offerings. And maybe, if you can, I don’t know how much of your secret sauce you want to divulge, but maybe talk about like a layer below, like what the AI/ML model inputs and outputs are in determining those long-term versus short-term forecasts.

Elliott: Yes. So the primary difference between a long-term forecast and a short-term forecast is really going to be around the weather. For a short-term forecast, what we consider our short-term forecast is operating dates. So today to operating day plus 14, so it goes out 360 hours into the future.

That is where we can get reliable. I use the word reliable in air quotes because we’ve all gotten angry at the Evening News Meteorologist for being wrong, but reliable, quantifiable forecast that we can then feed into our model.

Anything past that 14-day window, we have to rely on weather simulations. So for weather simulations, we’re looking at historical weather patterns. At Amperon, we take 15 years of historical weather data, and then we run about a thousand different simulations through our model, to then present a cone of probabilities for our customers.

So unlike in the short-term model where I can feed actual weather data, historical load data into two types of models. So we use a combination of regression models. If you remember old high-school linear regression, take two points, plot them out, and then try to find the line between them that represents what the future point would be.

Taking that same concept, doing higher order, polynomial regression analysis, combined with machine learning, specifically on the machine learning side, we use gradient-boosted models. A gradient boosted model is a type of decision tree that uses weak learning to reinforce patterns that result in, overall, better outputs. We combine those models in our ensemble model, which provides a discrete output that says, “Based on these weather inputs, here’s what Amperon believes that the resulting load or renewable generation will be.”

Again, on the long-term forecast, we’re giving a range of potential weather inputs because we don’t know what the weather is going to be. We don’t have reliable forecasts out that far. But again, the same concepts where we’re taking regression models as well as these gradient-boosted tree models, combining them into our final output, which is our forecast for our customers.

Kerim: And how do you measure, ultimately, the improved accuracy versus prior methods like when you are convincing a customer? Maybe they’ve become a customer a year ago, like when they look back, how do they…?

Elliott: Yeah. A big part of it is the MAPE calculation that I was referring to earlier, so mean absolute percentage error. There are a number of other accuracy metrics, some of which are better in certain circumstances than others. Some of them mathematically don’t work as well. For example, in a house that has a solar panel on its rooftop, where its load is going negative or approaching 0, that MAPE calculation doesn’t work.

So there’s a number of other calculations. I can list them up. There’s a bunch of acronyms, normalized mean absolute error, root square mean error. These are different calculations that our customers can use to evaluate the accuracy of these forecasts.

Kerim: And I’m assuming these are the kind of calculations you’ve done day in and day out, for years, as you were working in the electricity markets prior.

Elliott: Yeah. Some of these formulas I can still write without having to think too hard in Microsoft Excel, going back to my old days of being a quant, and having my fingers on the keyboard.

Grid-level vs meter-level energy forecasts

Kerim: Got it. And then we also briefly talked about grid level versus meter level forecast. Can you tell us a little bit about those, and how your approach works with them?

Elliott: Sure. So at Amperon, we forecast everything that’s ranging from an individual meter. My house, for example, is forecasted by Amperon. I’m a customer of one of our customers. So I can go look at my house’s forecast.

We do that, and then we scale up to entire operating grids. So the Australian electricity market operator, AEMO, is one of our customers. We do full-grid scale forecasts as well for all of the North American regional transmission operators and independent system operators.

The real nuance between those is going to be how we deal with the weather. When we have an individual meter, we can geolocate those weather points to be within two kilometers of the location. So, for example, I live outside of Houston, Texas. The old-style way of forecasting would use George Bush International Airport, which is, I don’t know, about 45 minutes away from my house. That would be the weather point that I would be subscribed to and that everything would be based off of.

We’re able to then geolocate that and put a weather point much closer to my house, which then results in a more accurate weather input into our already accurate models. It results in an overall more accurate forecast.

Now the challenge then becomes for grid-scale, we don’t have the location of every single meter on the grid, right? It’s private information. They’re not going to give us that information, no matter how nicely we ask. So instead, we have to take a little bit of a unique approach. We use satellite imagery data to look at population density. And then we layer in the weather points that we choose for those and then weigh those weather points to attribute for population density as well as load responsiveness to weather change. We do all that through a significant amount of back-testing quantifiable analysis to then result in these weather systems that we’re able to apply to our model.

So it’s really the difference between looking at an individual meter where there can be a fair amount of noise in that meter. For example, if I have guests in town, my electricity consumption could be very, very different than that week that I didn’t have a guest in town.

When you look at an entire grid, a lot of that noise kind of works its way out of the system. So we’re able to be significantly accurate at both the grid and the meter level, but find the scale of large numbers that allows for us to have even more accuracy at the grid scale.

Kerim: Got it. Thanks for that answer. 

Amperon’s perspective on comparing regional electricity market dynamics: TX, CA & Northeast

So I guess when you’re looking at markets that have different dynamics, whether it’s the California market or the Texas market or the Northeast market, what customer choice opportunities are there in those markets that justify different uses of Amperon’s solutions? How do you guys serve those customers?

Elliott: Yeah. So I guess we’ll take three examples. Let’s take Texas, California, and the Northeast. So Texas is the most liberalized market in the United States, liberalized, meaning that there is no utility of default that will serve me power. If I want to get my lights turned on in Texas, I have to call up a retail electricity provider. That retail electricity provider will work with my utility to turn on the data and turn on my power for that customer.

Kerim: Excuse my ignorance there. That’s part of the layer of the industry that I am not very familiar with and not so familiar with the text, so when you have a choice to pick 1, up to 2, 3, 4, or 5 retail electricity providers. But ultimately, they’re all buying the electrons that you’re using in your home, from, probably, the same source which is the main power plant in or near Houston, I assume, right?

Elliott: So they’re all buying from the ERCOT power pool. That’s accurate.

Kerim: Okay, ERCOT power pool, but not one single power source, right?

Elliott: Correct. Yeah, they’re serving at the load zone.

Kerim: Got it. Okay, please continue. I just clarified that.

Elliott: Yup, of course. So in Texas, as a homeowner, I am forced into customer choice. There is no incumbent utility. There is no backup utility. Now the opposite side of that would be somewhere like Georgia, for example, where Southern Company is the incumbent utility, that owns both the retail relationship with the end user as well as all the wires, transmission, and generation. There, there is no choice for that customer to do anything.

Kerim: It’s vertically integrated.

Elliott: Correct, vertically integrated.

Kerim: But they still do have very good pricing there, much better than we have here in California. Right?

Elliott: That’s true. Yup. There are advantages and disadvantages to a vertically integrated utility system. You know the advantage is that they have more insight into what load they are going to be serving, or need to be serving because they know what their service territory looks like.

The downside to it is obviously, that there is no choice, there are no, oftentimes, programs for renewable energy, lag behind. An example like that would be a state like California. So in California, we still have the three incumbent utilities San Diego Gas and Electric, Pacific Gas and Electric, and Southern California Edison. If I want to turn on my power, I pick up the phone, I call So Cal Edison, if that’s my service territory.

Now in California, the one unique thing is that if I want to buy renewable power, if I want to get 100% clean power, I have the option to buy my power from a community choice aggregator, which are regionally defined service territories, where I can buy 100% clean energy from a third party provider, the –

Kerim: For the same price.

Elliott: For the same price. So the franchise nature of that gives so that the price can be more stable, but it allows for me to get access to the more renewable power.

Then the difference between all of these markets for Amperon is in each and every one of these markets there’s a use case. So in Texas, for example, we have retail suppliers that have to buy power every day, every hour, from the market, as we were talking about their power pool, to supply their customers. Well, how much power? That’s their question. Every single day, every single hour, how much power? I don’t want to buy too much. I don’t want to buy too little. If you’ve been following the news power prices, even in the shoulder season, have been all over the place here in Texas this spring.

If you look at a company like Southern Company, they don’t want to make a big capital investment and say, “Hey, we’re going to build this new gas-fired peaker plant, or this new solar facility because we think load is going to be X, but in reality it was Y, and we now have either not enough of assets, or we have a stranded asset.

The So Cal Edison situation with the CCAs, the CCAs need to know how much power to go out and contract for, and because they have a commitment to buy 100% clean energy, they have to know how much to do that ahead of time. And that’s where Amperon’s forecasting comes in.

Now, the challenge for Amperon is that in each one of those situations, the data is different. Right? The data coming in, the raw data format that we’re getting in is in any variety of formats, but something that we again, take a lot of pride in, is owning our data pipelines. We don’t outsource that to anybody. We do all of the hard, nitty, gritty work to ingest, manipulate, and store that data because it’s such a key input to our models.

Kerim: Yeah.

Elliott: If we keep bad data in, we’re going to get bad data out.

Kerim: Right. So it is really important to get all that data yourself. And what are some detailed levels of activities you guys are doing to create that competitive advantage in a way, I guess, right?

Elliott: Yeah. So we connect with what’s called SCADA systems. I can’t remember the exact – system control automation detection, something to that effect, what exactly the acronym is, but these are the systems on the raw meters where live telemetry data is coming in from the marketplace. We connect to those systems which gives us access to real-time data coming, as fast as we could possibly want it.

The question then becomes about the accuracy of data and the cleanliness of that data because it’s not what we refer to as revenue quality data. Revenue quality data comes sometimes 180 days after the fact, where we get that revenue quality data. And so we were ingesting all of this SCADA data. We’re running a data cleaning algorithm on top of that, making sure that we’re not getting, you know, sometimes, we’ll get meter reads from the future. Meaning, three hours from now in the future, it’ll say “Demand was X”, and we’re saying, “But that happened in the future. That can’t be real.” Right? So you have to clean all these things up as they’re happening. If you look at it and you say, “Well, Elliott’s house now went from using 3 kilowatts to now it’s using 300 kilowatts.” You can either forecast off of that and have a probably incorrect forecast, or you can clean and scrub that anomalous data out.

Kerim: Yeah.

Elliott: As we go from SCADA data, we work our way through some different levels of more processed data. We integrate directly with the independent system operators, directly with the utilities, to receive what’s called EDI data, electronic data interchange. That’s for retail choice markets, so those are markets where customers either are forced to or have the choice to buy power from someone outside of the incumbent utility. That’s how all data is transferred back and forth, is via EDI.

As we work with utilities, we get access to AMI data, which is the advanced metering infrastructure system. So that gives us oftentimes, clean 15-minute data. Sometimes 5-minute data which allows for us to do even more hyper granular load forecasting.

And then eventually, as things make their way downstream, we’re getting revenue quality data, bill quality data, that’s being processed by the ISOs.

And we use that data to then redirect and check what’s happening in real-time, in this kind of interplay of different data sets, being married together, all in real-time to produce those forecasts.

Kerim: Wow, very fascinating world. This is a part of the industry that I wasn’t as knowledgeable on, but the fact that you have to accurately predict all that electricity consumption ahead of time, and I’m sure there are contracts you’re doing days ago, weeks ago, hours ago, maybe minutes ago, too, to supply, and the complexities of forecasting that exact amount, and what that’s worth is fascinating.

Amperon’s unique approach to energy forecasting: Data, Weather & Data Science

So if we were to summarize all of this, all the unique things you do, and your unique approach to forecasting, what would be a good way to summarize it all, for your customers?

Elliott: Yeah. For me, in a very simple way, it comes down to data, weather, and data science. When we put those three things together, it is the combination. There are lots of people out there that are really good at dealing with big data, and would scoff at the amount of data that we refer to as big data. They would say, “That’s not big data. That’s not that much data at all.” But it’s understanding that data, how it fits into the world. It’s our expertise with weather data.

Our second hire was a PhD meteorologist. It’s not typical hire for a tech company to hire a meteorologist, second, but that gave this the edge in how we think about the weather. And then the quality of our data science team is, as funny as it might sound, there are actual load forecasting competitions.

We’ve competed in them. We either win or hire the people that win, and so we’ve put together a team of data scientists that have been combined doing load forecasting now for 30, 40, 50 years and are putting out models that are far and away, the best out in the industry.

Kerim: No doubt and that has been rewarded by the market, rewarding you with tens of millions of dollars of financing over the past several years that I’ve been aware of. So Elliott, thank you very much for all these insights and knowledge you shared with us and our audience.

I’m sure we will continue learning more from your team, and we’ll share it as we go along. And thank you for everything you do for the energy transition that’s happening in the world. Thank you.

Elliott: It’s my pleasure. Thank you for your time. Thank you for everything that you’re doing to help with the transition. It’s much appreciated and look forward to talking soon.

Kerim: Thank you.