This is part four of a six-part series offering a comprehensive guide into the 6 Must-Have Requirements For Getting Meter Data With A Utility API.
Here are the six parts:
-
6 Must-Have Requirements For Getting Meter Data With A Utility API (Summary)
-
Utility API Must-Haves #1: Coverage, Utility Bill Data & Interval Meter Data
-
Utility API Must-Haves #2: Customer Experience For Accessing Utility Data
-
Utility API Must-Haves #4/5: Dev Friendly Meter Data Collection & Security
Introduction
Reliable customer authentication, data quality and retrieval are critical, and also the most difficult requirement to deliver. Picture this: your solution is up and running, then suddenly customer authentication stops working, or you’re not getting bills/intervals as expected. Your team will have an unplanned interruption and will need to:
-
Notify affected customers and modify your authentication flow to let new users know there’s a temporary issue.
-
Have your engineers investigate the root cause, which could be from your product, the utility api solution or directly with the utility.
-
Work with the solution provider’s support team to implement a fix.
-
Guide customers through re-authentication once the issue is resolved.
-
Make sure data flows correctly afterward and doesn’t require manual cleanup.
Utility Data Access Methods
To better understand reliability challenges we must first understand the possible direct utility data management methods. While briefly listed below, we’ll cover these in more detail in a separate post:
-
Non-Credential-Based: Examples include customers sharing bill PDFs/pictures or Green Button Download My Data (DMD) files from their utility, utility electronic data interchange (EDI), or old school letter of authorization (LOA) workflows.
-
Credential-Based (Recommended for Scale):
-
Custom Credential Collection / Utility Website Scraping: Companies build their own custom software to collect customer utility login credentials and then log into the utility website behind the scenes to download data.
-
Green Button Connect My Data (CMD) APIs: For each utility, register to integrate their Green Button Data API. Customers grant one-time access so data flows automatically from their utility account. This process and API will differ for each utility.
-
A Utility API: Companies use a third-party service like Bayou Energy that already handles customer utility credentials, scraping utility websites, and data normalization so you don’t have to.
-
What Causes Utility Data Management to Be Unreliable?
Both non-credential and credential-based methods suffer from:
-
Utility Website Downtime: Sometimes planned, sometimes not, and usually without any warning – who has the free time to track utility website downtime? If a utility site goes down, most data access methods will fail until it’s back online.
-
A tradeoff everyone loves – your existing roadmap or utility website, bill or API changes: Even if a Green Button Connect My Data (CMD) API gives notice of their upcoming changes, updates can be significant and happen on short timelines injecting chaos into your roadmap and engineering schedule. If you’re scraping utility websites (either with your own system or through a utility API), you’ll need to quickly adapt your scrapers or risk service disruption.
How to Handle Website or Green Button Data Reliability Challenges
Before committing to a solution provider, ask for success rates (by utility) for:
-
Customer Authentication (including credential sharing and multi-factor authentication)
-
Utility Bill Data Availability & Parsing
-
Interval Meter Data Availability
After deploying, you’ll want to monitor these same metrics for your user base and get alerts when a utility’s API or site goes down, so you can plan accordingly.
Ensuring Utility Data Quality
-
Bills & Account Data: Structured bill data values should have checks to handle various financial scenarios, participation in different utility programs, rooftop or community solar, third-party suppliers, time-of-use rates, and complex commercial/industrial bills.
-
Interval Meter Data: Quality checks need to account for problems you’d find when doing interval meter data analysis – duplicates sent by the utility, gaps in the data and not being timezone/daylight savings time aware.
Even the best solutions won’t have 100% uptime or success. Problems will happen—so it’s essential that your provider has solid internal monitoring, proactive fixes, and responsive support.
Key Questions for Potential Solution Providers
-
How does your solution get utility data?
-
What data-quality practices do you use for bills and intervals?
-
Do you perform your own interval meter data analysis? How do you handle duplicates, gaps, or daylight savings in intervals? Statistically, how common are these challenges present?
-
How often are utility websites down or changed, and how do you notify us? Can you break this out by individual utility?
-
What’s your success rate (by utility) for:
-
Authentication (including MFA)?
-
Utility bill data availability and parsing?
-
Interval meter data availability?
-
-
What do you monitor internally to maximize product quality and reliability?
-
Do you offer any ability for companies using your solution to monitor authentication, bill and interval success for its customer base?
-
What is your support process and what do response, diagnosis and fix times look like?
Wrap Up
If you have any questions about implementing a utility API or would like more details on how these requirements fit your unique use case, schedule a call with James (Bayou’s CEO). We’d love to learn more about what you’re building and how we might be able to help.
To continue reading, Here are the six parts of this series:
-
6 Must-Have Requirements For Getting Meter Data With A Utility API (Summary)
-
Utility API Must-Haves #1: Coverage, Utility Bill Data & Interval Meter Data
-
Utility API Must-Haves #2: Customer Experience For Accessing Utility Data
-
Utility API Must-Haves #4/5: Dev Friendly Meter Data Collection & Security