Forecasting and Visualizing Complex Data for an Energy Management System

The Zimbabwe-based startup, NeedEnergy, addresses Africa’s energy problems, focusing on renewable and equitable systems.

NeedEnergy answers the growing demand for reliable energy in Africa with innovative technology. It applies big data analytics, machine learning, and AI to optimize the utilization of the energy grid and other energy sources.

NeedEnergy provides intelligent monitoring and managing of energy systems. They design custom solutions for both individual and commercial clients that include:

• Measuring energy consumption and production data using Smart Meters
• Visualizing gathered data with interactive dashboards for better monitoring
• Forecasting both consumption and production so systems can be optimized and balanced

The company supports greener, more just, and decentralized access to energy sources with the mission to provide sustainable energy for Africa.

About the project

NeedEnergy wanted to visually present their data-based solutions for energy optimization to kick start a pilot with a leading microgrid operator in sub-Saharan Africa and to prove the concept in Australia’s remote regions energy market to secure support for growing the business. The visuals needed to capture the company’s business value, goals, and vision.

The role of Netguru

Armed with real energy consumption data from two sites to use for a forecasting prototype, they enlisted experts from Netguru to create:

• Advanced visualizations for the energy management system
• An interactive demo of an energy consumption forecasting model built with machine learning that uses real, historical data

Netguru was responsible for:

• Providing data science and UI design expertise to build a professional demo under a tight deadline
• Guiding the client on what is feasible, realistic, and would bring the most value to their product
  
  

Together with NeedEnergy, Netguru carried out a Data Strategy Workshop, during which we explored the critical areas that they should cover in a presentation and formed the requirements for the solutions we were working on.

Next, we created an interactive demo featuring all the most crucial functions. It simulated the energy consumption forecast based on a selected date and time, and then visualized it alongside the actual consumption for comparison. The demo was powered by a machine learning model that was tailor-made for the client.

Finally, Netguru created UI Design of wireframes for energy management system visualizations.

An exemplary monitoring of solar grid locations.

How Netguru did it — approach to the project

In order to demonstrate how historical data can be used to generate predictions, Netguru used available energy consumption data collected over two years as well as historical weather data collected using IBM’s The Weather CompanyAPIs to create a machine learning model demo.

The data, merged into time series with one-hour intervals, shows strong seasonality patterns; and the energy consumption tends to vary both daily and weekly.

An example of forecasts generated by demo application – energy consumption seasonal effects.

Simulating real forecasting usage scenarios

We trained and tested several different models to simulate real forecasting usage scenarios. NeedEnergy wanted to generate 36-hour predictions using not only long-term historical data, but also the most recent data collected since the previous prediction was made.

To simulate real forecasting scenarios, the dataset was split so that multiple models were trained and updated gradually over the last two months of the available time series data.

As is usual in such projects, after several rounds of experimentation, the best-performing model was selected and optimized for later use.

Demonstrating the model performance

The next key step was to demonstrate the model performance, and for that purpose we created a Streamlit web application that can interact with the model. To deliver the software in packages, we used Docker and then deployed it on AWS using Amazon ECS, Fargate, and Load Balancer services. The app is now live here.
Example forecasts generated by the demo application for a specific forecast date and time.

It allows for checking both the daily and weekly fluctuations in the grid’s energy usage. When changing the forecast point, the Energy Consumption chart updates in real time and demonstrates modelled predictions for the following 36 hours. It updates with the most recent data.

Similarly, the Forecast vs Actual difference chart updates according to the forecast point.

Overall, the results show that the model is able to correctly forecast regular energy consumption patterns, mainly during working days. However, it is not yet capable of predicting sudden and more random increases in energy demand. The model will keep improving as more data is collected.

The model and demo application were developed within a week and a half and their sole purpose was to demonstrate the value gained through ML model predictions in the context of the client's business model. The next iterations with more data and more experimentation will result in a production solution.

Energy management system main dashboard.

Results of the cooperation

• Netguru’s UI design and data science experts delivered both parts of the demo efficiently.
• The dashboards demonstrate all the key components of the Energy Management System and how insights found there would be used to monitor, plan, and optimize energy usage. For example, the user can see how actual energy usage compares to the forecasted energy mix split according to sources and calculations of generated revenue and savings. The user can also get relevant notifications from connected devices.
• Although the solutions achieved were more prototype-like, they were based on real data and had a strong focus on what is feasible and what will bring the most value to the business at later stages.
• Considering that an iterative approach works best for data science projects, this project was the major step in preparing NeedEnergy for further production implementation of predictive analytics.
• NeedEnergy is a part of the Techstars Web3 accelerator class of 2023.
• The company has been selected to join the Google Startups for Sustainable Development program.
• NeedEnergy was named as a top 15 most innovative startup in the world by the World Energy Council and German Energy Agency (Dena) through the SET awards.