joshua-sortino-LqKhnDzSF-8-unsplash (1)

Using Big Data Analytics to Build Smart Grids

With fast technological advancement and cloud computing, large amounts of data is being generated every second. Utility providers can conduct a rational, effective and efficient analysis of this data, to bring value and benefit to their daily economic activities. 

With depleting fossil fuel reserves and de-carbonization demands of numerous states in Europe, utility providers decided to deploy effective smart grids. These solutions help in accelerating the speed of electrifying human society by harnessing the power of renewable energy sources.

The traditional electric meters in distribution systems could only produce small amounts of manually collected and analyzed for billing purposes. Now, smart grids generate data that companies can use to determine customer behavior, demand, and energy generation optimization. 

Big Data

Big data is a large volume dataset that consists of various categories and complicated structures that requires a novel framework and techniques to extract practical information.

Following the 5V model, big data has the following characteristics: 

  • Volume
  • Velocity
  • Variety
  • Veracity
  • Value

Smart grids are an abundant, intelligent source of information as they cover data from the process of electricity consumption, generation, transmission, and distribution. The dataset includes electrical information from: 

  • Distribution stations
  • Distribution switch stations
  • Electrical meters
  • Non-electrical information:
  • Marketing
  • Meteorological
  • Regional economic data

Here are some state of the art intelligent devices that help in collecting data in smart grids:

AMI – Advanced Metering InfrastructureIntegration of data management systems and communication systemsRemote meter configuration, dynamic tariffs, power quality monitoring
PMU – Phasor Measurement UnitReal-time data measurements of multiple remote points used for synchronization (takes 30 to 60 samples per second)Measuring electrical waves on the power grid
WAMS – Wide Area Monitoring SystemActs as an application server to collect incoming information from PMUsEnsures the dynamic stability of smart grids
RTU – Remote Terminal UnitA device controlled by a microprocessor to transmit telemetry dataCollecting information regarding the system’s operation status
SCADA – Supervisory Control and Data AcquisitionCollecting data automatically and manuallyMonitoring the system, processing events and alarms
IED – Intelligent Electronic DeviceTo monitor and record status changes in the substation and outgoing feedersA combination of different relay protection functions that measure, record and monitor

Utility providers also need to take precautions while choosing the right analytics solution. Due to their immense volume of data, they need a solution that has: 

  • Scalability and flexibility
  • Latency
  • Computational complexity
  • Fault tolerance
  • Distributed storage capacity and configurations
  • Data processing modes
  • Data security

Big Data Analytics Implementation

Utility providers have to worry about terabytes of data. Therefore, managing big data integration efficiently requires a high velocity, scalability, and fault tolerance in data processing, storage and visualization. The various kind of analytics in smart grids are: 

1Signal analytics
Sensor signals, substation waveforms, line sensor waveforms
2Events analytics
Detecting, classifying, filtering, correlation
3State analytics
Real-time electrical state, grid topology, system identification
4Engineering operations analytics
Operational effectiveness, system performance, load trends.
5Customer analytics
Demand profiles, demand response, diversion analytics, customer segmentation

A utility company can take advantage of smart grids to better understand customer behavior and make strategic decisions. Since customers participate as end-users by using smart meters data, which offers them a better control over their consumptions. 

Utility providers deploy Demand Response programs that collect real-time information of the electrical demand-side management to calibrate the grid according to specific consumptions. Therefore, utility providers can also adjust their production curve to meet the demand efficiently and reduce the losses from overproduction. 

The main motivation behind DR is to improve customer engagement because utility providers can easily interact with the customers even when there is a power outage. 

Big Data Analytics in Smart Grids

Here’s how big data analytics in smart grids help utility providers in their daily routine: 

Detecting faults

The main driving force behind the construction of smart grids was carbon reduction and sustainability of the environment. Modern power distribution systems now employ distributed generator units, called microgrids,  to utilize renewable energy efficiency. Close distances between a generator and loads ensure reliable delivery of power and a decrease in power loss on transmission lines. 

However, the uncertainty in the power grid can increase due to the variable characteristics of renewable energy. Therefore, the best practice is to use Inverter Interfaced Distributed Generators (IIDGs) for their lower inertia. 

On the other hand, IIDGs pose a severe threat to microgrids if it becomes difficult to detect and resolve faults in a short time due to a limited current carrying capacity. 

Grid blackouts or severe weather conditions may trigger unintentional islanding accidents that compromise safety and cause technical issues. Artificial Neural Networks (ANNs) need to be trained with features to extract information from the differential rate of change of frequency signals to identify islanding accidents.

The Support Vector Machine (SVM) classifier can also have multiple features that extract information from system variables as an islanding detection approach; it also has a sliding window with an optimized width for high detection rates. 

Predicting maintenance

Distribution Automation (DA) focuses on the operation and system reliability at the distribution level of a smart grid. A DA is also capable of localizing and isolating faults that arise on the distribution system while reducing restoration time and improving customer satisfaction. 

DA collects its information from SCADA and AMI to monitor states and diagnose faults. Utility providers can take preventive measures with the help of a Pole Mounted Auto-Reclose (PAMR), which is an intelligent electronic device installed on the overhead lines of a distributed network. 

Since galloping of power lines can lead to structural and electrical failures, an intelligent data-driven model based on SVM known as Extreme Learning Machine (ELM) algorithm is perfect for an early-warning system to detect any upcoming risky events on the power system. 

Transient stability analysis

Utility providers are forced to operate near their secure limits to meet the increased demand for electricity, renewable energy sources, and a deregulated market. A Transient Stability Analysis helps in understanding the dynamic behavior of electromechanical and electromagnetic processes of a power system. 

TSA is used as a summarization technique to conduct pattern recognition and discover information from redundant measurements. There are two-contingency oriented Decision Tree frameworks used for the Dynamic Security Assessment that are trained with databases generated through simulations: 

1st Decision TreeFed with real-time wide-area measurements to identify potential security issues.
2nd Decision TreeProvides the online corresponding preventive control strategies to handle problems.

PMU and WAMS provide high-resolution datasets for engineers working for utility providers. Patterns of normal and abnormal operation can be discovered easily. However, utilities must deploy the random matrix theory to take full advantage of their massive power grid data.


Analytics in smart grids has made things easier for utility providers to manage their load and electricity generation procedures. Therefore, a secure and high-performance data analytics platform will go a long way in reducing costs, saving losses, and driving profits for your company. 

Aptimize offers easy integration of big data with its AI engine to help utility providers discover more efficient opportunities within their existing smart grids.

Share this post


Read also