Test Case 21

Performance characterization of new equipment and communication technologies

Identification

ID

21

Author

Felipe Castro

Version

1

Project

ERIGrid 2.0

Date

04/12/2020

Test Case Definition

Name of the Test Case

Performance characterization of new equipment and communication technologies

Narrative

The aim of this test case is to assess the performance of equipment used for communication in smart grids and power systems, such as Remote Terminal Units (RTU). The characterization of this performance is one of the major goals of this test case, taking into account the different communication technologies that could be supported in such devices (e.g. communication protocols) and the stochastic nature of communication effects.

In particular, the experiments in the test case allow characterizing the performance of communication equipment by means of communication delay calculation and the rate of dropped packets between two RTUs. Through this experiment, it is possible to determine the mean value and the standard deviation of the communication delay between two devices, together with the packet loss rate for different polling times.

Function(s) under Investigation (FuI)

The characteristics of two communication devices exchanging information. The delay of the information transmission and the loss packet rate, for different communication protocols.

Object under Investigation (OuI)

At least two communication components connected together.

Domain under Investigation (DuI)

Information and Communication Technology (ICT)

Purpose of Investigation (PoI)

Verify whether the performance of the devices is adequate for the applications for which their use is intended. Specifically, verify that the mean delay and packet loss rate of the communication complies with the minimum expected requirements for the application for which they will be used.

System under Test (SuT)

In order to execute this test, the following components are needed:

  • Real time simulator, such as OPAL-RT
  • At least two communication devices, such as remote terminal units (RTU)
  • One communication emulator, such as ATTERO
  • Ethernet connection cables (RJ45)

The test consists in the connection of the Real Time simulator to both communication devices, and the connection of these devices to the communication emulator.

Functions under Test (FuT)
  • Capability of the real time simulator to transmit data under different communication protocols (e.g. MODBUS, IEC 60870-5-104, IEC 61850)
  • Capability to generate and transmit data by the real time simulator with different characteristics (data profiles)
  • Data transmission from real time simulator to the OuI.
  • Capability of Communication protocol translation by OuI.
  • One communication emulator, such as ATTERO
  • Ethernet connection cables (RJ45)
Test criteria (TCR)
  • The real time simulator needs to send a data profile periodically using a certain pre-defined communication protocol.
  • Due to the stochastic nature of communication phenomena, the data profile needs to be sent periodically enough times to be able to obtain statistically valid results.
  • The real time simulator needs to register the time for which data is sent and for which data is received, in order to calculate the communication delay of each data value.
Target Metrics (TM)

Data profile:

  • The data profile needs to be arranged in a way that it is possible to determine the communication roundtrip time (delay) of each data value. A series of increasing integer values is proposed with an increasing value change periodicity. For example:

In the previous figure, the constant α represents the reduction of the time during a particular value is sent. It is noted that the polling time of the devices must be less than the time duration of a sent value.

  • The delay and the lost packages have to be registered:

  • The delay for each successfully transmitted data value can be used to create a normal distribution of the delay probability (obtaining the mean delay value and its standard deviation):

  • The successful transmission rate can be calculated for each value sent from the profile, allowing to determine the minimum periodicity that can be 100% successfully transmitted

Variability Attributes (VA)
  • Periodicity for which a data value changes.
  • The delay between the OuI devices set in the communication emulator.
  • The amount of repetitions of the experiment to obtain statistically valid results.
  • The communication protocols involved between the real time simulator and the OuI devices, and between each of the OuI devices.
Quality Attributes (QA)

For delay calculation, at least have a packet loss rate less than 1%

Qualification Strategy

The PoI can be addressed by firstly determining which are the requirements in terms of Quality of Service (QoS) of the application for which the communication is being assessed. These requirements are to be determined by the user depending on individual criteria.

Secondly, the minimum periodicity for which a value is sent and for which there is a packet loss less than 1% can be assessed by following the Test Specification NA4-TC21.TS1.

Finally, considering the results from NA4-TC21.TS1, the QoS performance indicators can be obtained for different emulated delays by following the Test Specification NA4-TC21.TS2.


Test Specification NA4-TC21.TS2

This test allows determining and verifying the successful transmission rate of a data profile for different delay emulations enabled by the communication emulator. This would allow to verify the minimum periodicity of data value sent determined in TS1 (with 99% successful transmission rate), and determine if the successfully transmitted data is similar or decreases when including emulated delays.

Test Specification TC21.TS1

Determining minimum periodicity of data transmission for a successful transmission rate over 99%