Data Readiness within the IoT for Energy

Posted on

< Project Overview >

Introduction

Within the Energy theme there was much discussion on the need for some unified set of  tools and methods around data cleaning, i.e. if intending to utilise the outputs from swarms of  IoT sensors then it must be understood that a fraction of them at any one time will be outputting  anomalous or corrupt data. There must therefore be a clear strategy around data cleaning,  though this should actually be a discussions around Data Readiness, i.e. how will we make  the data that is supplied from IoT ready, for consumption in a way that is reliable, repeatable  and with which we can have confidence in results gained. 

As part of the process around data readiness we need to understand what standards are in  use already within the sector and what methods may already be widely utilized in relevant sectors. From this this project intended to produce two relevant outputs of use by the consortia.

Project aims

The main aim of the project is to identify methods to significantly improve understanding how IoT (anomalous, dirty, even corrupt) sensor data can reliably be integrated into system decision-making processes. This project seeks to bring together and share global best  practices, with a view to specifying a toolset for researchers that an activity could develop. 

What was done?

Through the work of the recruited researcher the following activities were undertaken. 

A survey the metadata requirements for IoT device sources in energy was performed to create a dataset for future analysis. From this a catalogue of currently available data standards was  created “Existing IoT metadata standards”, to understand useful transforms and currently  adopted standards. Following this the second deliverable was created, “Applicable data  cleaning methods and schemas for their use”. This addresses the aspect on available data  cleaning methodologies and how to embed the information in an interoperable way in the IoT  landscape. It built on the results of the report on metadata standards mentioned above, and the  application of the FAIR principles to an IoT Smart Energy system. Finally, this report outlines how a skeleton architecture might look, which includes capturing the application of  data cleaning methodologies at relevant levels, while maintaining a FAIR approach to improve or achieve data readiness in the IoT Smart Energy domain. 

The partners within the project disseminated the survey and contacted relevant stakeholders of which they were aware to complete the survey as well as contributed to the development of the deliverables.

Results

The deliverables of the project highlighted how there are a number of well understood and utilized data standards already within the sector and that overall it is important that we note that IoT overall is developing well in this area and ‘Energy’ will be just another usecase not something special. 

For data cleaning the available tools and techniques as well as the application of FAIR  principles around IoT data will in some areas be challenging though not insurmountable. We highlighted a possible future architecture to improve the data readiness within an IoT for Energy domain. We did not ourselves continue to a follow on project.

Deliverables and other tangible outputs

D1 – Smart energy IoT device metadata survey 

D2 – Existing IoT metadata standards, Drescher, 2019 

D3 – Applicable data cleaning methods and schemas for their use, Drescher, 2019

Impact

Mr Michel Drescher was employed for the duration of the project on an uplifted contract beyond his previous part-time status.

Next steps

None currently though work within LEO includes discussions about the use of MQTT that was mentioned as one of the leading contenders for utilization in the energy domain by D2.

Lessons learned

The deliverables give a clear indication of the current state of the art in the area to the extent  that even now we are seeing activities in the IoT space realizing that MQTT is a good an  appropriate standard for messaging within the domain.

Surveys are extremely hard to engage stakeholders that are unclear of the value even if  they are well engaged with the survey developers beforehand in other areas. It would have  been more useful to hold more targeted webinars/workshops specifically around these topics. Face-to-face meetings always generate better results though if the right people can be encouraged to attend. 

Further dedicated resource to allow the researcher to spend more dedicated time on the activity, 30% is not a large amount when compared to other activities on which they were engaged.

What has Pitch-In done for you? 

Pitch-In allowed us to research into an area in which we had no specific current activities but which gaining and understanding of the local landscape was essential in the longer term where we consider how smart energy and systems may develop in the future. This  is most particularly important when we consider how data from sensors and system may be utilized in real time for network functions including detailed business relationships which must  be executed correctly and the data from which must be ready for utlisation both within the service itself but also when considering arbitration of disputes, breakdowns or system errors.

Project lead:

Professor David Wallom – University of Oxford

Partners:

  • The University of Oxford
  • The University of Sheffield
  • The University of Newcastle