Metadata Collection and Organization in Wastewater Treatment and Wastewater Resource Recovery Systems

This Scientific Technical Report demonstrates meta-data strategies for water resource recovery facilities (WRRF), including the essential data validation with machine learning and traditional methods. The report is the result of extensive work from many water data experts and a utility focus group, with the dedicated application of WRRF data.Emerging trends in artificial intelligence and machine learning project unforeseen possibilities in managing our WRRF’s. Hope is built upon the large data volumes collected with high frequency from both existing sensors, as well as new uncoordinated data sources outside the fence. The data variety however makes it challenging to reuse data, especially when the purpose changes. Without a proper data description (meta-data), modelling and autonomous digitalization will be difficult, and likely remain a vision. Likewise, quantified data quality are key meta-data to decide when data are fit-for-purpose.The report aims to fill the gap of how meta-data can be used in practice to leverage the value of data in a WRRF context. The report describes existing methods and systematic methodologies to collect and reconcile meta-data describing signal generation, signal quality, and contextual meta-data. The sometimes ambiguous data terminology is clarified with real WRRF examples to endorse adoption in practice. Guidelines for data quality assessment is a central part and cover both standard sensor validation protocols, as well as a separate chapter on data analytical techniques. The latter serves as a smorgasbord with mechanistic and data-driven algorithms for online sensor quality assessment.The report is intended as a reference guide for the practitioner who aims at future proofing, but also maximizing, the current use of today’s recorded WRRF data. The content bridges theory with current practices and provides a base tool for the WRRF data practitioner.

In recent years, the wastewater treatment field has undergone an instrumentation revolution. Thanks to increased efficiency of communication networks and extreme reductions in data storage costs, wastewater plants have entered the era of big data. Meanwhile, artificial intelligence and machine learning tools have enabled the extraction of valuable information from large-scale datasets.Despite this potential, the successful deployment of AI and automation depends on the quality of the data produced and the ability to analyze it usefully in large quantities. Metadata, including a quantification of the data quality, is often missing, so vast amounts of collected data quickly become useless. Ultimately, data-dependent decisions supported by machine learning and AI will not be possible without data readiness skills accounting for all the Vs of big data: volume, velocity, variety, and veracity.Metadata Collection and Organization in Wastewater Treatment and Wastewater Resource Recovery Systems provides recommendations to handle these challenges, and aims to clarify metadata concepts and provide advice on their practical implementation in water resource recovery facilities. This includes guidance on the best practices to collect, organize, and assess data and metadata, based on existing standards and state-of-the-art algorithmic tools. This Scientific and Technical Report offers a great starting point for improved data management and decision making, and will be of interest to a wide audience, including sensor technicians, operational staff, data management specialists, and plant managers.