South Australia’s Department of Environment & Water migrates their entire data analytics
Apr 27 2021
South Australia is the driest state on the driest continent, yet it has one of the world’s largest artesian basins in the world, stretching 1.06 million km². The Murray River Basin supplies irrigation for large-scale diverse agriculture and supplies drinking water to the 1.3 million residents living in the City of Adelaide. The Department of Environment & Water (DEW) is responsible for the vitality and sustainability of this water ecosystem which involves monitoring and managing a lot of data to ensure fair water sharing, flood control, and maintaining water quality through controlling barrages that open the lower lakes to the ocean.
In 2015 the Premier of South Australia mandated an open data policy requiring all public agencies to make their data accessible by the public. Peter Baylis, Project Manager at the Department of Environment & Water, said “Around the same time, we had a public scientist requesting data on the salinity of water as it passes through the system. While we were collecting lots of data, our old system was not capable of analysing and displaying the information in a meaningful way. After an in-depth search, we still could not find the data he was requesting. It was clear that we needed something modern and intuitive so our employees and other stakeholders could get what they were looking for in a timely manner.”
DEW went about the next logical step of upgrading their data management system but determined their aging legacy system couldn’t support Open Data or provide the Australian Bureau of Meteorology (BoM) and the South Australian State Emergency Service (SES) with real time data for emergency flood management. “This was a timely and painful exercise, and the data was unreliable,” said Peter.
Another challenge that DEW and many of its partner agencies are facing is an aging workforce on the verge of retirement. These personnel hold vast knowledge and a deep understanding of the idiosyncrasies of a legacy system that come with experience. The new generation has been raised on cutting edge technology that is intuitive and user-friendly; learning a legacy system is not only unproductive, it’s demotivating.
The overall legacy system’s poor performance came to a head during a severe weather event in 2016 that resulted in a statewide power blackout that caused a major disruption to the water data network and security, followed by flooding from the storm. “This was a wake-up call for us”, said Peter, “There was no redundancy in the system to deal with a simultaneous power failure and flood event, and we only had one experienced IT person who was able to jerry-rig a solution on his laptop to access the critical data we needed to manage the flood waters.”
In the search for an intuitive and reliable data management system, DEW developed a comprehensive wish list of 117 features from all their stakeholders and invited three companies to demonstrate what they could satisfy on the list. “It was by far the most comprehensive two hour showcasing of the Aquarius capabilities, but DEW has an enormous responsibility in managing the Murray River Basin, and what they were asking for was an absolute necessity to doing the job right.” said CEO of Aquatic Informatics, Ed Quilty, who worked with the project team to ensure all the requirements would be met.
With over 3,200 monitoring locations and 17,000+ data sets, DEW delivers critical information to several external stakeholders such as: BoM, SES and Murray-Darling Basin Authority (MDBA). There are also several DEW business units working with that data, including surface water operations, surface water science, groundwater operations, groundwater science, and IT. Needless to say, the enormity of the project and the importance of getting it right required two years of product development and another year for deployment to all stakeholders.
The complexity of the project required the use of the full Aquarius software suite to acquire, process, and model DEW’s data in real time. Dedicated team managers from DEW and Aquatic Informatics migrated the old data to the new system to ensure no historical information was lost; this would prove especially advantageous in using the new program’s trend and prediction analytic tools. The project team also worked to configure the functionality to meet the specific needs of DEW and its stakeholders.
A critical component of the project involved the enablement of real-time decision making for the operation of the Goolwa Barrages. The barrages play a vital role in South Australia’s ability to provide fresh water for irrigation. Comprised of five low-head diversion dams with gate control, the Goolwa Barrages are used to reduce salinity levels in the lower reaches of the River Murray and the associated lakes, as well as limit reverse flow of seawater into the lakes during storms, high swells, and high tides. Prior to this project, essential calculations used in the operation of the barrages were being done in Excel and then emailed to South Australia Water (SAW) operators at Goolwa. The SAW operator then opened the gate, and emailed DEW to confirm which gate or gates had been opened. Since moving to Aquarius, SAW operators now access a new Goolwa barrage calculator through a real-time dashboard that is easily accessible. Prior to Aquarius this process took three hours; now all data and actions are in real time, and both parties have visibility into gate status through the shared dashboard.
One key issue the new software was required to solve was the ability to quickly process large amounts of data, and to subsequently make said data available to stakeholders in an easy-to-understand format. “The old system limited how many surface water sites we could poll during a severe storm event and also the frequency at which we could pull the data. This meant we sometimes had to make a difficult compromise regarding which sites would be updated during storm events, and which ones would miss out on the regular update cycles” said Peter. Access to all data during times of crisis enables better, more informed decision-making. Access to all network data was a critical requirement for Murray river operations, and other surface water sites during significant rainfall events. The new system allows DEW to poll all surface water sites at five-minute intervals with no loss of data, enabling SES and MDBA to act quicker and have confidence in the accuracy of the data.
Auditing and tracking were another important area that Aquarius was brought in to fix. The old data review and approval process was time-consuming and as a result would sometimes be overlooked, which compromised data integrity. The new program automates this process and gives field managers the ability to audit grade thresholds for uploaded data on a monthly basis. “Now when publishing data to external agencies like BoM or our internal customers, we have greater confidence in the information presented and our ability to trace any adjustments back to the source or raw data” said Peter. This is significant when the MDBA or other Commonwealth agencies question the provenance of the data and processes around data governance.
The new software is designed to automate workflows by extracting data, turning it into meaningful information, and making it available to stakeholders that require it to do their job. By automating processes, human error from manual entry is reduced, and by using complex algorithms the data becomes more meaningful. In addition, the intuitive user interface for the Aquarius platform makes it easy for anyone to input or retrieve data regardless of their device. This is especially valuable for the next generation of employees to easily access information from anywhere and draw insight from the data to improve the management of water resources, without having to learn how to navigate a legacy data system.
“Now I feel confident as I retire, that DEW’s data is in good hands and that the new employees have the right tools to ensure the Murray River Basin continues to be sustainably-managed for generations to come,” said Peter.