How do you quickly start to monitor Data Quality without any tools?
Our client, a French Investment bank, wanted to improve their Data Governance, including Data Quality. They were investing heavily to achieve this goal.
There goals were not being reached as quickly as they wanted. A key challenge they faced was the organisational culture; it was a huge organisation, with many divisions, who often had conflicting goals.
This often led to big decisions taking 12+ months, especially when selecting software vendors.
The CDO couldn’t wait 12+ months for the organisation to choose a Data Quality software provider so they approached us to build a tactical solution.
How do you implement a working Data Quality solution from scratch?
With Alteryx already in use within the organisation, we chose that software to build a data quality monitoring tool.
Firstly we had to understand what the CDO meant by Data Quality. This defined the scope of the tool.
Data Quality was defined as up to 7 different specific quality checks per data source/field. It was data source dependent on which of the checks were required on which fields.
With every data source and every field being different, this meant the checks were bespoke. Therefore it wasn’t possible to build a tool where a random source is plugged in and a quality score comes out.
The tool had to provide, in a simple manner, a way for a data quality technician to write custom checks.
The output did have to be consistent.
Regardless of the data and check, the output had to aggregate and disaggregate, depending on the reporting requirements.
Alteryx is able to output to Tableau, so we created “data quality” data sources and built Tableau data visualisations providing data quality scores.
What happened next?
To build the tool took a few months for 1 consultant. After a few weeks Alteryx training, their offshore team of data quality technicians were able to ramp up the data quality capability of the CDO.
This tool was in use for over a year while the organisation were making the decision on the strategic data quality software.