Over the years we have completed many data and insight projects. Sometimes in the early stages of these data projects it can be a challenge to manage stakeholder expectations. Data sourcing and associated analysis can be particularly time consuming. Much of this work isn’t visible, hence the data insights iceberg.
An iceberg is a good way to represent these projects. A Data Insights Iceberg project. Where the bulk of the work happens unseen, under the water.Download Iceberg as PDF
Fortunately not all of this “underwater” work is necessary for all data insight projects. Each firm has different levels of data infrastructure.
What is happening under the water of the data insights iceberg?
Insights and Analytics projects require a key ingredient. Without this ingredient the project will fail before even starting. Good quality data.
However, before even getting to the data, it’s key to understand what the clients / users really want. Generally it is difficult for people to articulate what information would be useful for them. What it is they really want. Often this is because they don’t know. They have a feeling they want something like X but until they start to see X they genuinely don’t know.
Before knowing if you have good quality data, those likely “fluffy” requirements need working back into likely data source systems. Next is to source the data from the identified systems. And sourced with real data, not test data. If you show a business user a dashboard with test data the only feedback you’re likely to receive is, “That number is wrong”.
Sourcing data can be time consuming. Without full cooperation from the data providers it can be very time consuming.
Assuming all goes well to source the data, next each data set requires analysing and understanding. Does the data look correct? Does it match back to the source system? How should it be used? Are there business rules to be aware of? How does it join to other data sources?
There are many questions about the data and they all require an answer. Understanding the data is important to provide accurate insights.
These data sources then need joining / blending. This generally means writing ETL processes to map the data appropriately. This also takes time, it’s key the data coming out at the end of the joining process is accurate. These ETL outputs form the base for producing the business insights.
Converting the data into useful business information only really begins once the data steps are complete. Usually Insights are presented in dashboards. These are generally the outputs the business want to see. These are the piece of the iceberg above the waterline. As this is all the users see this is how your entire work efforts will be judged.
Managing stakeholder expectations
Due to all of the work happening below the waterline it can be difficult to manage stakeholder expectations. In organisations where the data infrastructure is relatively mature it is possible to rapidly create insights. Therefore expectations are far simpler to manage. If you can start with good data it should be possible to quickly produce insights.
However, for those organisations with a less mature data infrastructure, it can be more challenging. Projects that start needing to source large amounts of data tend to be tricky at the beginning. A significant amount of time will be spent at the base of the iceberg, working below the waterline. When the work is less visible to the business stakeholders it becomes more important to manage expectations.
The best option is to create a matrix showing known data sources. The matrix should:
- Indicate the source provider
- Flag whether the data has actually been sourced – sometimes this can take months
- Indicate whether it has been analysed
- Show the quality of the data contained
- Flag any issues found
This is likely to be the best option of showing progress to senior management in the business. For many stakeholders a powerpoint slide is easier to read and understand than showing an ETL workflow!
Ultimately communication has to be solid within these iceberg projects. It is likely not the developers fault things are taking longer than expected. However it also often easy to blame the developer. Therefore managing stakeholder expectations could be the difference between project success and project failure.