In 2006 Clive Humby, the architect of Tesco's Clubcard said "Data is the new oil. It's valuable, but if unrefined it cannot really be used. It has to be changed into gas, plastic, chemicals, etc to create a valuable entity that drives profitable activity; so must data be broken down, analyzed for it to have value."
The interpretation of this analogy has shifted somewhat, initially meant to describe data as the most valuable resource on the planet, but since used to more negatively align internet giants such as Google, Amazon and Facebook with the actions of oil cartels, hoarding a scarce resource with the intent of profit maximisation.
This is where the analogy begins to breakdown. Unlike oil, data is not gone once it is consumed, it can be reused over and over, simultaneously even, and by different parties. Further, a consumer could even pay for a service with their data, but in doing so they have not depleted their reserves of data.
A tech company building a massive dataset is not hoarding an asset in the same way that an oil rich producer might, rather, through network effects they are increasing the value of the data.
What are you doing to increase the value of your data?
What data sets are you combining?
What new patterns and relationships have you uncovered in your data that have guided your business towards new opportunities?
If your organisation is not making insights and gaining full value from your data the chances are you are leaving the door open for your competitors to take business away from you.
GDPR
The General Data Protection Regulation (GDPR) came into force in the European Union in March 2018 with the intention of giving consumers more control over the data being held about them and the way that data is used. Companies found to be holding data on EU citizens without appropriate consent or even processing it in ways that were not consented to could receive significant fines.
This regulation served to shine a spotlight on personally identifiable information (PII) in terms of how it was being stored, where it was being stored, what was being done with it, and how easily it could accessed, updated or deleted.
A typical finding for many organisations was that PII existed in many different data sources across the organisation. Retrieving it was a time consuming manual process, updating it was more troublesome and using it was unreliable because discrepancies across data sets confuses which version of the data is accurate.
As a result many companies engaged in last minute projects to upgrade their systems, integrate data sets, create single customer views and much more.
This imposed change does also create opportunity. Think about your own organisation. Could you create a single customer view? Could you update all your systems while entering data only once? Could you accurately identify the same customer in various databases or do formatting differences, typos and address changes prevent you from being able to join up your customers in this way?
GDPR could be the catalyst for change that allows your company to open new opportunities, better understand your customers and increase revenues.
Data Quality
Duplicate records, formatting differences, redundant reference data and outright errors are just some of the problems likely to be present in your data. Without remediation these problems can grow to the point where your data is no longer an asset. Instead the amount of time your team spend on workarounds and manual processes means your data is a liability that is killing productivity and morale.
Fortunately the tools exist to allow you to profile your data to discover where problems might exist, master your data to establish a single version of the truth and remediate your data to achieve high quality, trusted data on which your business can base decisions.
Data Driven Decision Making
With high quality data you can produce dashboards and reports that your teams can use to detect and respond to real time scenarios. Perhaps you have received an inbound phone call from a long standing customer who makes frequent, high value purchases. Wouldn't it be great if you could see on one screen their past order history, the high value content of their current shopping basket on your website, some recommended products to offer based on buying patterns of other customers with similar past order histories and a note advising you to "say thank you to the customer for being a loyal customer for 10 years, a complimentary gift card will be in the post".
Or perhaps your buying team are reviewing sales data that demonstrates that due to increased Google Adwords competitors the margin on certain products versus the customer acquisition cost just isn't viable.
Or perhaps you identify that your mobile app is great at converting users in to sales if users are 30 years old or younger, but hopeless if they are 50 or over, prompting you to offer them a different user experience, tailored to their needs.
It doesn't only apply to customers though. You can use data to automatically expand or shrink your cloud based infrastructure based on usage patterns. You could improve your project estimation accuracy by analysing real data on real projects and you could even improve your processes by using data to identify bottlenecks so that they can be eliminated.
A/B Testing
One type of data driven decision making is A/B testing. A/B testing will allow you to validate a hypothesis by ensuring that the change you are making results in improvements in your defined indicators.
For example, perhaps your conversion rates are below industry standards and you want to fix that. You task your UX team with improving your checkout process. Once the dev team have made the necessary changes you do not get rid of the old checkout process but instead you run the two side by side, routing 50% of your users to the old checkout and 50% to the new checkout.
You let both versions run for a period of time and you log sufficient data about every session so that at the end of the experiment you can make a statistically sound assessment about which version is most effective, and then you move to another round of refinement.
However, it is no good if it takes a huge amount of time to deploy your A/B tests and/or process the results, otherwise your test results risk being irrelevant by the time they are available.
Therefore your software needs to be architected to support such tests, your processes need to be efficient enough that you can get your tests into production easily and the data you gather needs to be accurate and accessible.
AI/ML
Machine Learning (ML) is a subset of Artificial Intelligence (AI). Specifically, ML can be considered as one of the techniques used to achieve AI, where machines are used to analyse potentially vast amounts of data and learn, without constant supervision. By providing a machine with training data that allows that machine to understand the expected outcome of a decision, the machine is able to learn how to weight the input variables it receives in order to make decisions yielding the correct outcomes with the greatest degree of accuracy on aggregate.
Once the training is complete and sufficient accuracy has been achieved, the machine can begin processing live data and suggesting or even taking decisions.
When your data quality is high and your ML models produce accurate results data driven decision making can also be automated where appropriate, giving terrific ability to scale certain operations rapidly.
Customer Centricity
An area of rapidly growing interest is Customer Centricity, which has a significant dependency on knowing a customer's Customer Lifetime Value (CLV). When your data allows you to understand your relationships with your customers over their lifetime you can use that information to determine not only how much you spend acquiring customers, but also how to target your marketing and customer service towards those customers who most deserve the special attention and less so towards those who are unlikely to demonstrate brand loyalty.
Touchpoints
Cleansing and mining your data is not the whole story, in fact it is not even the beginning of the story. The first stage is gathering data. If you would like to optimise your customer experience then the ideal approach is to deploy sufficient touch points to communicate with your customers to ensure that when they need information to move along the buying journey you are able to provide it to them, regardless of the time of day or the channel they happen to be using.
Each touch point should be producing sufficient data for you to both service this customer and optimise your customer journey for other customers.