Chasing rainbows

It cropped up again last week; the dreaded statement from a BI professional

We must have a single version of the truth…

Anyone working in IT / BI / decision support / enterprise reporting / analytics etc. for more than a couple of months will have heard this, and for those of us working in the field for 20+ years we may wince slightly and wish that we had a pound for every time…

For many years I went along with the idea, and of course the principle is an enticing and appealing one. No-one wants to spend minutes, let alone hours, arguing with colleagues as to who has “the correct numbers” – it would be far more convenient if everyone adhered to the same definition of every business term, listed all their caveats and stuck to the same KPI calculations… as I’m sure we would all welcome free energy, a cure for all cancers and that beauty pageant favourite; world peace. But wanting something isn’t enough to make it happen, unfortunately, and “a single version of the truth” (“SVotT”) can be unpicked in so many ways; philosophically, psychologically and technologically to name but three.

The search for truth has dominated the lives of Eastern and Western philosophers over the past few millennia, and most have concluded that it’s a psychological illusion. Neuroscience is establishing that we can’t even establish a single truth within our own minds; patients with severe damage to the corpus callosum suffer from the split-brain phenomenon where the two brain hemispheres operate independently and unaware of each other; memory has been shown to be highly fallible and our minds fabricate large portions of the story we tell ourselves… even about ourselves! With such internal inconsistency, how can we expect to establish an objective external truth and then agree to it?

Operationally and technologically, within the ever-changing organism that is a modern business, how can a SVotT exist even minute to minute? See how many of the following questions you can answer when next handed (or handing) a report:

• What period does the report cover – how accurate is the definition of this (to the day, hour, minute)?
• Which functions of the business are included/excluded?
• Which products / brands does it cover?
• Which customer segments / distribution channels / sales territories does it include or exclude?
• When was it prepared – has anything changed since which could have changed its content?

These are just the first five which come to mind, and I doubt anyone could answer all five with certainty on any corporate report. Typical caveats – if you ask these questions, and press for conclusive answers – will include:

• “Up to the end of last week… but I’m not sure exactly what time…”
• “Ah yes, good point, we have excluded acquisitions/disposals from the last month…” (retailers, for example, are dogged by an obsession with “life-for-like” reporting… itself having many meanings!)
• “Of course, we have excluded/include own-label products…”
• “Oh, and it doesn’t cover new customers who registered over the weekend…”
• “I ran it yesterday, so – yes – it could have changed today after store updates…”
  etc.

I don’t mean to denigrate such caveats, rather to highlight their necessity and the absurdity of the notion of SVotT. That said:

• It is worthy and noble to strive for a consensus on how to record and report key measures
• Organisations do need an internal lexicon which defines and consistently applies definitions of terms like “Gross Margin”, “Lifetime Customer Value”, “Stock at Hand” etc.
• Certain reports – legal, regulatory etc. like annual accounts and disclosure notices – must have consistency and rigour applied

In such cases, a degree of precision should be lost in order to achieve accuracy and consistency over the long-term (note to self: write a piece on precision vs accuracy some time soon).

My advice to all anyone faced with a request for a SVotT is to recognise – and, if feeling brave, try to explain – that this is a psychological illusion; a mirage, or better still a rainbow. For like a rainbow, a “single version of the truth” may look very beautiful but, however hard you pursue it, it will always remain out of reach.

Far better it to accept that SVotT is unattainable in practice, but is a valuable guiding principle. Adherence to the principle, rather than the non-existent object, encourages:

• A serious approach to Data Governance and stewardship, recognising that organisational data is a precious and rare resource (when clean, accurate, timely, consistent etc.)
• A primary store for organisational master data, and controlled disciplines around Master Data Management, including its use in all regular reporting (and ad hoc analysis, where appropriate)
• Rigour in identify Data Quality weaknesses – not just instances of erroneous data but serious enquiry into both sources of such error, and their systematic removal through process improvement and effective control

These principles, employed broadly and managed effectively across an organisation, help multiple versions of many truths flourish in a data-driven culture where different perspectives – valid, insightful and sometimes counter-intuitive – enable the organisation to better, faster, more thoroughly understand its challenges and opportunities, rather than follow its traditions.