Data ranges from dirt to diamond, from the inconsequential tidbits cluttering databases to the most valuable strategic information for the relevant sector. Banking, supply chain, credit/debit cards, healthcare, government, research, online retailing or whatever. We do not usually destroy data, other than by accident. Like archives of documents, it may be required for checking or forensic purposes in the future or the sector has an obligation of data retention for years.
“Today there is a pervasive culture of fast-moving business, driven by the need to be customer-centric, innovative and agile. Organisations of all kinds are more collaborative, which in turn is based on the technology and more and more on the data. Data-driven decisions have become the norm, frequently automated within business rules”
The growth of data is one of the smaller challenges of our digital age. It simply means investing in data storage. Cloud or in-house, commodity data storage (spinning disks) is the staple, with RAM and SSD at the top end where live data has to be processed at speed. Big Data is a simple term, like Cloud, but shares an irony of complexities layered behind it. Once that was known as The Three Vs — Volume, Velocity and Variety. At least two Vs have been added: Value and Veracity. This column suggests a sixth — Vital. As with all kinds of information, some elements have critical relevance and others are rubbish, in the context or anyway.
Big Data is complex, in spite of the simple term. Typically, it involves unstructured data and concealed elements that might — on analysis — turn out to be valuable or of no value. Big Data depends on analytics for value, mainly through a team of different skills. The same applies to Simple Data, commercial or other, which is mainly comprised of customers or subscribers and their purchases — and preferences — together with partners, suppliers and every party that the enterprise has engaged with. Business analytics teams seek patterns and insights for better marketing and refined processes and disruption in the target markets. Their aims are developing ideas for new products, services and even entire new business models.
Pervasive culture
Today there is a pervasive culture of fast-moving business, driven by the need to be customer-centric, innovative and agile. Organisations of all kinds are more collaborative, which in turn is based on the technology and more and more on the data. Data-driven decisions have become the norm, frequently automated within business rules. Combine that with the customer-centric business norm and it means that change and advances are now evolutionary rather than the big project, big step forward approach of a decade ago.
In general people are accustomed to apps and mobile working and in turn business is more willing to accept the new ecosystem of apps, cloud and certainly the power of analytics. We live in a hybrid tech world and that means the data is spread rather than collected in formal databases. That in turn means business analytics tasks are more difficult because data is no longer confined to monolithic, old-fashioned databases. For example, scanned documents convert paperwork into digital elements, which means a challenge to contents analysis. Another challenge is that data sources are now mixed media: recorded speech, photos and video not to mention all of social media.
This is the ‘new’ challenge for data analytics. There are many varieties of data in the digital world—and growing. Variety may be the most challenging of the six Vs, relevant to simple data as well as Big Data. Business data analytics is usually thoroughly pragmatic and focussed largely on customer-centric objectives and better and faster decision making. The idea is to understand and predict customer behaviours to improve their experience and so enhance business performance. The difference today, is the variety and volume of data sources, spanning transactions, loyalty cards, other and multi-channel interactions, social media and other third-party, customer-related information that are all contributing to a more complete picture of customers’ preferences and demands.
Other disciplines may be more imaginative and creative, from astronomy to sociology, from all types of science to CERN, not to mention government intelligence and state cyber-warfare. There is a vast, possibly limitless opportunities for the application of Big Data analytics. Techniques inevitably trickle down to the more mundane world of business, especially online, and Not So Big Data.
80/20 rule
In mass data analytics, the ancient 80/20 rule inevitably applies. The team can do a first or indeed several cuts at the mass of data and achieve potentially useful insights. Not only is there no definitive need to cleanse or prepare the data—some considerable part of the value actually resides in the sheer comprehensiveness of the data resources, its totality.
Once they have got some insight or clues, on the other hand, of course a more targeted analysis is the way to go or indeed a series of alternative slice/dice approaches to see if anything more detailed or more nuanced might be discovered. That is after all the scientific method: hypothesis, test, informed by results, re-calibrate and try again. Error is informative.
The objective and the methodology of business analytics can be summed up in a little mantra—Find Any, Find Many, Find All. That is about it. Patterns, deep insights, infographics, modelling of possibilities all then follow. But the other major challenge of masses of data is that some of it has owners—with rights. That can be a little inconvenient because there are all sorts of laws and regulations and Best Practice and voluntary codes and ‘the right to be forgotten’ that all have to be provably built in to the analytics engines. The advent of GDPR means the regulatory framework has changed and a complex set of business rules is necessary.
There is a new mix of skills required to continue to exploit the potential of Big Data and the less massive data warehouses. The team needs data scientists and statisticians to design the analysis and predictive modelling. But there is an essential need of people with the domain knowledge. Business analytics does not result in a decision-making system. The insights are only part of the overall business process. Any organisation, business or otherwise, needs senior people to decide on what is to be achieved by any insights, how valid for the business or organisation they are and what actions should ensue. To be of value, data analytics needs an executive process to take the logical decisions that arise.
Sophisticated automation
Data analytics is a growing resource to feed Artificial Intelligence (AI) and its impact on business. This column is convinced that AI is a super-sophisticated form of automation rather than an independent ‘mind’. In that context, AI thrives on the analysis of data. AI can even be deployed to perform data analytics. There is a global swing to mining and analysing data and a recognition that recorded data is a resource and potential wealth in business and other contexts.
So far it is confined to large organisations which can afford it. But there is an inevitable progression to data analytics services, likely to be cloud services since of their nature they will be massive. AI can increase the power of data analytics services. It is likely that such services can penetrate down to ‘medium’ sized organisations that have amassed large quantities of data. The vision of the future is high-tech, combining state-of-the-art machine intelligence and AI with instant computing across business sectors and all significant organisational activities.
But the key is data — diamonds or dirt.
Ltd
