Stitching data together to weave a smart fabric of huge business value
In association with InterSystems
In the space of a few years, businesses have leapfrogged from spreadsheets to data warehouses and into the cloud. At least that’s the theory.
In practice, most enterprises still have data all in many forms and locations – in spreadsheets, separate departmental or organisational databases, in data lakes and warehouses and in the hybrid cloud.
The picture is complicated by legacy systems that make data workloads expensive to transfer to the cloud, inhibiting innovation. Many of these remain on-premises. Analysts at IDC estimate that by the end of 2022 more than 90% of enterprises worldwide will be relying on a mix of on-premises/dedicated private clouds, multiple public clouds and legacy platforms to meet their infrastructure needs. And even though global tech consultancy Gartner estimates more than 70% of companies have now put some of their workloads in the public cloud, most organisations have much more to do.
Data sprawl is growing making it harder to extract value and innovate
This evermore dispersed and diverse data estate confronts organisations with substantial challenges when they want to manage, prepare and clean their data for integration and analysis. They have little choice in the matter. To excel in a digital world, businesses must become data-driven – able to act on insights extracted rapidly from data irrespective of its location. Yet to bring data together from many locations and sources for real-time analysis and insight is extremely tough, especially when data scientists are in short supply. All data has to be managed, cleansed and sense-checked before anything is done with it. It is a massive IT challenge.
Many organisations will seek to rely on a mix of solutions to achieve the necessary level of management and integration. In its 2020 Cloud Data and Analytics Adoption Survey, Gartner found 57% of respondents managed or expected to manage data using a hybrid approach of cloud service provider combined with on-premises data management solution. However, as businesses are using more and different clouds out of necessity rather than choice, they are only adding to the complications of data management. Each new cloud demands a new set of management tools; a new set of skills and the ability to adapt to different vendors’ updates, which can render applications unusable without intervention.
Poor cloud data management ramps up costs
One of the major problems with managing data in the cloud is the lack of detailed visibility, which leads to higher costs. Admittedly, some of these excess costs arise from over-provisioning in the first place, but many organisations continue using costly storage capacity when they no longer need it, or they have the same data in different places, racking up unnecessary costs. The major cloud vendors provide tools to address these challenges, but the reality is often they are clumsy and imprecise. Small wonder that Gartner predicts that by the end of 2024, 60% of organisational infrastructure and operations leaders will have public cost overruns that hit their on-premises budgets.
Businesses that are cloud-native such as fintechs should be further forward in their ability and agility in data management and preparation. But the problem they face is effective governance of their on-premises data, especially for regulatory purposes. This is a challenge that could inflict severe reputational damage and financial penalties if they fail to overcome it.
The smart data fabric is more than a comfort blanket
Despite the sprawling complexity of so many organisations’ data estates and the difficulties they have in managing and cleaning data from so many disparate sources and systems, more effective alternatives are at hand.
More specifically, the smart data fabric is an approach that draws data together for analysis and rapid insight. It places the most valuable data in the hands of those who use it, rather than requiring the constant intervention of IT.
A smart fabric will integrate many systems, employing a data platform to weave information together to meet business requirements for advanced analytics and the development of applications. By eliminating duplication and storing data for use only once, data platforms relieve organisations of cloud costs, including for example, the high cost of managed storage instances with the hyperscalers’ data warehouses.
Legacy applications and data remain in place, removing the need to ‘rip-and-replace’ existing technology, which enables companies to maximise their previous technology investments, including the data lakes and data warehouses they have built up in different locations.
The fabric unifies data from across the entire enterprise. In the financial sector, for instance, this enables real-time queries for regulatory reports or ad-hoc investigations and allows firms to track data lineage. In doing so, they not only satisfy regulatory requirements, they also generate reports and model risk quickly and relatively easily. It is why the smart data fabric is proving popular among forward-looking financial firms keen to address their key data management challenges and expand the capabilities of their risk management and regulatory compliance functions.
The data platform that underpins the fabric
The data platform weaves the fabric. Multi-modal database management systems process data in any format, providing a high level of flexibility. Embedded analytics execute analytic SQL, and provide business-user access to business intelligence, machine learning and natural language processing. Front-line employees can more easily visualise, analyse and interrogate live data, for instance.
Organisations can integrate advanced analytical capabilities into real-time applications to transform the efficiency of their business teams and power faster and more effective decision-making at all levels.
Teams can employ user-friendly applications to pull together masses of historical and live data for collation, analysis and insight. Thanks to embedded interoperability with other formats and systems, data platforms ingest many kinds of unstructured data, IoT and device data, batch or high-speed transaction data. As a result, they are able to run more scenarios on huge datasets and achieve sub-second performance in, for example, risk calculation or predictive modelling.
Yet rather than being a quick-fit solution that is just switched on, the smart data fabric provides long-term incremental business value. It is a phased organic solution aimed at providing business value on a big scale. The data platform that is its true power point enables businesses to simplify the growing complexity of today’s IT architectures, streamlining the preparation, cleansing, management and integration of highly variegated data. Organisations are able to put their most valuable data in the hands of the people who will use it, giving them new capabilities and insights to transform the entire organisation’s efficiency and innovation.
Redmond O’Leary is sales manager for Ireland at InterSystems
Subscribers 0
Fans 0
Followers 0
Followers