Data architecture – I always think this topic sounds very grand and it conjures up images of very precise and neatly laid out diagrams. However, the reality is usually very different – it’s a hugely complex and messy subject and getting your data architecture right can be a daunting task. Data architecture forms the core of your digital platform, provides the key capabilities to enable your customer proposition, and translates directly to the bottom line through enabling you to be a data and insight driven organisation. So why is it so hard to get right, what are the considerations when building and launching a digital bank, and what does a good digital data architecture look like? To start, let’s rewind a few years….
Historically, banks were a bricks and mortar businesses – their business model was to serve customers and support the physical distribution of cash across society. Operating required a specific architecture characterised by large, monolithic applications and servers housed in their own IT data centres. Fast forward to today, and broadly, this model is still in operation – over time banks have layered in new channels, starting with ATMs, then telephone banking, followed by web, and more recently mobile – but these changes, although not simple or easy, have all been built on the same core processes and functions that were designed and built for a different era. Lipstick on pig is the phrase that comes to mind. This leads to a very fragmented and silo’d data architecture, with a lack of transparency over lineage and quality, varying levels of data integrity, and key data locked inside core applications. This model is not fit for purpose in today’s digital banking world, where there is an unlimited demand for answers and an insatiable demand for faster, better and more timely data and insights. When talking to business leaders and CDOs across the banking industry, nearly all of their key challenges are related to data architecture.
So how are a digital banks needs different, and how does it address these issues?
A digital bank has digital technology and data and insights embedded at every point across the full value chain - from systems of innovation that drive customer engagement, to systems of record that provide the core applications and through to systems of control – e.g. risk, finance and operations. By its very nature, it will capture and process a broader variety and volume of data due to the digital footprint and data exhaust that comes with digital applications – voice, clickstream, social, weblogs, third party, unstructured and external data. The data architecture needs to integrate this diverse range of data sources, process it, and then distribute through digital networks and interactions (e.g. web, mobile, APIs, push alerts, notifications) to deliver an innovative and personalised customer experience. To do this effectively, it must have a digital and data enabled core, use data and insights to build customer intimacy, as well as driving and automating decision making across the business.
So what is needed? Lambda? Kappa? Kappa+? From my experience, a digital bank needs four critical data architecture components to form a modular architecture:
- Data Distribution Layer (DDL) – this can also be called the data fabric or mesh (or whatever the current buzzword is) but the core capability enables you to distribute data across your organisation to feed both your operational and analytical features and processes, and is the backbone of your digital bank. The real paradigm shift that I’ve observed in successful digital banks is the move to event-based architectures and streaming data. This trend is well embedded in non-financial services organisations, and is a powerful capability that opens up many opportunities for innovative customer features, for example real-time notifications, alerts, nudges and recommendations as well as continuous monitoring of customer behaviour and risks. If you don’t have an event based architecture, you’re not going to succeed as a digital bank.
- Operational Data Store (ODS) – modern digital banks are built on a microservices architecture (and if they’re not, they won’t be able to support the pace of change that the industry now demands). This paradigm is well established and most microservices will need to manage state and persist data – this is where the ODS has its time to shine. The ODS provides the data stores for your microservices. In reality the ODS will be many separate databases, each managed by their specific microservice, and each microservice may have different storage requirements, read/write patterns and technology needs, so this requires a well thought through design.
- Data lake – the ying to the ODS's yang – your data lake should suck in all your data, structured, unstructured, voice calls, webchats, clickstream, social and third party (e.g. example accounting data, or bureau data). This data rich environment can then be used to explore, innovate and ideate – it breaks down the legacy silos and enables new connections and insights to be identified. It’s where your data scientists, marketing teams and customer experience analysts should spend their time testing and learning new concepts, building ML models, identifying opportunities for new revenue streams, and driving business decisions. In short it’s one of your most critical assets that, if used correctly, can provide opportunities for significant differentiation and competitive advantage for your digital bank.
- Data warehouse – yes, you did read it correctly – I said the dreaded DW words. Everyone has their own horror stories about data warehouses and they definitely have negative connotations, mainly due to lots of failed projects in the early 2000’s. However there is still an important role for the DW concept in today’s modern digital architecture. The data warehouse is needed to deliver consistent, repeatable, high quality and highly governed MI and reporting – it enables you to provide transparency and full lineage on your data, which is crucial for regulatory, risk and financial reporting, for example BCBS, MIFID, COREP, FINREP, BOE etc. If used in this limited way, your will learn to love your DW again, and it will deliver business value.
As I said in the beginning, getting your data architecture right is a challenge. In addition to technical components, it will need exec level buy in. There must be a desire to improve things and a long-term data strategy all wrapped by an effective data governance framework which will enable the appropriate access, sharing and management of data and drive the right data culture across the organisation (much more on this in a later blog post).
We are only at the start of the digital journey, so your data architecture will need to be a living, breathing, continuous transformation that evolves to meet the new customer demands and business needs. I’ve purposely not touched on technology in this post, but clearly cloud and cloud native services are driving architectural change, and I only see this trend accelerating – Garner predicts that by 2022, 75% of all databases and data applications will be deployed in the cloud.
In the next decade, when all banks become more data-centric, the ones that succeed will be the ones that have the right architecture to unlock the value of their data.
In the next post, I’ll talk in a lot more detail about the event-driven architecture concept which is so critical to a digital bank, and the data engineering, transformation and DataOps needed to deliver this.
This blog is the second one in a mini-series exploring the increasingly important role that data, and the analytics and AI that it enables, plays in digital banking.