Chief data officers across the federal government are working to optimize the government’s use of data by the year 2030. The data mesh approach to data architecture has been central to the conversation around this work. It has also been embraced in the Army Unified Data Reference Architecture and the Data Mesh Reference Architecture of the DoD’s Chief Digital and Artificial Intelligence Office (CDAO).
We wanted to get a sense of how members of the defense and intelligence communities are viewing the transition from legacy data architectures to data mesh. So, we asked ECS Vice President of Defense & Intel Strategy Karthik Srinivasan for his take on the subject.
Karthik Srinivasan
Vice President of Defense Solutions
Q: How are federal technology leaders viewing the transition to data mesh?
A: Many leaders — including our customers at the CDAO, U.S. Army, Department of Homeland Security, and intelligence agencies — understand the potential of data mesh and how it provides a framework for improved data quality, agility, data governance, lineage, and reduced costs.
Some leaders who are considering the transition, however, are hesitant to make the changes required to their legacy data systems. This is understandable, because there are vast amounts of data and various data types and security levels across the DoD and intelligence communities.
We’re talking about financial data, mission data, C5ISR and CJADC2 data on land, air, and sea, plus historical data from the last 30 years. All this data resides in different places — on premises, in AWS, in Azure, on Google Cloud, and in legacy systems — with varying data ontologies and catalogs, and without standardized data architecture in most cases.
It’s information about military assets, warfighter locations, the health of personnel, geographic and situational awareness, and much more. You can see why the transition to data mesh may look complex, worrisome, and difficult to agency leaders.
So, we assure them that data mesh has been successfully deployed across commercial enterprises and that it can enable them to access, normalize, secure, and analyze data for driving critical command decisions in near real time.
We also show them that — while the organizational change needed for the transition to data mesh will take time — we can help them stand up the required technology stack quickly and securely.
Q: How do you show a customer that we can stand up a data mesh technical solution quickly?
A: ECS has built a robust, production-ready enterprise data mesh solution that complies with the VAULTIS framework and the reference architectures and requirements set out by the CDAO and Army. It’s highly configurable, so we can quickly and easily tailor it to the customer’s specific mission needs and then demonstrate it for them. We create multiple instances of all the environments that reside on the customer’s end, show them the architecture, and then walk them through the solution’s salient features. This helps customers get past their concerns about complexity and delays.
Customers get to see the ECS solution and they can gauge the cost savings that will result from no longer duplicating data. They also learn that our data mesh platform, product management, and analytics layers are all AI-ready.
These demonstrations show customers that getting started with data mesh is relatively easy and nothing to fear, and that usable data products can be developed quickly and built upon to create modern data architecture at scale.
Q: What advice would you give to agency leaders who are considering the transition to data mesh?
A: First, I would explain that it will take time to recognize the full value of data mesh due to the organizational changes required. While we can accelerate the technical implementation, data producers across the enterprise will need to buy into the data mesh model and understand how to produce data products and make those products available through the mesh. This is not a quick journey.
Another challenge will be the need to work through the security model for data products in an automated way. Data mesh fundamentally changes how data is distributed and managed, creating unique security challenges. The security model needs to balance the autonomy of producers who own their domain data with an overarching, organization-wide security policy that governs access to the data products.
Agency leaders should also understand that data lakes and warehouses often cause you to duplicate your data. For large organizations, duplicating all that data can lead to exorbitant infrastructure costs, consume available network bandwidth, and create delays waiting for those massive datasets to traverse the network and be prepared for analysis. For a successful data mesh implementation, it’s important to use an API-driven approach that guarantees data as a product, federated decision-making, and the ability to securely self-service data analytics.
I would also explain that transitioning to data mesh doesn’t mean you have to abandon existing technology investments. It is possible to build upon your existing data management stack and leverage existing features when transitioning to a data mesh architecture.
Ultimately, defense and intelligence agencies need a decentralized approach to access data, with domain-ownership for data and system owners retained. Leaders should understand that the data-as-a-product approach ensures data is discoverable, addressable, self-describing, federated, and interoperable. ECS’ data mesh platform provides a self-service federated data infrastructure with security and computational governance through our APIs. Our customers are now able to solve their monolithic data warehouse bottlenecks, scale across global networks — including DDIL access points — and ensure access to authoritative data sources.
With the advancements we have in tools, technologies, and AI, ECS can show you how to start your journey quickly and begin marching toward a mature enterprise data mesh architecture.
