Datamesh-architecture
https://www.datamesh-architecture.com/
97 Things Data Engineers
- 88 - What is a Data Mesh, and How Not to Mesh It Up - Barr Moses and Lior Gavish
- As first defined by Zhamak Dehghani, the original architect of the term, a data mesh is a type of data-platform architecture that embraces the ubiquity of data in the enterprise by leveraging a domain-oriented, self-serve design.
- This idea borrows from Eric Evans’s theory of domain-drive design, a flexible, scalable software development paradigm that matches the structure and language of your code with its corresponding business domain.
- If you haven’t already we highly recommend reading Dehghani’s groundbreaking article
- How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh?
- https://martinfowler.com/articles/data-monolith-to-mesh.html
- or watching Max Schulte’s tech talk on
- Why Zalando transitioned to a data mesh
- https://databricks.com/session_na20/data-mesh-in-practice-how-europes-leading-online-platform-for-fashion-goes-beyond-the-data-lake
- Why Zalando transitioned to a data mesh
- How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh?
- Domain-oriented data architectures, like data meshes, give teams the best of both worlds: a centralized database (or a distributed data lake) with domains (or business areas) responsible for handling their own pipelines.
- Data mesh actualy mandates scalable, self-serve observability in your data
- https://towardsdatascience.com/what-is-data-observability-40b337971e3e
- Domains cannot truly own their data if they don’t have observability
- Encryption for data at rest and in motion
- Data product versioning
- Data product schemas
- Data product discovery, catalog registration, and publishing
- Data governance and standardization
- Data production lineage
- Data product monitoring, alerting, and logging
- Data product quality metrics
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes Beyond the Data Lake
