- Overview
- References
- Data product specifications
- Data contract specifications
- Books
- Articles
- From Data Catalog to Data Marketplace
- Exploring the Integration of OpenLineage with ODPS
- ๐๐จ๐ฐ ๐๐จ ๐๐จ๐ฎ ๐๐๐ง๐๐ ๐ ๐๐๐ญ๐ ๐๐ซ๐จ๐๐ฎ๐๐ญ๐ฌ?
- The Data Product Ecosystem: Core, Analytic, and Data Science Products
- Adopting a product mindset
- 6 Essential Lessons for Building Great Data Products
- The state of Data Products
- How To Build a Data Product with Databricks
- Data Pipelines for Data Products
- How to Grow Product-Minded Engineering Teams
- A good engineer thinks like a product manager
- Frameworks
Created by gh-md-toc
This project intends to document requirements and referential material to implement data contracts in the perspective of data engineering on a modern data stack (MDS).
Data contracts are essential to decouple data producers from data consumers, while having both parties taking responsibility for their respective parts.
Even though the members of the GitHub organization may be employed by some companies, they speak on their personal behalf and do not represent these companies.
- Material for Data platform - Data products (this repository)
- Material for Data platform - Data quality
- Material for Data platform - Data contracts
- Architecture principles for data engineering pipelines on the Modern Data Stack (MDS)
- Specifications/principles for a data engineering pipeline deployment tool
- Linux Foundation's Open Data Product Specification (ODPS): https://opendataproducts.org/
- Innoq's specification for Data Products: https://dataproduct-specification.com/
- Open Data Mesh (ODM)'s Data Product Descriptor Specification (DPDS): https://github.com/opendatamesh-initiative/odm-specification-dpdescriptor
- Open Data Contract Specification (ODCS)
- Reader-friendly, dedicated site: https://bitol-io.github.io/open-data-contract-standard/latest/
- GitHub home page: https://github.com/bitol-io/open-data-contract-standard
- Innoq's Data Contract specification: https://datacontract.com/
- Title: Data Products for all ages
- Author: Jean-Georges Perrin (Jean-Georges Perrin on LinkedIn)
- Available on Amazon: https://www.amazon.fr/dp/B0DPL3MCWJ
- ASIN: โ B0DPL3MCWJ
- ISBN-13: โ 979-8341255999
- Number of pages:โ 36
- Title: From Data Catalog ๐ to Data Marketplace ๐
- Author: Jochen Christ (Jochen Christ on LinkedIn)
- Date: Jan. 2025
- Link to the LinkedIn post: https://www.linkedin.com/posts/jochenchrist_datamarketplace-datamarketplace-dataproducts-activity-7281953125140246528-BExu/
- Link to the Data Mesh Manager blog post: https://datamesh-manager.com/learn/data-catalog-vs-data-marketplace
- Title: Exploring the Integration of OpenLineage with ODPS
- Author: Jarkko Moilanen
(Jarkko Moilanen on LinkedIn)
- Jarkko Moilanen is one of the main maintainers of the Linux Foundation's Open Data Product Specification (ODPS)
- Date: Dec. 2024
- Link to the post: https://www.linkedin.com/posts/jarkkomoilanen_datalineage-openlineage-datagovernance-activity-7275156333790715905-t5-7/
๐๐จ๐ฐ ๐๐จ ๐๐จ๐ฎ ๐๐๐ง๐๐ ๐ ๐๐๐ญ๐ ๐๐ซ๐จ๐๐ฎ๐๐ญ๐ฌ?
- Title: ๐๐๐ญ๐ ๐๐ซ๐จ๐๐ฎ๐๐ญ๐ฌ ๐๐ข๐ฅ๐ฅ ๐๐ ๐ญ๐ก๐ ๐๐จ๐ญ ๐๐จ๐ฉ๐ข๐ ๐จ๐ ๐๐๐๐, ๐๐ฎ๐ญ ๐๐จ๐ฐ ๐๐จ ๐๐จ๐ฎ ๐๐๐ง๐๐ ๐ ๐๐ก๐๐ฆ?
- Author: Fouad Talaouit (Fouad Talaouit on LinkedIn)
- Date: Dec. 2024
- Link to the post: https://www.linkedin.com/posts/fouadtalaouit_dataproducts-datamanagement-datamesh-activity-7273218047044194305-2wC-/
- Title: The Data Product Ecosystem: Core, Analytic, and Data Science Products
- Author: Fouad Talaouit (Fouad Talaouit on LinkedIn)
- Date: Dec. 2024
- Link to the article: https://www.linkedin.com/pulse/data-product-ecosystem-core-analytic-science-products-talaouit--iwr0e/
- Title: Adopting a product mindset
- Author: Andrew Jones (Andrew Jones on LinkedIn)
- Date: Dec. 2024
- Link to the article: https://andrew-jones.com/newsletter/2024-12-06-adopting-a-product-mindset/
- Title: 6 Essential Lessons for Building Great Data Products
- Author: Dr Sven Balnojan (Dr Sven Balnojan on LinkedIn, Dr Sven Balnojan on Substack)
- Date: Dec. 2024
- Link to the article: https://www.thdpth.com/p/6-essential-lessons-for-building
- Publisher: Substack
- Title: The state of Data Products
- Date: July 2024
- Author: Wannes Rosiers (Wannes Rosiers on LinkedIn, Wannes Rosiers on Medium)
- Link to the article on Medium: https://medium.com/conveyordata/the-state-of-data-products-9e1bc5c39bcb
- Title: How To Build a Data Product with Databricks
- Author: Jochen Christ (Jochen Christ on LinkedIn, Jochen Christ on GitHub)
- Date: Apr. 2024
- Link to the article: https://www.datamesh-architecture.com/howto/build-a-dataproduct-with-databricks
- Associated GitHub repository (with source code): https://github.com/datamesh-architecture/databricks-dataproduct-example
- Title: Data Pipelines for Data Products
- Author: Bruno Gonzales (Bruno Gonzales on LinkedIn, Bruno Gonzales on Substack)
- Date: Oct. 2023
- Publisher: Modern Data 101 on Substack
- Link to the article: https://moderndata101.substack.com/p/data-pipelines-for-data-products
- Title: How to Grow Product-Minded Engineering Teams
- Author: Nicola Ballotta (Nicola Ballotta on LinkedIn, Nicola Ballotta on Substack)
- Date: Jan. 2024
- Link to the article: https://hybridhacker.email/p/how-to-grow-product-minded-engineering
- Title: A good engineer thinks like a product manager
- Author: Gregor Ojstersek (Gregor Ojstersek on LinkedIn, Gregor Ojstersek on Substack)
- Date: Nov. 2023
- Link to the article: https://newsletter.eng-leadership.com/p/a-good-engineer-thinks-like-a-product
- Home page: https://datamesh-manager.com/
- Free demo: https://demo.datamesh-manager.com/
- Main company behind: InnoQ
- Overview: Manage data products, agree on data contracts, and automate data governance. Get an enterprise data marketplace.
- GitHub page: https://github.com/conveyordata/data-product-portal
- License: Apache License 2.0
- Overview:
- The Data Product Portal enables to scale building data products across all departments in an organisation in a self-service manner. It does so by providing a guided setup for creating data products, with proper approval processes that will enable governance by design for data initiatives. The portal is a process tool that helps data professionals do their work more efficiently while providing governance and insights into how data is being used throughout the organisation.
- Unlike traditional data catalogs that primarily focus on describing data, the Data Product Portal guides you through the entire data product development lifecycle. This includes self-service and secure access to tools, data platforms, data sources, and sharing concepts, ensuring control and oversight for business stakeholders.
- Our goal is to bridge the gap between data governance, data platforms and data catalogs and provide a 360 view of all ongoing data initiatives that is easy to understand by everybody.
- To read more about it, please checkout our announcement blogpost