Data Engineering
The Awakening of the Data Lake
On schema, sediment, and the uneasy consciousness of accumulated files
She looked out across the parquet files and understood, with some alarm, that they were all slightly different.
The data lake was once promised freedom from the constrictions of the warehouse, and like many freedoms it soon required partitioning, cataloguing, lineage, access policy, and a naming convention nobody loved. Chopin examines ingestion, lakehouse design, data quality, schema evolution, and the moment a team realises that 'raw' is not a synonym for 'unaccountable'.
Contents
- A Swim Among Untyped Events
- The Grand Isle of Object Storage
- Schema Evolution at Dusk
- Lineage, or the Sound of the Sea
Published by Austenpunk, the distinguished imprint for engineers who believe that every production incident would be improved by a stronger sense of narrative irony.
This essential volume combines literary feeling, technical anxiety, and just enough documentation to suggest that somebody once understood the system.
