Level Up Your Lakehouse with Data Source Control & Cross Collection Consistency

Data Lakehouse is combining the best elements of data lakes and data warehouses into a single platform to assist data teams to operate efficiently. With this modern data stack and Lakehouse capabilities, we can enable multiple types of data transformations to co-exist while eliminating the data silos in data teams. That means better data flows, simpler operational maintenance, and overall better data products! But, what happens when our transactions' logic contains more than one table? How can we attain cross-collection consistency with foreign keys using multi-statement transactions? That might corrupt our data products! In this session, you will learn how to leverage Delta Lake and LakeFS to reach cross-collection consistency when operating on multi-statement transactions.

Speaker Bio:
As Vice President of Developer Experience at Treeverse, Adi helps build lakeFS, git-like interface for the data lakehouse. In her work, she brings her vast industry research and engineering experience to bear in educating and helping teams design, architect, and build cost-effective data systems and machine learning pipelines that emphasize scalability, expertise, and business goals. Adi is a frequent worldwide presenter and the author of O'Reilly's upcoming book, "Machine Learning With Apache Spark." Adi is also a proud Databricks Beacon supporting Data & AI practitioners around the world! Previously, she was a senior manager for Azure at Microsoft, where she focused on building advanced analytics systems and modern architectures.

When Adi isn’t building data pipelines or thinking up new software architecture, you can find her on the local cultural scene or at the beach.