Data Engineering Weekly

Exploring the Evolution of Lakehouse Technology: A Conversation with Vinoth Chandar and Onehouse CEO

In this episode, Ananth, author of Data Engineering Weekly and CEO of Onehouse, discusses the latest developments in the Lakehouse technology space, particularly focusing on Apache Hudi, Iceberg, and Delta Lake. They discuss the intricacies of building high-scale data ecosystems, the impact of table format standardization, and technical advances in incremental processing and indexing. The conversation delves into the role of open source in shaping the future of data engineering and addresses community questions about integrating various databases and improving operational efficiency.

00:00 Introduction and New Year Greetings

01:19 Introduction to Apache Hudi and Its Impact

02:22 Challenges and Innovations in Data Engineering

04:16 Technical Deep Dive: Hudi's Evolution and Features

05:57 Comparing Hudi with Other Data Formats

13:22 Hudi 1.0: New Features and Enhancements

20:37 Industry Perception and the Future of Data Formats

24:29 Technical Differentiators and Project Longevity

26:05 Open Standards and Vendor Games

26:41 Standardization and Data Platforms

28:43 Competition and Collaboration in Data Formats

33:38 Future of Open Source and Data Community

36:14 Technical Questions from the Audience

47:26 Closing Remarks and Future Outlook