It has been almost 1000 days since we started work on DuckDB. In this talk, we reflect on the process and revisit the initial goals. We also describe major additions to the system such as automatic query parallelization. We discuss the major differences between developing a research prototype system that is mostly built to investigate specific research questions and support claims in a paper and a general-purpose analytical data management system such as DuckDB.
Hannes Mühleisen is a senior researcher at CWI’s Database Architectures Group. His research focuses on analytical data management systems and data management for Data Science. Recently, he spends much of his time working on the DuckDB data management system.