Columnar Current
Columnar-related news, posts, and talks from around the web
Recent posts
-
Preview of a New ADBC Driver for Apache Druid
Connect your ADBC tools to Druid’s real-time analytics engine.
-
Preview of a New ADBC Driver for Apache Cassandra
Cassandra gets Arrow reads and bulk writes in one driver.
-
Preview of a New ADBC Driver for Presto
Presto, the distributed SQL engine that can run fast analytical queries across massive datasets, now has a pre-release ADBC driver, installable with dbc.
-
chDB as an ADBC driver
chDB, the embedded ClickHouse engine, is now available as an ADBC driver, bringing in-process analytics to any application that speaks ADBC.
-
Feeding the Agents: Fast Structured Data Retrieval with Arrow
Ian Cook joins Kostas and Nitay to connect ADBC’s end-to-end columnar data path to a new bottleneck: AI agents that can reason faster than their database tools can return results.
-
Apache Arrow ADBC 24 (Libraries) Release
This quarterly release of the ADBC libraries brings full-featured dynamic driver loading to C# and Java, adds URI-based driver resolution, and reorganizes the documentation.
-
Introducing the ClickHouse ADBC Driver
ClickHouse announces its official ADBC driver, which uses Apache Arrow to deliver query results directly in columnar format for zero-conversion, end-to-end data movement across languages including Python, R, Ruby, and C.
-
ADBC 101: Fast Database Access for Humans and Agents
Ian Cook and Emil Sadek join Joe Reis to present an intro to ADBC and to talk about tools like dbc and databow.
-
Contribute a dbt Core v2 adapter
A guide to building a dbt Core v2 adapter, which relies on pre-compiled ADBC drivers to handle connection management and wire protocol details instead of standalone Python packages.
-
Transition from ODBC to ADBC drivers in Power BI and Microsoft Fabric
Documents how Power BI and Microsoft Fabric are transitioning data source connections from legacy embedded ODBC drivers to ADBC drivers.
-
The Apache Arrow Ecosystem
Explores the Apache Arrow ecosystem for building data applications, with practical examples of Flight, Flight SQL, and ADBC.
-
Apache Arrow ADBC 23 (Libraries) Release
The version 23 release of the ADBC libraries and drivers brings improvements including a new Node.js driver manager, connection profiles, and expanded JNI functionality.
-
New ADBC Driver for SingleStore
Introduces a pre-release ADBC driver for SingleStore, installable via dbc, with support for query execution and initial connectivity features.
-
Building My First Data Tool With ADBC
Introduces a CLI built in Go that enables direct data transfers between database systems without intermediate materialization, showcasing the ADBC ecosystem as a collection of primitives for data interchange.
-
New ADBC Driver for Exasol
Introduces the new ADBC driver for Exasol, installable via dbc, with support for query execution and catalog inspection and additional features like bulk ingestion and bind parameters under development.
-
Updated ADBC drivers for BigQuery, MySQL, Oracle Database, Redshift, SQL Server, Snowflake, and Trino
Updated ADBC drivers bring security updates, bug fixes, and other enhancements across seven database platforms.
-
How Apache Arrow Made Spark Faster: A 10-Year Journey
Columnar co-founder Matt Topol discusses the 10-year evolution of Apache Arrow’s integration with Apache Spark and how the columnar in-memory format has accelerated Spark’s performance.
-
Machine Learning with ADBC, DuckDB & XGBoost: No pandas, Just Pure Arrow Tables
Demonstrates building a machine learning pipeline that queries data from DuckDB using ADBC and trains with XGBoost, all while working exclusively with Arrow tables and avoiding pandas conversions.
-
Faster ADBC Drivers for BigQuery, MySQL, SQL Server, and Trino
Updated ADBC drivers for BigQuery, MySQL, SQL Server, and Trino bring significantly improved query and bulk ingest performance.
-
ICYMI - WTF is ADBC?
Explains why ODBC and JDBC have become bottlenecks in modern analytics and how ADBC provides a vendor-neutral, Arrow-native API that eliminates the costly row conversion overhead of legacy drivers.
-
From ODBC to ADBC: Modernizing the Data Stack for AI and Analytics w/ Ian Cook
Joe Reis and Ian Cook discuss why ADBC is finally replacing ODBC and JDBC by eliminating the serialization tax of converting columnar data to rows for transport and back.
-
Preview of a New ADBC Driver for ClickHouse
Introduces a preview ADBC driver for ClickHouse, developed by ClickHouse using HTTP transport with native Apache Arrow support, available for installation with dbc.
-
3.7x Faster EL Pipelines: Arrow + ADBC vs. SQLAlchemy
Benchmarks Arrow with ADBC against SQLAlchemy for moving 5 million rows from DuckDB to MySQL, reducing time from 344s to 92s by eliminating row-by-row serialization overhead.
-
New ADBC Driver for Databricks
Introduces an early version of the ADBC driver for Databricks, bringing initial support for high-performance columnar data connectivity with Databricks SQL warehouses and clusters through the Arrow Database Connectivity API.
-
ADBC: An Intro to NextGen Database Connections
Introduces ADBC (Arrow Database Connectivity), a modern database access API that keeps data in columnar format throughout data pipelines, covering driver management with the dbc CLI tool, Python implementation, bulk data ingestion, and streaming large datasets between databases.
-
Apache Arrow ADBC 22 (Libraries) Release
The version 22 release of the ADBC libraries and drivers brings improvements including driver managers that can open connections using only a URI, bulk ingestion support in the Flight SQL driver, and transaction isolation level support in the PostgreSQL driver.
-
Updated ADBC Drivers for BigQuery, SQL Server, MySQL, Redshift, Snowflake, and Trino
New versions of ADBC drivers are now available through dbc, introducing enhancements such as improved type support, better authentication options, and bug fixes across six major database platforms.
-
ODBC Takes an Arrow to the Knee: ADBC
Columnar co-founder Matt Topol explains how ADBC avoids the conversion costs of row-oriented APIs like ODBC and JDBC by providing result sets directly in Arrow columnar format, enabling efficient access to data lakehouse systems like Dremio as well as data warehouse and relational database systems.
-
What the Heck Is dbc?
A hands-on walkthrough of dbc showing how a few simple commands can install ADBC drivers and connect to databases, lowering the barrier to entry for working with columnar data.
-
Apache Arrow’s Final Frontier: Replacing Outdated Database Drivers
Columnar launches with $4M in seed funding to address the database communication bottleneck using ADBC, reducing query times by more than 90% in many applications compared to legacy standards like ODBC and JDBC.
-
Columnar launches to redefine data connectivity with Arrow-powered ADBC drivers
Columnar, a startup founded by core Apache Arrow developers, secured $4 million in seed funding to accelerate data connectivity using Arrow-based drivers, particularly benefiting AI applications that require fast access to structured data.
-
Meet the founders of Columnar: Ian Cook, David Li, and Matt Topol
Bessemer Venture Partners leads a $4M seed round for Columnar, a startup founded by Apache Arrow contributors to modernize data connectivity, offering Arrow-native ADBC drivers that deliver 10-100x faster query retrieval.
-
Announcing the ADBC Driver Foundry
Introduces the ADBC Driver Foundry, a new open source hub for collaborative driver development within the Apache Arrow ecosystem, providing per-project repositories and shared resources for growing the ADBC project.
-
Where We’re Going, We Don’t Need Rows: Columnar Data Connectivity with ADBC
Seminar presenting ADBC (Arrow Database Connectivity), Apache Arrow’s answer to ODBC and JDBC, exploring its architecture and adoption across major data systems including dbt, Databricks, DuckDB, and Snowflake.
-
Fast, Universal Data Access with Apache Arrow ADBC
Columnar co-founder David Li presents ADBC as a modern alternative to ODBC and JDBC, demonstrating high-performance database access using familiar Python DB-API interfaces.
-
Data Wants to Be Free: Fast Data Exchange with Apache Arrow
Examines how Apache Arrow improves data serialization efficiency compared to legacy formats like PostgreSQL’s binary protocol, and presents various Arrow-based tools including Arrow IPC, Arrow HTTP, Arrow Flight SQL, and ADBC for building efficient data interchange protocols.
-
How the Apache Arrow Format Accelerates Query Result Transfer
Explains how Apache Arrow’s columnar data format reduces serialization and deserialization bottlenecks in query result transfers, with real-world improvements ranging from 10x to several hundred times faster.