Apache Hudi Lakehouse Tables in VS Code

Overview

Apache Hudi is an open table format for data lakes. Each table keeps a timeline of commits next to its data files, which gives you upserts, incremental processing, and access to earlier versions of the data. DBCode reads Hudi tables through DuckDB’s hudi extension:

• Copy-on-write and merge-on-read tables: Read both table types, including Hudi table versions 6, 8, and 9
• Multiple storage options: Tables on a local disk, Amazon S3, Google Cloud Storage, or Azure Storage
• Table discovery: Point DBCode at a folder or bucket and it finds every Hudi table underneath
• Table metadata: See each table’s settings, commit timeline, and properties in the explorer
• Time travel and incremental reads: Query a table as it was at an earlier commit, or only the records that changed between two commits

DBCode downloads the DuckDB hudi extension the first time you connect.

Connecting

To connect to your Hudi tables in DBCode:

1. Open the DBCode Extension: Launch Visual Studio Code and open the DBCode extension.
2. Add a New Connection: Click on the “Add Connection” icon.
3. Fill in the connection settings:
   • Select Apache Hudi as the database type
   • Choose the Storage: Local Directory, Amazon S3, Google Cloud Storage, or Azure Storage
   • Enter a Root Path, one or more Table Paths, or both
   • For cloud storage, choose an authentication profile
4. Connect: Click save to establish your connection.
5. Start Querying: Browse your Hudi tables and open their data.

For detailed instructions on connecting to databases, refer to the Connect article.

Root path and table paths

• Root Path: DBCode scans this folder, and every folder beneath it, for Hudi tables. A Hudi table is a folder holding .hoodie/hoodie.properties. Hudi’s internal metadata tables are skipped.
• Table Paths: Hudi table folders to add as well as, or instead of, the tables found under the root path. Use them for tables that live outside the root, or to list your tables directly in a large bucket so DBCode reads only those.

DBCode names each table from its hoodie.properties, matching what Spark and Hive show: the schema is hoodie.database.name (or default when it is not set) and the table is hoodie.table.name. When two tables share a schema and a name, both are named by their path relative to the root path instead.

Refresh the connection to pick up tables added since you connected. Tables that are gone drop out of the explorer.

Storage and authentication

Storage	Root path example	Authentication profile
Local Directory	Pick a folder	None
Amazon S3	s3://my-bucket/lake	AWS: static credentials (with an optional session token) or a named AWS profile, including SSO, with the bucket’s region as the Default Region. See the AWS guide.
Google Cloud Storage	gs://my-bucket/lake	AWS, with static credentials holding a Cloud Storage HMAC key: the access ID as the Access Key ID and the secret as the Secret Access Key.
Azure Storage	az://my-container/lake or abfss://my-filesystem@myaccount.dfs.core.windows.net/lake	Azure Storage: connection string, account key, SAS token, credential chain, managed identity, or service principal.

• Google Cloud Storage: Create the HMAC key in the Google Cloud console under Cloud Storage settings, on the Interoperability tab, for a service account that can read the bucket.
• Azure Storage: An az:// path names the container, and the storage account comes from the authentication profile. An abfss:// path names both the file system and the account. For sovereign clouds, set the profile’s Endpoint Suffix (for example core.usgovcloudapi.net or core.chinacloudapi.cn).

Hudi Features in DBCode

Browsing tables

Hudi tables appear as tables under their schemas, ready to browse and query:

• Open data: Browse a table’s rows in the grid, with filtering and sorting
• Columns: See each table’s columns and data types
• Scripts and tools: Generate SELECT scripts, export data, search, build queries with the query builder, and view the tables in a diagram

Hudi tables are read-only in DBCode. You read and query them here, and write to them with your Hudi pipelines (Spark, Flink, or another Hudi writer).

Table metadata

Expand a table and open its Metadata folder. It lists the table’s location, table type (copy-on-write or merge-on-read), table version, record key, partition fields, ordering field, and base file format, followed by two folders:

• Timeline: The latest commit, the number of completed commits, and the ten most recent instants with their action and completion time
• Properties: Every setting in the table’s hoodie.properties

The commit instants in the Timeline are the values you use for time travel and incremental reads.

Querying with SQL

Each Hudi table is a view you can query with DuckDB SQL in SQL files and notebooks:

SELECT * FROM "default".trips LIMIT 100;

You also get SQL autocomplete, EXPLAIN plans, and query cancellation.

Time travel and incremental reads

Call read_hudi with the table’s path to read it at a point on its timeline. Pass hudi_option := getvariable('__dbcode_hudi') so the read uses the connection’s storage credentials.

To read a table as it was at a commit, pass the commit instant as as_of:

SELECT *
FROM read_hudi(
  's3://my-bucket/lake/trips',
  as_of := '20260307162612542',
  hudi_option := getvariable('__dbcode_hudi')
);

To read only the records written between two commits, pass start and "end". Write "end" in double quotes, because end is a SQL keyword. Leave out "end" to read every change after start.

SELECT *
FROM read_hudi(
  's3://my-bucket/lake/trips',
  start := '20260307162604102',
  "end" := '20260307162612542',
  hudi_option := getvariable('__dbcode_hudi')
);

Use as_of and start/"end" in separate queries. For Google Cloud Storage, write the table path with s3:// in place of gs:// (for example s3://my-bucket/lake/trips), because DBCode reads Cloud Storage through its S3-compatible endpoint.

Note

The __dbcode_hudi variable holds the connection’s storage credentials, so any SQL run on the connection can read them, including SQL from AI tools.

By using Apache Hudi with DBCode, you can explore and query your lakehouse tables directly within Visual Studio Code, without starting a Spark cluster.

For more information about Apache Hudi, check out Apache Hudi.