Files and zarr

Beyond tables, the lakehouse holds generic tables: named locations for files. This package reads and writes them through signed requests.

Files

from pathlib import Path
from databaas_utils import GenericTableStore

store = GenericTableStore(settings)           # or get_store() in a notebook / DAG
uri = store.put("grib", "2026/09/30/run.grib2", Path("run.grib2"))
store.get("grib", "2026/09/30/run.grib2", Path("copy.grib2"))
list(store.list("grib", prefix="2026/09"))
store.delete("grib", "2026/09/30/run.grib2")

put creates the table if needed. get, list, open and delete require it to exist. open() returns a file object, and object_store() returns the raw obstore handle.

LocalStore has the same put/get/uri and writes under a directory, for tests. It is never chosen for you.

Zarr

from databaas_utils import open_zarr_group

group = open_zarr_group(settings, "harmonie_zarr", path="2026093000")
group.create_array("temperature", shape=(10, 10), dtype="f4")

Install databaas-utils[zarr], never zarr[remote]: the latter pulls in upstream obstore and breaks signed access.

The file index

Lakekeeper stores only a location for a generic table, not a file list. A FileIndex is an Iceberg table that records what landed, so you can query it.

from databaas_utils import FileIndex, FileRecord

index = FileIndex(settings)
index.record(FileRecord(uri=uri, key="2026/09/30/run.grib2", size_bytes=1024,
                        run_date="20260930", cycle="00"))
index.latest()      # "2026093000"
index.scan(limit=5) # Arrow table

Only uri, key and size_bytes are required. Put anything else in attributes. NullIndex logs instead of writing, for runs with no catalog.