Databend Python Binding

This crate intends to build a native python binding.

Installation

pip install databend

Usage

Basic:

from databend import SessionContext

ctx = SessionContext()

df = ctx.sql("select number, number + 1, number::String as number_p_1 from numbers(8)")

# convert to pyarrow
df.to_py_arrow()

# convert to pandas
df.to_pandas()

Register external table:

supported functions:

• register_parquet
• register_ndjson
• register_csv
• register_tsv

ctx.register_parquet("pa", "/home/sundy/dataset/hits_p/", pattern = ".*.parquet")
ctx.sql("select * from pa limit 10").collect()

Tenant separation:

ctx = SessionContext(tenant = "a")

Development

Setup virtualenv:

python -m venv .venv

Activate venv:

source .venv/bin/activate

Install maturin:

pip install "maturin[patchelf]"

Build bindings:

maturin develop

Run tests:

maturin develop -E test

Build API docs:

maturin develop -E docs
pdoc databend

Storage configuration

• Meta Storage directory(Catalogs, Databases, Tables, Partitions, etc.): ./.databend/_meta
• Data Storage directory: ./.databend/_data
• Cache Storage directory: ./.databend/_cache
• Logs directory: ./.databend/logs

More

Databend python api is inspired by arrow-datafusion-python, thanks for their great work.