Python assets

The rules of a Python asset, what a deploy checks, what a task may do, how a failure is handled and the operator's settings. A first example that builds a table from a Python asset is in Projects and deploys; why Python assets exist and where their limits lie is in Python assets: what they are for and how far they are trusted.

The rules of a Python asset

• One file, one asset. assets/<namespace…>/<name>.py is the table <namespace…>.<name>. The function's name does not matter. Every .py file under assets/ is an asset. Put helper code at the repository root or under src/; both are importable at run time. A helper module that has the same name as an installed package or a standard library module cannot be imported under that name, because the repository is searched last.
• Inputs. inputs maps each parameter of the function to a table or an artifact. A parameter that names a table receives the whole table as a polars.DataFrame (a table in memory). A parameter that names an artifact receives a pathlib.Path to a read-only copy of its files (see the reference). It is read as the run principal, so row filters and column masks apply, and it is bound by the query limits (LAKANA_QUERY_LIMITS__MAX_RESULT_ROWS, default one million rows). If an input is too large, narrow it first with a SQL asset (usually a view) and declare that as the input. An asset that reads its own table declares it as an input. The table must exist before such an asset runs: the first run of an asset that reads itself fails at once, without retries, with <table> reads its own table as an input, but the table does not exist yet. Create the table with a first version of the asset that does not read itself, run it, then deploy the self-reading version.
• Decimals. DuckDB types a literal such as 12.0 as DECIMAL, a type of exact decimal numbers. Your function receives such a column as floating-point numbers (a Float64 column), also inside a list or a struct, so value * 0.5 works. An asset that must not round, for example one that adds up money, asks for the exact values with @asset(decimals="exact"): the column then arrives as a Decimal column of decimal.Decimal values, which cannot be multiplied by a float. A float holds about 15 significant digits: a decimal value beyond 2^53 (about 9 * 10^15) or with more than about 15 significant digits, a decimal identifier or the sum of a wide column for example, arrives rounded, without an error. Such a column needs decimals="exact". What your function returns is stored as it is: a float column as a double, a decimal column as a decimal. Older Lakana releases always handed a Python asset decimal.Decimal values; an asset written for them that relies on that needs decimals="exact" after an upgrade.
• Output. The function returns a polars.DataFrame. It replaces the asset's table in one snapshot. Zero rows are fine. With output="artifact" it returns a pathlib.Path inside lakana.output_dir() instead, and each run stores a new version of the artifact (see the reference).
• Metadata. description (default: the first paragraph of the docstring), owner, tags, columns (not for an artifact asset), retries, retry_delay and timeout work as in SQL front-matter, with the same defaults from lakana.yaml and the same operator maximums (see Retries and timeouts).
• Only literals. Deploy reads the decorator without running your code, so every argument must be written out as a literal: strings, numbers, lists and dicts of strings. Variables, **unpacking and function calls are refused.
• Lineage. A Python asset has dependency lineage (its inputs) and no column lineage. A SQL asset can read a Python asset's table by name; the run orders them as usual.
• Who sees a dropped table in table lineage. Lineage lists only tables you may read. A table that was dropped stays in the history of the tables it fed, marked dropped, and is shown only to someone who may read its whole namespace (a grant on the namespace, the environment or the instance) and never to a task token. A table created later under the same name is a new table: a grant on it does not show the old one. It is counted in hidden_nodes instead. To let a colleague see the history, grant them read on the namespace.

Polars 1 and Polars 2

lakana[assets] asks for polars>=1.30 and sets no upper bound, so a project that runs uv lock now gets Polars 2 (released in October 2026). Lakana runs assets on both, and its own tests run on 2 and on 1.30. A project that was locked earlier keeps the Polars version in its uv.lock until it runs uv lock --upgrade-package polars. What changes for the code of an asset when a project moves from 1 to 2 (the full list is Polars' own upgrade guide):

• LazyFrame.collect() uses the streaming engine and does not keep the row order after joins and group-bys unless you ask. If your asset depends on an order, sort explicitly (.sort(...)) before you return the frame, or use maintain_order where the method offers it.
• Map columns arrive as pl.Map(key, value). In Polars 1 they were a list of structs. A table stores no map, so only an input that a view returns can hold one, and returning a map column is refused (Column 'x' has an unsupported Arrow type: map<…>): turn it into columns first.
• Adding a signed and an unsigned integer column gives Int128, not a float. A table cannot store Int128 (or UInt128): the task fails with column 'x' has type Int128, which cannot be stored in a table; cast it to Int64 …. Cast the result to pl.Int64, or to a decimal.
• An Arrow extension type Polars does not know loads as pl.Extension. Take its stored values with pl.col("x").ext.storage() before you return it.
• Some casts are stricter and raise where Polars 1 converted silently. A task that fails with a Polars ComputeError about a cast after an upgrade needs the cast to say what should happen.

To stay on Polars 1, pin it in the project's own pyproject.toml and lock again:

dependencies = ["lakana[assets]==<version>", "polars>=1.30,<2"]

Run uv lock, commit uv.lock and deploy. Polars 1.30 is the oldest version Lakana supports. One convenience needs a newer one: passing a Polars Schema to client.tables.create(...) needs Polars 1.37 or newer. On an older Polars the call fails with a message that says so; pass pyarrow.schema(...) or a list of columns instead. Spilling to disk, the one thing Polars 2 does that can use the worker's disk, is described under "Polars inside a task" in Run.

What Python assets do not do (yet)

• materialize: table only. incremental is not supported (#23), and view is refused too.
• Table inputs and a table output are whole DataFrames in memory. There is no streaming (#22), so the worker needs memory for your inputs, your output and your own work. An artifact is a file on disk and is not held in memory by Lakana.
• A changed output schema is a permanent failure: the asset cannot pass evolve_schema. Restore the columns and types the table has, or drop the table (lakana table delete) and run the asset again, so it is created with the new schema.
• No async def.
• @asset(...) must be the only decorator on the function. To wrap behaviour, do it inside the function.
• Local path and workspace dependencies in uv.lock are refused. Put helper code in the repository, or publish it as a package.
• No column lineage, no lazy input API, no resource classes.

What a deploy checks

A deploy never runs your code. It reads each .py file with Python's ast module and the two dependency files as TOML. Nothing is installed at deploy. Every refusal names the file and, where there is one, the line, as path:line: message. Fix the cause and deploy again.

What a task may read and write

Each attempt of a Python asset is a task that runs in its own process on a worker. The worker gives the process a task token: a credential that is valid for this one attempt and only while it runs. With it, your code can:

• read the tables and artifacts declared in inputs (and, when an input is a view, the tables that view reads: a view is not a security boundary here);
• replace its own table, or add a version to its own artifact if it produces one, whether or not the environment is protected (a token belongs to one environment);
• read the deploys and runs of its environment.

It can do nothing else: no other table or artifact, no DROP, no deleting an artifact version, no listing of tables, no administrative action. It also cannot do more than the run principal could (see Run principals): grant that principal READ_TABLE on the inputs (tables and artifacts) and CREATE_TABLE and WRITE_TABLE on the output. The token never appears on a command line, in the environment or in the task log.

The worker unpacks your repository fresh for every attempt into a private, read-only directory, which is the process's working directory. The process gets a private temporary directory and a small environment (PATH, HOME, TMPDIR, two PYTHON* settings, a Polars spill directory and, when the worker has a memory limit, two more POLARS_* settings, see "Polars inside a task" in Run). Its output and the output of the environment build are in lakana runs logs, up to 1 MiB per attempt (see Run).

Failure, retry and cancel

A task's token must outlive the task. If LAKANA_TASK_TOKENS__MAX_TTL_SECONDS is below the asset's timeout plus the kill grace (LAKANA_RUN__KILL_GRACE, 10 s) plus 60 s, the task fails at once and the message names the value to set. Retries, retry delay and timeout follow Retries and timeouts.

Dependencies

A project with a Python asset has a pyproject.toml and a committed uv.lock at the repository root. The lock must contain lakana at a version compatible with the instance (the same minor version before 1.0, the same major version from 1.0) and polars; depending on lakana[assets] gives you both. It must be one of the project's own [project] dependencies: a task installs no dependency group and no extra, so lakana in [dependency-groups] is refused at deploy. requires-python must admit the worker's Python. Your repository's own code is not installed: it is imported from the checkout.

The first task that needs an environment builds it from the lock (uv sync --frozen) and caches it, keyed by the hash of the lock together with your pyproject.toml, the Python version and the platform. Projects whose lock and pyproject.toml are both identical share the environment; a change to either builds a new one.

Put a Python asset's dependencies in [project] dependencies. Packages in a dependency group ([dependency-groups], including dev, and [tool.uv] default-groups) are not installed for a task. After a platform upgrade that changes the compatible lakana version, update the lock and redeploy: tasks of the old deploy fail with a message that says so. To install a package of your own from an index with a password, see Use a private package index.

What the build can run: it installs only what the lock names, but a package that is built from source (a source distribution) runs its own build backend, as the worker's user, with network access, and it can see the index and proxy credentials the worker passes to the build. That is why a build that uses a registered index's credential builds nothing from source: uv runs with --no-build, and a package without a wheel fails the build with a message. The worker builds with uv sync --frozen --no-config: your [tool.uv] settings shape the lock on your machine (when you run uv lock) and are ignored at build time on the worker (a uv.toml is refused, see What a deploy checks; the worker still reads the lock and the index definitions in [[tool.uv.index]]), and the lock decides which hosts the build contacts. Prefer wheels, and review dependencies as you review code.

The lock may name only the public index, the folder wheelhouse, the operator's extra index and registered package indexes; see What a deploy checks and Use a private package index.

Operator settings

The last four follow the rules of the LAKANA_GIT__* settings (see Git network settings); the index and the proxy may carry a password, and it is redacted from errors, logs and uv's output. Set LAKANA_RUN__* where the worker reads them. Related settings are LAKANA_RUN__DRAIN_GRACE_SECONDS and LAKANA_TASK_TOKENS__MAX_TTL_SECONDS. The latter caps the lifetime of every task token and is read by the server and by the worker: set it to the same value on both. A cap lowered on the server only is not applied to the tokens the worker mints. A task fails at once if the cap is below its needs (see "Failure, retry and cancel" above).

The worker needs uv on its PATH (the Lakana image ships it) and logs at startup whether it can run Python assets, and why not. Its cache directory is $XDG_CACHE_HOME/lakana-worker, else ~/.cache/lakana-worker:

• venvs/ holds the venvs (at most LAKANA_RUN__VENV_KEEP), next to one pair of empty lock files (<key>.lock, <key>.build.lock) per environment ever built there; they are never removed by the worker, and you may delete the ones of environments that no longer exist while no worker is running. uv/ holds uv's own download cache. Nothing limits or cleans the uv cache: it grows with every package version a project locks. To reclaim the space, stop the worker and delete the uv/ directory (the next build downloads what it needs again), or run uv cache clean as the worker's user with UV_CACHE_DIR set to that directory. A process that a task detaches from its process group is not killed when the task ends.
• attempts/ holds one directory per running attempt. It is removed when the attempt ends. After a hard kill of the worker, the leftover is removed by the worker's sweep: at start and then every LAKANA_RUN__SWEEP_INTERVAL seconds (the same setting paces the artifact store's sweep, and 0 turns both off) it deletes directories here that are named like an attempt directory (attempt- and eight characters), are real directories owned by the worker's user, have not changed for longer than a task can run (the asset timeout ceiling, the kill grace, 60 s and the venv build timeout) and are not locked. A worker holds a lock on its attempt directory while the attempt runs (it ends with the worker process), so workers that share a cache directory never remove each other's running attempts. The sweep never follows a link and leaves anything else in the directory alone, with one log line per name. A directory whose own mode user code set to 000 is made accessible again (mode 0700) once it is old enough to sweep, then removed like any other. If that fails (for example the cache is on a filesystem that refuses the change), it is left in place and a warning names it once per worker start: remove it as the worker's user. The sweep runs only between tasks, so one long task delays it.
• Upgrade every worker before you deploy Python assets, and before you deploy an artifact asset with a newer platform version: an older worker fails a Python task permanently. A worker that knows manifest versions 1 to 4 and meets a newer one refuses it for good with the project version's manifest has version 5, but this worker understands versions 1 to 4; update the worker to the platform version that deployed it, or redeploy with this worker's version (the numbers differ). Upgrade the workers first, so that no worker is older than the platform. The order and the way back are in Upgrade to the release with artifacts.
• The settings for artifacts (LAKANA_ARTIFACTS__*) are in deploy-with-compose.md, "Storage for artifacts".
• The Compose bundle's setup is in deploy-with-compose.md, "Python assets".