Skip to content

API (PlanFrame core)

This is a light API reference intended for quick discovery. For the full surface area, see the generated reference for each module.

Optional API skins

Typed mixins (import from planframe.spark / planframe.pandas, or from planframe import spark / pandas) layer familiar naming on Frame without adding backend dependencies. Guides: PySpark-like API, pandas-like API.

planframe.spark.frame.SparkFrame

Bases: Frame[SchemaT, BackendFrameT, BackendExprT], Generic[SchemaT, BackendFrameT, BackendExprT]

Mixin-style subclass adding PySpark naming; combine with a concrete backend frame.

Example::

from planframe.spark import SparkFrame


class Users(PolarsFrame, SparkFrame):
    id: int

union(other)

PySpark union preserves duplicates (SQL UNION ALL).

planframe.pandas.frame.PandasLikeFrame

Bases: Frame[SchemaT, BackendFrameT, BackendExprT], Generic[SchemaT, BackendFrameT, BackendExprT]

Mixin-style pandas-flavored API; combine with a concrete backend frame.

assign(**columns)

Pandas-like assign, lowered to repeated with_column.

filter(*predicates, items=None, like=None, regex=None)

Row filter (PlanFrame) or column filter (pandas-style), depending on arguments.

  • df.filter(predicate) behaves like PlanFrame Frame.filter.
  • df.filter(items=...|like=...|regex=...) behaves like pandas column selection.

groupby(by, *, sort=False, dropna=True)

Pandas-like groupby, lowered to core group_by.

Note: PlanFrame does not implement pandas index semantics; this returns a PlanFrame grouped object with .agg(...).

melt(*, id_vars, value_vars, var_name='variable', value_name='value')

Pandas-like melt, lowered to core unpivot.

query(expr) 

query(expr: str) -> PandasLikeFrame[Any, Any, Any]

query(
    expr: Series[bool] | Expr[bool],
) -> PandasLikeFrame[Any, Any, Any]

Pandas-like query.

Supported forms:

  • Typed: Series[bool] / Expr[bool]
  • String (tiny subset): col op literal (no boolean ops, no functions, no parens)

rename_pandas(*, columns=None, errors='raise')

Pandas-style rename(columns=..., errors=...).

planframe.frame.Frame

Bases: Generic[SchemaT, BackendFrameT, BackendExprT]

source(data, *, adapter, schema) classmethod

schema()

plan()

optimize(*, level=...)

collect(*, name=..., options=...)

acollect(*, name=..., options=...) async

to_dicts(*, options=...)

ato_dicts(*, options=...) async

to_dict(*, options=...)

ato_dict(*, options=...) async

planframe.execution_options.ExecutionOptions

Backend-agnostic execution-time hints.

These options are only consulted at execution/materialization boundaries (e.g. collect, to_dicts, to_dict) and must not affect schema evolution.

planframe.plan.join_options.JoinOptions

Optional join hints for backends that support them (for example Polars).

Fields that are None are omitted when calling the backend so its defaults apply.

streaming / engine_streaming mirror :class:planframe.execution_options.ExecutionOptions (user-level streaming vs engine-level streaming); backends may support none, one, or both.

Polars (planframe-polars) mapping and precedence

Optional fields are forwarded to :meth:polars.LazyFrame.join when the installed Polars version supports the corresponding keyword. Applied in this order; later steps override earlier ones where both set the same underlying argument:

. coalesce, validate, join_nulls (as nulls_equal), maintain_order.

. streaming — sets allow_parallel = not streaming (disable parallel join paths

when the user prefers streaming-style execution).

. allow_parallel — overwrites allow_parallel from the previous step.

. force_parallel — sets Polars force_parallel (separate from allow_parallel).

. engine_streaming — set when supported by Polars (not all versions expose it).

planframe.execution.execute_plan

Execute a :class:planframe.plan.nodes.PlanNode tree.

This is the supported public plan interpreter used by :meth:planframe.frame.Frame.collect.

Important: - This returns the backend frame after applying the plan, but it does not call :meth:planframe.backend.adapter.BaseAdapter.collect unless collect=True is provided.

planframe.execution.execute_plan_async

Async counterpart of :func:execute_plan.

Runs the synchronous plan interpreter in a worker thread (:func:asyncio.to_thread) so callers can await plan execution without blocking the event loop. Transform interpretation remains synchronous; pair with :meth:~planframe.backend.adapter.BaseAdapter.acollect (or other async materializers) when the backend supports async I/O.

See also :meth:planframe.frame.Frame.acollect_backend / :meth:~planframe.frame.Frame.collect_async.

See also: Async execution contract and Thread-safety expectations in the Creating an adapter guide (docs/planframe/guides/creating-an-adapter.md; ReadTheDocs path planframe/guides/creating-an-adapter/).

planframe.materialize

Adapter-friendly materialization boundaries (columnar export + optional factory).

These mirror :meth:planframe.frame.Frame.to_dict / :meth:~planframe.frame.Frame.ato_dict with the same :class:~planframe.execution_options.ExecutionOptions contract — use them when you want a stable import (from planframe.materialize import ...) instead of calling Frame methods directly. See the Creating an adapter — Columnar boundary <https://planframe.readthedocs.io/en/latest/planframe/guides/creating-an-adapter/#columnar-boundary-materialize>__ section.

For chunked columnar export (optional adapter protocol, not wired here yet), see the Columnar streaming design note in the PlanFrame docs and AdapterColumnarStreamer in planframe.backend.io.

PlanFrame does not construct Pydantic/dataclass models here; supply a factory when needed.

amaterialize_columns(frame, *, options=None) async

Async columnar materialization (same as :meth:~planframe.frame.Frame.ato_dict).

amaterialize_into(frame, factory, *, options=None) async

Like :func:materialize_into, using the async to_dict path.

materialize_columns(frame, *, options=None)

Return columnar data for frame (dict[column_name, column_values]).

Thin wrapper around :meth:planframe.frame.Frame.to_dict — options are forwarded unchanged.

materialize_into(frame, factory, *, options=None)

Materialize columns, then pass them to factory.

factory can wrap Pydantic models, dataclasses, Arrow tables, or any custom type; PlanFrame stays agnostic to the output shape.

planframe.backend.io.AdapterColumnarStreamer

Optional protocol (spike) for chunked columnar batches — see Columnar streaming (design).

Bases: Protocol[BackendFrameT]

Optional adapter surface for chunked columnar export (design spike for 1.3+).

Each chunk is a columnar mapping dict[column_name, list[values]] where every value list has the same length (rows in that chunk). Chunk boundaries are adapter-defined (e.g. engine batch size). Column names should be consistent across chunks for a given materialization.

This is not the same as :class:AdapterRowStreamer, which streams rows (dict[str, object] per row) and is integrated with Frame.stream_dicts / Frame.astream_dicts. Columnar chunking is for hosts that want to build Arrow tables, batched numpy/Pandas loads, etc., without holding a full dict[str, list[object]] in memory.

Integration status: PlanFrame core does not yet call this protocol from :func:planframe.materialize.materialize_columns or Frame.to_dict. Adapters may implement it so hosts can isinstance(adapter, AdapterColumnarStreamer) and call the iterators after collect / acollect, forwarding the same :class:~planframe.execution_options.ExecutionOptions you would pass to to_dict. Use streaming / engine_streaming hints the same way as for other materializers.

Contract: if you claim support, implement both :meth:iter_columnar_chunks and :meth:aiter_columnar_chunks (mirroring :class:AdapterRowStreamer).

See the PlanFrame design note Columnar streaming (docs/planframe/design/columnar-streaming.md).

planframe.compile_context.PlanCompileContext

Internal helper shared by Frame and execute_plan for compiling expression IR and related structures. Adapter authors rarely import it directly; see Core layout.

Bases: Generic[BackendFrameT, BackendExprT]

Holds (adapter, schema) and compiles expressions and join metadata once per context.

compile_project_items(items)

Lower :class:~planframe.plan.nodes.Project items to :class:CompiledProjectItem.

planframe.plan.walk.iter_plan_nodes

Iterate plan nodes in a deterministic pre-order traversal.

Pre-order means the current node is yielded before its children.

By default this walks only the linear prev chain (the primary pipeline). Nodes that reference other frames (Join.right, concat other) are treated as boundaries and are not descended into unless include_side_frames=True.

When include_side_frames=True, the traversal yields: - the current node - then its prev subtree (depth-first) - then any side frame subtrees (depth-first), in a stable order: - for Join: RHS (right) after the left chain - for concats: other after the left chain