AI Agents¶

Top limit + 1 values of column by frequency; the extra row reveals truncation.

Whether some tables are missing from the data summary, so browsing the catalog can find more.

Run a read-only SQL statement on the named Lumen source and return a text preview of results.

Intended for LLM tool calls: the model supplies the logical source name and sql_query; the sources map must be provided by the caller (e.g. closed over when building the tool).

Parameters:

Returns:

Render an exploration query result as a compact preview for the LLM.

Reports the fetched row count (or a lower bound if capped), column dtypes, and a few example rows. Markdown omits index and alignment padding.

Only the handful of cells actually rendered is converted to pandas, so a polars or pyarrow result reaches the model without the whole frame being copied. The dtypes reported are the preview's for that reason, which is also what keeps a pandas caller's output exactly as it was.

Expose :func:make_apply_filter_tool as a context-gated llm_tools entry.

Returns the tool only when a pipeline is present in working memory, so it is offered to the LLM exactly when there is an existing exploration to filter.

Build a :class:~lumen.ai.tools.FunctionTool that filters pipeline.

The tool lets the chat agent narrow the data already loaded in the current exploration -- subsetting an xarray coordinate dimension (time/lat/ lon/level) or a tabular column. It adds a :class:~lumen.filters.base.Filter to the pipeline (the same machinery as the manual "Add Filter" UI), which subsets the data without modifying the SQL query.

Known limitation: this tool is registered on :class:SQLAgent, whose response also emits a SQL query. A single "filter" request can therefore both apply this in-place pipeline filter to the current exploration and open a new exploration whose SQL carries an equivalent WHERE clause. Giving the tool a dedicated, non-SQL-emitting home is tracked as follow-up work.

Tool wrapping :meth:~lumen.ai.schemas.Metaset.table_list for incremental catalog browsing.

Tool that lists the most frequent values of one column, optionally filtered by a pattern.

A focused alternative to free-form exploration for the most common reason to explore: finding the exact stored spelling of a literal before filtering on it.

Tool returning full column statistics for specific catalog tables, computing them if the prompt only had names and types.

Build a :class:~lumen.ai.tools.FunctionTool that runs :func:execute_exploration_sql for sources.

Create a SQL query model with source/table validation.

Parameters:

Create a table model with constrained table choices.

Lower-cased names of the columns sql_query references.

None when every column is referenced (a * projection) or the query cannot be parsed, so callers fall back to considering all columns.

Resolve the source argument of an exploration tool call.

Weak models often pass a table name, a generic source/<table> or a differently cased name instead of the source name, so any reference that identifies one source unambiguously is accepted.

Resolve a model-supplied table reference to its source and table name (see :func:resolve_table_slug).

Whether sql_query collapses many input rows into fewer output rows.

Such a query has already collapsed its inputs by the time it returns, so nothing about the result shows what those inputs carried: a -9999 placeholder is already inside SUM, and a blank string is already a GROUP BY key. Profiling the result therefore says nothing about the data the numbers came from, and the source rows have to be profiled separately -- otherwise a chart of SUM(revenue) silently inherits every placeholder in the table it was built from.

Unparseable SQL returns False: the caller only uses this to decide whether to spend an extra query, and guessing "yes" would spend it on every failed parse.

Whether the outermost SELECT collapses everything into a single row.

True for SELECT COUNT(*) FROM t or SELECT MAX(x) / MIN(x) FROM t: aggregates in every projection and no GROUP BY. Unparseable SQL returns False.

Aggregates inside a subquery or wrapped in a row-generating function do not count: SELECT (SELECT MAX(b.v) FROM b WHERE b.id = a.id) FROM a and SELECT UNNEST(LIST(x)) FROM t return many rows.

Condense an attempt's tool calls into what a retry needs to know.

A retry that sees what was already looked up can go straight to fixing the query instead of repeating the discovery that preceded the failure.

Number each chart/table so the story can reference them by their catalog number.

Deliberately omits the view's title: it is the pipeline step name (e.g. "Prepare data", "Create pie chart"), which would leak into the story. The data summary describes what each figure actually shows instead.