Starlark

aq has three Starlark entry points:

• aq --starlark 'EXPR'
• aq --starlark-file path/to/script.star
• aq -P

Inline mode evaluates a Starlark expression with data bound to the current input value. File mode executes the file and requires main(data).

For arbitrary no-input Starlark evaluation, use -n:

aq -n --starlark '1 + 2'

Execution Model

• Inline mode executes inline Starlark source as a module and returns the final expression value.
• File mode executes the file, then calls main(data).
• REPL mode evaluates one snippet at a time in a persistent Starlark module, so assignments and def statements survive across later inputs.
• data is the incoming aq value for the current document.
• aq is the helper namespace.
• log(value) writes one Starlark value to stderr and returns None.
• Standard Starlark builtins like dir, len, type, comprehensions, and load(...) are available through the embedded engine.
• If you do not pass -n, Starlark mode follows normal aq input rules:
  • no file arguments means read stdin
  • use - as an explicit stdin file argument when you want stdin at a specific file position
  • one or more file arguments means parse those files
  • --slurp changes data from “current document” to “array of all input documents”
• REPL mode binds one session-level data value:
  • -n starts with data = null
  • --slurp starts with data = [...] across all loaded input documents
  • one loaded document starts with that value directly
  • multiple loaded documents without --slurp collapse into data = [...] for one interactive session
  • if stdin is already being used for input data, REPL commands are read from the controlling terminal instead
• Output to stdout is the returned value from the inline program or main(data), rendered through normal aq output flags like -r, -c, and --output-format.
• There is no print(...) helper. Use log(value) for stderr diagnostics. If you want text on stdout, return a string, often via aq.render(...).

REPL

The REPL is intentionally small, but it is a real persistent session:

aq -n -P

When the REPL is attached to a real terminal, it now uses a line editor:

• arrow-key history navigation
• in-line editing
• tab-triggered IDE-style completion menus for REPL commands, top-level session names, paths, formats, and aq.* helpers
• completion details for commands, constants, formats, and individual aq.* helpers
• aq.* helper details include both a call signature and a one-line summary in the right-hand menu pane
• Shift-Tab and arrow-key menu navigation
• Ctrl-C cancels the current input without leaving the REPL
• Ctrl-D exits from an empty prompt
• multiline entry for def, dict/list literals, and other incomplete input

Interactive history is saved between sessions when a writable history path is available:

• $AQ_STARLARK_REPL_HISTORY if set
• otherwise $XDG_STATE_HOME/aq/starlark-history.txt
• otherwise $HOME/.local/state/aq/starlark-history.txt
• on Windows, %USERPROFILE%/.aq/starlark-history.txt

If you drive the REPL through a pipe instead of an interactive terminal, aq keeps the older line-at-a-time fallback so scripted REPL tests and command feeds still work.

Useful REPL commands:

• :help Shows the built-in REPL commands.
• :data Prints the current top-level data value.
• :data EXPR Evaluates EXPR, stores the result back into data, and prints the new value.
• :type EXPR Evaluates EXPR and prints its Starlark type without changing ans.
• :doc NAME Prints a structured help record for a REPL command, aq.* helper, well-known constant, or current session binding.
• :constants Prints the well-known REPL constants and literals, including data, aq, ans, prev, _, None, True, and False.
• :capabilities Prints which restricted Starlark capability groups are enabled in the current session.
• :globals Prints the current top-level names defined in the session.
• :aq [PREFIX] Prints the available aq.* helper names, optionally filtered by a prefix like slug or sha.
• :load PATH Evaluates another Starlark file inside the current session, relative to the current base directory unless PATH is absolute.
• :format Prints the current REPL output format.
• :format FORMAT Changes the REPL output format for aq-compatible results.
• :pwd Prints the current base directory used for relative Starlark path operations.
• :reset Resets the session back to the original data value and clears earlier definitions.
• :quit, :exit Leaves the REPL.

The REPL also keeps result variables:

• ans The most recent non-None result.
• prev The previous ans value.
• _ Alias for ans.

Examples:

aq> x = 1
aq> x + 2
3
aq> :type x
"int"
aq> aq.sl<TAB>
[menu]
aq.slug
[detail]
aq.slug(text)
normalize text to a URL-safe slug
aq> :doc aq.slug
{"name":"aq.slug","kind":"helper","signature":"aq.slug(text)","description":"normalize text to a URL-safe slug"}
aq> ans * 10
30
aq> def inc(x):
...     return x + 1
aq> inc(4)
5

To explore piped input interactively:

cat config.json | aq -P

In that shape, aq reads config.json from stdin first, binds it to data, then switches REPL command input to the controlling terminal.

When the REPL starts on a real terminal, it prints a short session banner with the current data shape, detected input format, output format, enabled capability groups, and the main discovery hints.

Discoverability

The helper surface is introspectable from inside Starlark:

aq -n --starlark 'dir(aq)'

That prints the current aq.* helper names, including capability-gated helpers. Names may still require the relevant flag at runtime, for example aq.read(...) still needs --starlark-filesystem, and aq.now() still needs --starlark-time.

Inside the REPL, :aq is the quick version:

aq> :aq sha
["aq.sha1","aq.sha256","aq.sha512"]

For input-aware inspection, the useful top-level anchors are:

• data The incoming value for the current document, or an array under --slurp.
• aq The helper namespace. Use dir(aq) or :aq to see the available aq.* helpers.
• ans, _ The most recent non-None result from the REPL. They appear after the first successful result.
• prev The previous ans value. It appears after the second successful non-None result.
• None, True, False Standard Starlark literals, available in the REPL like any other Starlark module.
• aq.format() The detected input format for the current document, or "mixed" when slurping heterogeneous inputs.
• aq.base_dir() The current filesystem base directory for relative Starlark path operations.

Starlark is restricted by default. These capabilities are opt-in:

• --starlark-filesystem enables path helpers, aq.read*, aq.write*, and local load(...)
• --starlark-environment enables aq.env(...)
• --starlark-time enables aq.timestamp(), aq.now(), and aq.today()
• --starlark-unsafe enables all of the above

Engine

aq currently embeds Meta's Rust starlark crate. That is an implementation choice, not a public API promise.

For v1, the supported contract is the CLI behavior documented here: the available aq.* helpers, the capability flags, and the data model visible to Starlark scripts. Internal embedding details may change later if a different engine or integration shape becomes a better fit.

Support Snapshot

aq Starlark is in good shape for real CLI work now:

• inline eval with data
• file mode via main(data)
• a persistent REPL with history, completion, and session commands
• a bridge back into the aq query engine
• typed temporal values
• a real structured-data/path helper layer
• optional filesystem, environment, and time capabilities

The easiest way to read the helper surface is by group, not by a flat dir(aq) dump.

Helper Reference

Type Conventions

Term	Meaning
value	Any aq-compatible structured value.
values	list[value].
text	UTF-8 string.
format	One of json, jsonl, toml, yaml, csv, tsv, or output-only table.
path	`list[string
paths	list[path].
date	Typed Starlark date value.
datetime	Typed Starlark datetime value.
timedelta	Typed Starlark duration value.
transform(value)	Callback returning a replacement value.
transform(path, value)	Callback receiving the current path and value, then returning a replacement value.
predicate(path, value)	Callback returning bool.

Query and Rendering Helpers

Helper	Inputs	Returns	Notes
aq.format()	none	string	Current detected input format, or "mixed" when --slurp loaded heterogeneous inputs.
aq.query_all(filter, value)	filter: string, value: value	values	Runs an aq query and returns the full result stream.
aq.query_one(filter, value)	filter: string, value: value	value	Requires exactly one result.
aq.parse(text, format)	text: string, format: string	value	Parses one text payload using aq's normal collapse rules.
aq.parse_all(text, format)	text: string, format: string	values	Always returns every parsed document.
aq.render(value, format, compact = False)	value: value, format: string, compact: bool	string	Renders one value to text.
aq.render_all(values, format, compact = False)	values: values, format: string, compact: bool	string	Renders multiple values using normal aq multi-result output semantics.

Temporal Helpers

Helper	Inputs	Returns	Notes
aq.date(text)	text: string	date	Parses an ISO-style date.
aq.datetime(text)	text: string	datetime	Parses an ISO-style datetime.
aq.timedelta(...)	duration fields	timedelta	Supports weeks, days, hours, minutes, seconds, milliseconds, microseconds, nanoseconds.
date methods	receiver: date	date, datetime, int, string	.weekday(), .isoformat(), .replace(...), .at(...), plus .year, .month, .day, .ordinal.
datetime methods	receiver: datetime	date, datetime, float, int, string	.date(), .timestamp(), .weekday(), .isoformat(), .replace(...), plus .year, .month, .day, .hour, .minute, .second, .ordinal.
timedelta.total_seconds()	receiver: timedelta	float	timedelta is Starlark-only and cannot be emitted directly to aq output formats.
typed temporal preservation	parsed TOML dates/datetimes	typed values	TOML dates and datetimes stay typed inside Starlark instead of flattening to strings immediately.

Structured Data and Path Helpers

Helper	Inputs	Returns	Notes
aq.merge(left, right, deep = False)	value, value, bool	value	Right-biased merge. deep = True merges nested objects recursively.
aq.merge_all(values, deep = False)	values, bool	value	Folds aq.merge(...) across a non-empty list.
aq.drop_nulls(value, recursive = False)	value, bool	value	Removes null object fields and array elements.
aq.sort_keys(value, recursive = False)	value, bool	value	Sorts object keys lexicographically.
aq.get_path(value, path)	value, path	value	Missing paths and out-of-range indexes yield null.
aq.set_path(value, path, replacement)	value, path, value	value	Creates missing object/array structure as needed.
aq.delete_path(value, path)	value, path	value	Removes one path.
aq.delete_paths(value, paths)	value, paths	value	Removes many paths.
aq.pick_paths(value, paths)	value, paths	value	jq-style projection, including null for missing picked paths.
aq.omit_paths(value, paths)	value, paths	value	Removes explicit path lists.
aq.walk(value, function)	value, transform(value)	value	Bottom-up recursive transform.
aq.walk_paths(value, function)	value, transform(path, value)	value	Like aq.walk(...), but includes the current path.
aq.paths(value, leaves_only = False)	value, bool	paths	Stable depth-first traversal, excluding the root path.
aq.find_paths(value, function, leaves_only = False)	value, predicate(path, value), bool	paths	Returns the matching paths.
aq.collect_paths(value, function, leaves_only = False)	value, callback, bool	values	Returns callback results in traversal order.
aq.pick_where(value, function, leaves_only = False)	value, predicate(path, value), bool	value	Predicate-driven projection.
aq.omit_where(value, function, leaves_only = False)	value, predicate(path, value), bool	value	Predicate-driven redaction.
aq.clean_k8s_metadata(value)	value	value	Keeps portable manifest metadata, currently name, generateName, namespace, labels, and annotations, while dropping live-object fields like uid, resourceVersion, managedFields, and ownerReferences. Applies at manifest scope and within List.items.

Text, Regex, URL, Hashing, and Release Helpers

Helper	Inputs	Returns	Notes
aq.regex_is_match(pattern, text)	string, string	bool	Built on Rust regex.
aq.regex_find(pattern, text)	string, string	`string	None`
aq.regex_find_all(pattern, text)	string, string	list[string]	Every match in order.
aq.regex_capture(pattern, text)	string, string	`dict	None`
aq.regex_capture_all(pattern, text)	string, string	list[dict]	Capture metadata for every match.
aq.regex_split(pattern, text)	string, string	list[string]	Splits text by regex pattern.
aq.regex_replace(pattern, replacement, text)	string, string, string	string	First replacement only.
aq.regex_replace_all(pattern, replacement, text)	string, string, string	string	Global replacement.
aq.base64_encode(text, urlsafe = False, pad = True)	string, bool, bool	string	UTF-8 text in, base64 text out.
aq.base64_decode(text, urlsafe = False)	string, bool	string	UTF-8 text only. Binary payloads should use file helpers instead.
aq.slug(text)	string	string	URL-safe slug.
aq.snake_case(text)	string	string	snake_case normalization.
aq.kebab_case(text)	string	string	kebab-case normalization.
aq.camel_case(text)	string	string	camelCase normalization.
aq.title_case(text)	string	string	Title-cased label text.
aq.trim_prefix(text, prefix)	string, string	string	Leaves input unchanged when the prefix is absent.
aq.trim_suffix(text, suffix)	string, string	string	Leaves input unchanged when the suffix is absent.
aq.regex_escape(text)	string	string	Escapes literal text for safe regex embedding.
aq.shell_escape(text)	string	string	Escapes one POSIX shell argument.
aq.url_encode_component(text)	string	string	Percent-encodes one URL component.
aq.url_decode_component(text)	string	string	Percent-decodes one URL component. Does not treat + as space.
aq.hash(text, algorithm = "sha256", encoding = "hex")	string, string, string	string	Supports sha1, sha256, sha512, and blake3.
aq.sha1(text, encoding = "hex")	string, string	string	Convenience wrapper.
aq.sha256(text, encoding = "hex")	string, string	string	Convenience wrapper.
aq.sha512(text, encoding = "hex")	string, string	string	Convenience wrapper.
aq.blake3(text, encoding = "hex")	string, string	string	Convenience wrapper.
aq.semver_parse(text)	string	dict	Parses SemVer 2.0.0 into structured fields.
aq.semver_compare(left, right)	string, string	int	Returns negative, zero, or positive ordering.
aq.semver_bump(text, part, prerelease_label = "rc")	string, string, string	string	Supports major, minor, patch, prerelease, and release.

Filesystem and File I/O Helpers

These require --starlark-filesystem or --starlark-unsafe. In --starlark-file mode, relative aq.read* paths and load("...") resolve against the script file directory. Inline --starlark mode resolves them against the current working directory.

Helper	Inputs	Returns	Notes
aq.base_dir()	none	string	Current base directory for relative filesystem operations.
aq.resolve_path(path)	string	string	Lexically resolves against the current base directory.
aq.relative_path(path, start = ".")	string, string	string	Lexical relative path helper.
aq.exists(path)	string	bool	Path existence check.
aq.is_file(path)	string	bool	File check.
aq.is_dir(path)	string	bool	Directory check.
aq.list_dir(path = ".")	string	list[string]	Sorted directory entry names.
aq.walk_files(path = ".", include_dirs = False)	string, bool	list[string]	Recursive relative walk.
aq.glob(pattern, include_dirs = False)	string, bool	list[string]	Supports *, ?, and **. Absolute patterns are rejected.
aq.mkdir(path, parents = False)	string, bool	string	Creates a directory and returns the resolved path.
aq.read_text(path)	string	string	Reads UTF-8 text.
aq.read_text_glob(pattern)	string	list[dict]	Returns [{path, text}].
aq.rewrite_text(path, function)	string, callback	int	In-place text rewrite. Callback receives (path, text) and returns replacement text.
aq.rewrite_text_glob(pattern, function)	string, callback	list[dict]	Batch text rewrite over matching files.
aq.hash_file(path, algorithm = "sha256", encoding = "hex")	string, string, string	string	Hashes raw file bytes, including binary payloads.
aq.read(path)	string	value	Reads and parses another file using normal detection rules.
aq.read_as(path, format)	string, string	value	Explicit format override.
aq.read_all(path)	string	values	Always returns every parsed document.
aq.read_all_as(path, format)	string, string	values	Explicit format override.
aq.read_glob(pattern)	string	list[dict]	Returns [{path, value}].
aq.read_glob_as(pattern, format)	string, string	list[dict]	Explicit format override.
aq.read_glob_all(pattern)	string	list[dict]	Returns [{path, index, value}].
aq.read_glob_all_as(pattern, format)	string, string	list[dict]	Explicit format override.
aq.write_text(path, text, parents = False)	string, string, bool	int	Atomically writes UTF-8 text and returns bytes written.
aq.write_text_batch(entries, parents = False)	list[dict], bool	list[dict]	Writes [{path, text}] and returns [{path, bytes}].
aq.write(path, value, format, compact = False, parents = False)	string, value, string, bool, bool	int	Atomically writes one rendered value.
aq.write_all(path, values, format, compact = False, parents = False)	string, values, string, bool, bool	int	Atomically writes a multi-result rendering.
aq.write_batch(entries, format, compact = False, parents = False)	list[dict], string, bool, bool	list[dict]	Writes [{path, value}] and returns [{path, bytes}].
aq.write_batch_all(entries, format, compact = False, parents = False)	list[dict], string, bool, bool	list[dict]	Writes [{path, values}] and returns [{path, bytes}].
aq.stat(path)	string	`dict	None`
aq.copy(source, destination, overwrite = False)	string, string, bool	int	Copies one file and returns the copied byte count.
aq.rename(source, destination, overwrite = False)	string, string, bool	string	Renames or moves a path and returns the resolved destination.
aq.remove(path, recursive = False, missing_ok = False)	string, bool, bool	bool	Removes a file or directory.
load("...")	relative or absolute path	module names	Enabled for local Starlark files when filesystem capability is on.

Restricted Helpers

These stay gated behind explicit flags even though dir(aq) will still show the names.

Helper	Requires	Inputs	Returns	Notes
aq.env(name)	--starlark-environment	string	`string	None`
aq.timestamp()	--starlark-time	none	datetime	Current UTC datetime.
aq.now()	--starlark-time	none	datetime	Alias-style current UTC datetime helper.
aq.today()	--starlark-time	none	date	Current UTC date.
all restricted groups	--starlark-unsafe	n/a	n/a	Enables filesystem, environment, and time helpers at once.

Pipeline Patterns

Starlark mode is most useful when you want just a bit more control flow or library-style structure than a plain aq query gives you.

Read one document from stdin and return a transformed value:

printf '{"service":{"name":"api","port":8080}}\n' \
  | aq --starlark '{"name": data["service"]["name"], "port": data["service"]["port"]}'

Slurp many documents and work over the whole batch:

printf '{"name":"api"}\n{"name":"worker"}\n' \
  | aq --slurp --starlark '[item["name"] for item in data]' -r

Return rendered text instead of structured output:

aq --starlark 'aq.render({"app": data["name"], "port": data["port"]}, "yaml")' -r config.json

Write side files and still return a structured summary to stdout:

aq -n --starlark --starlark-filesystem '
written = aq.write("out/config.json", {"name": "aq"}, "json", compact = True, parents = True)
{"bytes": written, "path": "out/config.json"}
'

Bridge back into the aq query engine from Starlark when that is simpler than rewriting the query in pure Starlark:

aq --starlark '{"active": aq.query_all(".users[] | select(.active)", data)}' users.json

The practical rule is:

• return values for stdout
• use aq.render(...) when stdout should be plain text
• use aq.write* or aq.rewrite_text* for filesystem side effects
• use --slurp when the Starlark logic wants the whole batch at once

Restricted helpers:

• aq.env(name)
• aq.timestamp()
• aq.now()
• aq.today()

Examples

The full runnable catalog lives in examples/starlark/README.md in the repo root. That is the best place to browse copy-pasteable commands.

The examples roughly break down like this:

• Query and transform: users_over_30.star, manifest_summary.star, embedded_config.star, merge_overlay.star
• Temporal values: shift_release_window.star, calendar_rollup.star, time_snapshot.star
• Cleanup and structured redaction: clean_k8s_metadata.star, normalize_strings.star, find_secretish_fields.star, redact_secretish_fields.star
• Filesystem and reporting: config_map_names.star, self_inventory.star, library_index.star, fingerprint_library.star, stage_report.star
• Batch read/write: manifest_tree_summary.star, emit_manifest_summaries.star, emit_note_copies.star, normalize_note_files.star
• Text and hashing: sanitize_contacts.star

Worked Example Runs

These are the best representative runs to skim first.

Clean Kubernetes metadata before re-rendering YAML:

cat <<'YAML' | aq --starlark-file examples/starlark/clean_k8s_metadata.star --output-format yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: staging
  uid: abc123
  ownerReferences:
    - apiVersion: apps/v1
      kind: Deployment
      name: app
  resourceVersion: "7"
  creationTimestamp: "2024-01-01T00:00:00Z"
  managedFields:
    - manager: kubectl
  annotations:
    note: keep-me
  labels:
    tier: backend
YAML

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: staging
  annotations:
    note: keep-me
  labels:
    tier: backend
    managed-by: aq

Normalize and patch nested strings:

printf '%s\n' '{"metadata":{"labels":{"tier":"backend ","name":" api "}},"items":[" one ",2]}' \
  | aq --starlark-file examples/starlark/normalize_strings.star --output-format json --compact

{"metadata":{"labels":{"tier":"BACKEND","name":"api"}},"items":["one",2]}

Generate a TOML config with no input:

aq -n --starlark-file examples/starlark/generate_app_toml.star --output-format toml

[app]
name = "aq"
port = 8443
features = [
    "query",
    "starlark",
]

[database]
host = "db.internal"
pool = 16

For the broader command catalog, see examples/starlark/README.md in the repo root.

Current limits:

• timedelta is currently a Starlark-only value. It cannot be emitted directly as an aq output value or rendered to output formats.
• --stream and --explain are not supported in Starlark mode yet.
• The aq.* surface is intentionally curated rather than exhaustive. It now covers a useful path/filesystem layer, but richer schema helpers and broader format helpers are still future work.