Add and tune sources¶
Bring a new source into Jezero without flooding the library: try it on a few sessions, check the result, then let it run on its own. This guide also covers projects, summarising again, redaction and the summarizer settings worth tuning.
Before you start¶
- A working server, as in Run Jezero as a service. The commands below run in
the folder that holds
jezero.toml, with.venv/bin/jezerooruv run jezero. - Read access, from the server, to the source you want to add (an agent's session folder or database), or a computer that uploads it with the plugins (see Manage computers and keys).
What a connector is¶
A connector reads one source and turns what is new into
documents for the summarizer. Each one is a [[connectors]] block in jezero.toml:
nameis the source's name. It appears on every item, and clients are limited to sources by name patterns, such asagent-*. Lowercase letters, digits,-and_.typepicks the implementation.optionsholds the settings of that type: paths, filters, how long to wait.
Each connector keeps a cursor, so a run only reads what changed since the last one, and a session already stored with the same text is never summarised twice.
| Type | Reads |
|---|---|
claude_code |
Claude Code transcripts (*.jsonl under a projects folder). |
codex |
Codex CLI transcripts (*.jsonl under a sessions folder; the path may be a glob). |
hermes |
Hermes Agent sessions, from its state.db or from device uploads. |
opencode |
OpenCode sessions, from its SQLite store or from device uploads. |
antigravity |
Antigravity conversations, from its local store or from device uploads. |
paperclip |
Issues of the Paperclip orchestrator, with comments and documents, through its API. |
jsonl is not a connector to schedule: load such files with
jezero import-jsonl. A [[connectors]] block
of type jsonl stops the config from loading (connector 'notes': jsonl is import-only:
use `jezero import-jsonl` instead of a [[connectors]] entry). gmail is planned.
jezero check-config marks the types your version does not implement yet. Every option
of every type is in Connectors.
Add a source¶
Pilot every new source before it runs on a schedule: it is cheaper to fix a filter on three sessions than on three thousand.
- Add the connector, disabled. For example, Claude Code sessions on the server:
[[connectors]]
name = "agent-claude-code"
type = "claude_code"
enabled = false
interval_minutes = 15
[connectors.options]
projects_dir = "~/.claude/projects"
Sessions from other computers need no block written by hand:
jezero device approve adds one disabled
connector per agent for that computer.
- Check that the config loads:
You should see the connector listed:
- Count the work without calling the model.
--connectorruns a connector even when it is disabled;--dry-runstores nothing and leaves the cursor alone:
You should see:
{"connector": "agent-claude-code", "docs": 3, "unchanged": 0, "stored": 0, "failed": 0, "deferred_sessions": 0, "redactions": 0, "input_chars": 3186, "input_tokens": 0, "output_tokens": 0, "pruned": 0, "dry_run": true, "sample": false, "note": null, "errors": [], "warnings": []}
docs is how many documents the first full run would summarise (a long session gives
several), input_chars how much text the model would read, and deferred_sessions
the sessions changed in the last settle_minutes (30 by default), which wait until
they are quiet. If the numbers are far from what you expect, tighten the filters (each
type has its own, such as skip_cwd_globs or min_chars) and count again. A path
option that points nowhere fails here already, with FileNotFoundError: projects_dir
folder not found: PATH.
- Summarise a sample.
--sample Ntakes the newest N sessions and stores their items. It is a pilot: the connector's cursor, last run and last success stay as they were, so the schedule andjezero healthstill see a connector that never ran:
You should see "stored" above 0, "failed": 0 and "sample": true:
{"connector": "agent-claude-code", "docs": 2, "unchanged": 0, "stored": 2, "failed": 0, "deferred_sessions": 0, "redactions": 0, "input_chars": 2124, "input_tokens": 200, "output_tokens": 100, "pruned": 0, "dry_run": false, "sample": true, "note": null, "errors": [], "warnings": []}
When no session is ready yet, the report says "note": "nothing to summarise" (with
the number still deferred) and the pilot needs no summarizer at all.
- Read what it wrote:
You should see one line per item, with its id, date, source, project and title:
Check that titles are specific, that projects are right (see Projects) and that nothing secret got through.
- Enable the connector (
enabled = true). Since the pilot did not count as a run, the next tick of the timer (jezero run --due, every 5 minutes) runs it at once. This first run reads the whole source, oldest first, and the sampled sessions come back asunchanged, without summarizer calls:
{"connector": "agent-claude-code", "docs": 3, "unchanged": 2, "stored": 1, "failed": 0, "deferred_sessions": 0, "redactions": 0, "input_chars": 1062, "input_tokens": 100, "output_tokens": 50, "pruned": 0, "dry_run": false, "sample": false, "note": null, "errors": [], "warnings": []}
{"due": {"ran": ["agent-claude-code"], "not_due_until": {}}}
For a large source, run it by hand in steps instead, with --limit N, which processes
the oldest N sessions and moves the cursor; repeat until docs reaches 0. If the timer
fires during your run, it skips with {"skipped": "another run is in progress"}.
From then on the timer runs it. Keep an eye on Jezero tells you if it stops.
Choose how often it runs¶
The timer starts jezero run --due every 5 minutes. --due runs each enabled connector
whose interval_minutes (15 by default) has passed since its last run, successful or not.
A value below 5 therefore means every 5 minutes, and a broken connector retries once per
interval, not on every tick.
How fresh items are also depends on settle_minutes: a session is read only once it has
been quiet that long, so a session reaches Jezero roughly between settle_minutes and
settle_minutes + interval_minutes after its last message.
Tag items with the right project¶
Each item gets a project, so agents can search one project and see its recent work. Jezero takes it from the source when it can, which beats the model's guess:
- A device's answer: sessions uploaded by the plugins carry the project their computer found.
[projects] paths: folders you map by hand.~matches any home folder, on any computer. The longest match wins.- The git repository the session ran in; for a worktree, the main repository.
[projects] roots: the folder right below one of these names (projects,code,dev,src,reposand other common ones by default).- The connector's own
project, a default for everything it reads. - Nothing found: the summarizer chooses, preferring the slugs in
summarizer.known_projects.
aliases rename the result, for every connector. The session connectors also accept a
project_aliases option that applies to that connector only. paperclip maps its own
project names with project_map, and the [projects] aliases and its project_aliases
then rename the result, as for the other connectors.
- Add your rules:
[projects]
paths = { "~/work/billing-api" = "billing", "/srv/stacks" = "atlas" }
aliases = { "shop-web" = "shop", "atlas-wt" = "atlas" }
[summarizer]
known_projects = ["atlas", "billing", "shop"]
- Preview what changes for items already stored.
jezero retagreads the source again from the start and applies the current rules, without calling the model:
You should see how many items would change:
- Apply it:
retag only changes items whose source gives a project; items where the model chose stay
as they are. The git rule needs the session's folder to exist on the server. For
sessions from other computers, set the same rules in the device's
~/.jezero/client.json, under "projects", so its plugin tags them before upload (see
Client).
Summarise items again¶
Summarise stored items again after you change the model or output_language, or when a
batch came out poor. Jezero works from the original text it kept (after redaction), and
keeps each item's source, id, date and project. It calls the model once per item.
- Find the ids.
jezero searchprints them in brackets. For every item of one source:
python3 -c "import sqlite3; db = sqlite3.connect('file:data/jezero.db?mode=ro', uri=True); [print(r[0]) for r in db.execute(\"SELECT id FROM items WHERE source = 'agent-claude-code'\")]" > ids.txt
- Summarise them:
Or name a few with --ids 12,15,31. You should see a report like the one of a run,
with "stored" equal to the number of items and "failed": 0.
Hide more secrets¶
Jezero redacts common credential shapes (API keys, tokens, private keys, passwords in assignments and URLs) before anything is summarised, stored or indexed. Add your own patterns for what only you know, such as internal host names or customer ids.
- Add regular expressions (Python syntax) to
[redaction]. Single quotes keep backslashes as they are:
Each match becomes [REDACTED:custom]. An invalid pattern makes the config fail to
load, with the error.
- New sessions are redacted with the new rules from the next run. For items already stored, preview, then apply:
You should see:
It rewrites titles, summaries, questions, atoms and the kept original text, and keeps the search index in sync. It makes no model calls.
Warning
Redaction catches common shapes; it is not a guarantee. Keep sources known to carry
secrets out with the connector's filters, such as skip_cwd_globs or
skip_dir_globs. Files uploaded to the inbox are kept as sent: redaction applies when
they are read, not to the files.
Tune the summarizer¶
The summarizer settings live in
[summarizer]. The ones worth tuning:
| Key | Default | Change it when |
|---|---|---|
model |
none | Always set it. A small, fast model is enough; try a few items with --sample before a backfill. |
output_language |
the document's | Your sessions mix languages. One language for every item lets one query find them all. Run resummarize on older items afterwards. |
max_input_chars |
60000 |
Long sessions are split into parts of at most this size, each its own item. Raise it for fewer, broader items, within the model's context window. |
max_concurrency |
4 |
You hit rate limits (lower it) or a backfill is slow and your quota allows more (raise it). Jezero already retries on 429 and server errors. |
timeout_seconds |
120 |
A slow local model times out on long parts. |
known_projects |
empty | The model invents project names for sessions without a folder hint. |
Note
Set max_input_chars before the first backfill. Parts are identified by where they
start, so after a change a session that is read again is cut differently and its parts
are summarised again. Parts that no longer exist are removed (pruned in the report).
Sessions read before the change keep their old parts until they are read again: see
Read a source again.
Read a source again¶
A connector reads only what changed since its cursor. To make it read its whole source
again, for example after you changed its filters, [redaction] or max_input_chars, or
after an update that reads more of a format, forget the cursor:
You should see:
reset agent-claude-code: its next run reads the whole source again. Stored items are kept; units whose redacted text is unchanged are skipped by content hash, without summarizer calls.
Without --yes it asks first, and in a script without a terminal it refuses with
error: pass --yes to confirm (no terminal to ask). A connector that never stored a
cursor answers agent-claude-code has no cursor yet: its next run reads everything
anyway. It also refuses while a run is in progress.
The next run, by hand or by the timer, then reads everything. What it costs depends on what you changed:
- Nothing that changes the text sent to the model: every unit comes back
unchanged, and the run makes no summarizer calls.
{"connector": "agent-claude-code", "docs": 3, "unchanged": 3, "stored": 0, "failed": 0, "deferred_sessions": 0, "redactions": 0, "input_chars": 0, "input_tokens": 0, "output_tokens": 0, "pruned": 0, "dry_run": false, "sample": false, "note": "nothing to summarise", "errors": [], "warnings": []}
- Redaction rules, filters that change the rendered text, or
max_input_chars: the units whose text changed are summarised again, one call each. With a newmax_input_chars, parts that no longer exist are removed:
{"connector": "agent-claude-code", "docs": 3, "unchanged": 0, "stored": 3, "failed": 0, "deferred_sessions": 0, "redactions": 0, "input_chars": 3186, "input_tokens": 300, "output_tokens": 150, "pruned": 3, "dry_run": false, "sample": false, "note": null, "errors": [], "warnings": []}
Run jezero run --connector NAME --dry-run after the reset to see docs and
input_chars before you pay for it. To only fix projects, use jezero retag, which
calls no model; to change the summaries themselves, jezero resummarize.
Next steps¶
- Connectors: every option of every type.
- Keep an eye on Jezero: alerts when a source stops.
- Manage computers and keys: sessions from other computers, and which sources each key may read.
- How summaries are made: items, atoms and what they cost.