How does Daemion work?

Daemion is a local-first agent OS: a persistent gateway running on your machine, a 6-substrate kernel that handles everything from storage to streaming, and a universal extension model where every capability is data — not code.

System overview

Phone / Browser (PWA)
        │
        │  HTTPS via Tailscale (or localhost)
        ▼
┌──────────────────────────────────────────┐
│           Gateway  :3001 (default)       │
│                                          │
│  HTTP/WebSocket API                      │
│                                          │
│  ┌────────────────────────────────────┐  │
│  │         6 Kernel Substrates        │  │
│  │  Extension  │ Context  │ Execution │  │
│  │  Trigger    │ Storage  │ Presentation│ │
│  └────────────────────────────────────┘  │
│                                          │
│  ┌────────────────────────────────────┐  │
│  │          Local Storage             │  │
│  │  SQLite  ·  Engram (Neo4j)         │  │
│  │  Filesystem                        │  │
│  └────────────────────────────────────┘  │
└──────────────────────────────────────────┘
        │
        │  Agent SDK (not child_process)
        ▼
   Claude API

The frontend is a static PWA served from Vercel — just a window into your local system. All messages, threads, extensions, and config live in SQLite on your machine. The gateway serves everything; the frontend never talks to an external database.

What is the gateway?

The gateway is a local HTTP/WebSocket server (port 3001 by default). It is the single entry point for all client communication — every message, thread list, extension CRUD, job run, and streaming response goes through it.

Core API surface:

Method	Path	What it does
`GET`	`/health`	Health check, no auth required
`POST`	`/chat`	Send a turn, get a streaming response
`GET`	`/threads`	List conversation threads
`GET`	`/threads/:id/turns`	Turns for a thread
`POST`	`/threads`	Create a thread
`POST`	`/run/:job`	Execute a job by name
`GET`	`/extensions`	List all extensions
`POST`	`/extensions`	Create or update an extension
`DELETE`	`/extensions/:id`	Remove an extension
`POST`	`/reseed`	Re-sync built-in extensions from disk (no restart needed)
`WS`	`/stream`	Streaming turns and tool-call events

The API data model uses “turns” throughout — not “messages.” A turn is one exchange unit in a thread.

The gateway binds to 127.0.0.1 only. Remote access goes through Tailscale (a private WireGuard mesh), which means no open ports and no public internet exposure. Bearer token auth is required for all endpoints except /health.

What are the 6 kernel substrates?

The substrates are the OS primitives. Everything else — jobs, agents, commands, themes — plugs into them as extensions.

1. Extension Substrate

The meta-substrate. Registers, validates, loads, and manages all 12 extension types. Extensions are stored as JSON/YAML in SQLite — not compiled code. The agent can create extensions at runtime through chat.

See Extending Daemion for the full extension model.

2. Context Substrate

How the agent knows things. Inspired by Recursive Language Models — externalizes context rather than bulk-loading everything into the prompt.

Per request, the Context Substrate:

Loads the last 10–15 turns from SQLite (always present, full text)
Runs Engram recall in parallel (semantic + BM25 search over the knowledge graph)
Provides the agent with 5 history tools: search_history, get_thread, list_threads, find_relevant, search_all

Older turns are queryable on demand — the agent retrieves them when it detects it needs them. This keeps prompts lean while preserving access to full history.

3. Execution Substrate

How the agent does work. Manages model selection, tool access, budgets, turn limits, streaming, cancellation, and concurrency.

Request type	Model	Max turns	Budget
Chat (quick)	sonnet	10	$0.50
Chat (complex)	sonnet/opus	25	$5.00
Job execution	configurable	30	$5.00
Build task	sonnet	50	$10.00

All Claude invocations use the Agent SDK — never child_process (known hang bug #771). The SDK provides streaming, tool access, and session management.

4. Presentation Substrate

How output appears in the UI. Renders all content types, streams tokens, shows tool calls as collapsible step indicators, and handles interactive elements (approve/deny buttons, forms).

Content pipeline: Agent output → type detection → renderer selection → display

Custom renderers are extensions of type renderer — a proposal-card renderer, a diff renderer, etc.

5. Trigger Substrate

What causes things to happen. Evaluates conditions and fires the appropriate response.

Type	Fires when
`message`	User sends a turn
`command`	User types `/command`
`cron`	Time-based schedule
`watch`	File changes on disk
`webhook`	HTTP request received
`event`	Internal event fires
`chain`	Another extension completes

6. Storage Substrate

All data persists locally. No cloud database.

Backend	Stores
SQLite	Turns, threads, extensions, config, metrics, costs
Engram (Neo4j)	Knowledge graph — facts, patterns, insights, relationships
Filesystem	Project files, images, attachments, job outputs

What is the extension model?

Everything that isn’t the kernel is an extension. There are 12 types:

Type	What it is
`command`	Input handler (`/`, `@`, `!`, `#`)
`theme`	Visual identity (colors, fonts)
`job`	Autonomous work unit
`renderer`	Custom content display component
`integration`	External service connection (GitHub, Slack, Vercel)
`action`	Per-turn contextual action (copy, edit, regenerate)
`widget`	Dashboard UI component
`app`	Embedded Vite application
`artifact`	Agent-created output (code file, document)
`capability`	Agent skill or behavior
`control`	System configuration (budget limits, model defaults)
`agent`	Persistent agent identity

Extensions are data stored in SQLite — they don’t require compilation or deployment. The primary way to create one is by asking Daemion in chat. Agent-created extensions start disabled and require your approval to activate.

POST /reseed re-syncs built-in extensions from disk without restarting the gateway process.

See Extending Daemion for full schema, lifecycle, and examples.

How does a message flow end to end?

1. You type a message in the PWA

2. Frontend sends POST /chat &#123;"thread_id": "thr_01abc123", "content": "..."&#125;
   via Tailscale (or localhost) to the gateway

3. Gateway receives the request and authenticates the bearer token

4. Context Substrate assembles knowledge:
   a. Load last 10–15 turns from SQLite (always present)
   b. Query Engram for knowledge relevant to this turn (parallel)
   c. Check for extension-provided context (integrations, workspace state)

5. Execution Substrate invokes the agent (Agent SDK):
   a. Select model from request metadata or thread default
   b. Apply budget and turn limits based on detected complexity
   c. Stream tool calls + text tokens back via WebSocket /stream

6. Presentation Substrate formats output:
   a. Tool calls appear as step indicators ("Reading file...")
   b. Text streams token-by-token
   c. Code blocks get syntax highlighting on completion
   d. Final turn stored to SQLite

7. Frontend displays the streamed response

For jobs (autonomous, no user turn):

1. Trigger fires (cron schedule, file watch, event chain)
2. Engine loads the job definition (extension of type "job")
3. Context Substrate assembles job-specific context
4. Execution Substrate invokes the agent with the job prompt
5. Output routed: file write, Engram store, notification push
6. If job has chains → trigger the next job

What events does the WebSocket send?

The WebSocket at WS /stream sends 12 event types. All events are JSON with a type field:

json

{ “type”: “connected”, “threadId”: “thr_01abc123” } { “type”: “start”, “messageId”: “trn_07xyz456”, “model”: “claude-sonnet-4-5” } { “type”: “text-delta”, “messageId”: “trn_07xyz456”, “delta”: “Hello” } { “type”: “tool-start”, “messageId”: “trn_07xyz456”, “tool”: “Read”, “input”: ”…” } { “type”: “tool-end”, “messageId”: “trn_07xyz456”, “tool”: “Read”, “output”: ”…” } { “type”: “finish”, “messageId”: “trn_07xyz456”, “costUsd”: 0.003, “durationMs”: 1240 } { “type”: “error”, “messageId”: “trn_07xyz456”, “error”: “budget exceeded” } { “type”: “stopped”, “messageId”: “trn_07xyz456” } { “type”: “warning”, “text”: ”…” } { “type”: “extension-changed”, “extensionId”: “ext_09def789” } { “type”: “thread-updated”, “threadId”: “thr_01abc123” }

The frontend reconstructs the full turn from streamed events. Tool calls render as collapsible step indicators in the Presentation Substrate.

What are the storage backends?

SQLite is the primary store — turns, threads, extensions, config, and cost metrics all live there. Path defaults to ~/.daemion/daemion.db, overridable via DAEMION_DB_PATH.

Engram is optional. When Neo4j is running and credentials are set, Daemion stores and retrieves knowledge graph data across sessions. If Engram is unreachable, the gateway logs a warning and continues — responses still work, just without cross-session memory.

Filesystem access is provided via GET /filesystem/ls and GET /filesystem/search. Both take a path param that defaults to os.homedir() — the agent can browse your home directory. Scope this appropriately for your threat model.

Common questions

Q Are Daemion agents separate AIs?

No. Agents are Claude operating in a specific role, with a tailored system prompt and permissions. The literal system prompt in src/gateway/agent.ts reads: "You are Claude, operating as a Daemion agent." This is per Anthropic's licensing.

Q Why local-first instead of cloud?

Your turns, threads, extensions, and config live on your machine in SQLite. The Vercel PWA is just static HTML/JS — a window into your local system. There's no external database. Tailscale provides encrypted remote access without any cloud intermediary.

Q What's the difference between the gateway and the daemon?

The gateway is the HTTP/WebSocket API server. The daemon is the gateway plus autonomous capabilities: heartbeat (ambient awareness on wake), cron scheduler, file watcher, and job executor. daemion start runs the gateway only. The background service (launchd/systemd) runs the full daemon.

Q What is the heartbeat?

The heartbeat is a core engine concept — not a job. On each system wake, the daemon reads HEARTBEAT.md, checks ambient state, and replies HEARTBEAT_OK if nothing needs attention, or sends a notification if something does. It's how Daemion maintains ambient awareness without polling.

Q How does context work for long conversations?

The last 10–15 turns are always loaded in full. Older turns are queryable via 5 history tools the agent has access to (search_history, get_thread, etc.). Engram surfaces relevant knowledge via semantic search. There's no compression or summarization step — the agent retrieves what it needs.

What can go wrong

Common architecture questions

Gateway unreachable from phone — Tailscale must be connected on both devices. The gateway binds to 127.0.0.1 only; Tailscale routes correctly when both devices are on the same Tailscale network. Check tailscale status on both.

Engram recall not working — Verify Neo4j is running (brew services list | grep neo4j) and that NEO4J_URI, NEO4J_USER, and NEO4J_PASSWORD are set in the environment where the gateway starts. The service file and plist both need these if running as a background service.

401 {"error": "unauthorized"} — The bearer token is missing or expired. Re-pair the device: run daemion start, scan the QR code, and let the new token replace the old one in localStorage.

Filesystem endpoint returns homedir contents unexpectedly — GET /filesystem/ls defaults to os.homedir() when no path param is provided. Always pass an explicit path if you’re using this endpoint programmatically.

What’s next?

Full Setup Guide — Engram, Tailscale, and background service setup
The Kernel — deep dive into each substrate
Extending Daemion — create jobs, agents, and commands as extensions