# Psyclone AIOS — Complete LLM Reference Guide

**Audience:** This document is a complete, self-contained teaching reference for an LLM assistant that must answer questions about the Psyclone AIOS, author and debug PsySpec XML specifications, and write crank code in C++ or Python. It is **code-truthful**: every capability is marked **SHIPPED/IMPLEMENTED**, **STUB (parsed-only)**, or **ROADMAP** based on verification against the actual source tree (`Psyclone/src/Node.cpp`, `Networking.cpp`, `ServiceManager.cpp`, `CMSDK/src/PsyAPI.cpp`, etc.). Never claim a STUB or ROADMAP feature works.

---

## 1. What Psyclone Is

Psyclone AIOS is a **distributed, real-time, publish/subscribe AI Operating System** from CMLabs. It runs component-based systems — perception pipelines, robot controllers, dialogue systems, simulations, data-integration grids — described in a single XML file (the **PsySpec**) and executed by the Psyclone runtime across one or more computers.

Core facts:

- **Language:** Core in C++ (the `Psyclone` executable + the `CMSDK` library). User processing functions ("**cranks**") can be written in C++ (DLL/SO libraries) or **Python** (SWIG-exposed CMSDK; script file or inline in the spec).
- **Messaging:** All communication is asynchronous pub/sub `DataMessage`s with hierarchical dotted types (`input.audio.raw`) matched exactly or by wildcard. Local message dispatch is sub-100 µs class (the bundled PsyTest reports p50 latencies in the tens of microseconds).
- **Topology:** One system spans multiple **nodes** (computers) and **spaces** (OS processes) transparently; components are placed by attribute, not by code.
- **Configuration:** Everything is declared in the PsySpec XML — components, dataflow, contexts, parameters, network interfaces. Crank code refers only to *names* (trigger/post/query names); the spec binds names to message types, so systems are rerouted by editing XML, never code.
- **Introspection:** **PsyProbe**, a built-in web server, shows live nodes, components, message activity, subscriptions, logs, dataflow graphs, and supports custom views.

### The core mental model

1. A **PsySpec** declares components: **modules** (process messages), **whiteboards** (store messages for retrieval), **catalogs** (queryable stores/conduits), plus network **services/interfaces**.
2. Components never call each other. A `<post>` publishes a typed message; `<trigger>`s subscribe by type (wildcards allowed). Unmatched messages are discarded (unless posted with `ttl`).
3. When a trigger matches (type + active context + filters), the component's **crank** function runs with a `PsyAPI*` handle — its whole world: incoming messages, posting, parameters, retrieves, queries, logging, private data.
4. At startup Psyclone posts `Psyclone.Ready` and prints `SYSTEM READY`; the system runs until something posts `Psyclone.Shutdown`.
5. **Contexts** gate which triggers/cranks/posts are live; one context-change post can reroute the entire system.
6. Watch everything live at `http://localhost:10000/` (PsyProbe on the main port).

---

## 2. Architecture & Key Concepts

### Nodes
Every Psyclone instance runs a local **Node** managing subscriptions, services, interfaces, components and process spaces. Extra computers run as **satellites** (start Psyclone with no `spec=`, optionally `port=N`); the master's PsySpec claims them with `<node name="Node1" addr="host" port="10000"/>`. Components are placed with `node="Node1"`; no attribute = the config node. Nodes can mix Windows and Linux. Per-node crank libraries: `<library>` children of `<node>`. — **IMPLEMENTED** (multi-node networking is working core).

### Spaces
Within a node, each **Space** is a separate OS process. `Root` is implicit (components run as threads in one process). Extra spaces buy crash isolation: if a space crashes, the node restarts it and recreates its components; private data survives. Python-crank modules automatically get their own space. **External spaces** (`<space name="X" type="external"/>`) let a third-party program host components: it links CMSDK, creates a `PsySpace`, connects, and fetches a `PsyAPI` per crank. — **IMPLEMENTED**.

### Components
Six kinds are recognised by tag name (`Node::componentSpecFromXML`):

| Kind | Status | Role |
|---|---|---|
| `<module>` | IMPLEMENTED | Processing: triggers → crank → posts |
| `<whiteboard>` | IMPLEMENTED | Message store with retrieval query language (`root=` disk persistence is parsed-only STUB) |
| `<catalog>` | IMPLEMENTED | Module + synchronous query answering; built-in and custom types |
| `<service>` | PARTIAL | Built-in PsyProbe/Console work; **custom services have a NULL receiver → inbound requests discarded (STUB)** |
| `<stream>` | STUB | Component created; backing function empty |
| `<feed>` | ROADMAP | Log-only branch; never becomes a component |

Every component has at least one crank: modules declare theirs; whiteboards have a built-in one; for catalogs the crank *is* the catalog `type`.

### Messages & PsyTypes
Every message is a **DataMessage**: a header (type, from, to, creation time in synchronised 64-bit microseconds, tags, internal reference) plus named user entries (String/Int/Float/Time/Binary/nested Message, each also in array and map form). Types are dot-notation ASCII, increasingly specific left to right (`input.audio.raw`). Triggers match exactly or with wildcards: `input.audio.*`, `input.*.raw`. Unmatched messages are discarded unless posted with `ttl` (seconds, kept in shared memory).

**Delivery guarantee:** posts are **guaranteed** by default (held until every matching subscriber receives them). `<post ... guaranteed="no">` marks a message **best-effort** (`MESSAGE_NON_GUARANTEED`): it may be dropped rather than queued when a consumer is slow or a buffer is full, and the sender never blocks. Use best-effort for high-rate latest-value-wins streams (video frames, telemetry); keep the default for commands/state a downstream module must not miss. It affects buffering/backpressure, not ordering or acknowledgement. Threading note: one-shot cranks run on a pool thread per trigger, so the same crank code can run concurrently for different messages — guard shared/private state accordingly.

### Triggers & Cranks
A `<trigger>` subscribes a component to messages (by type/from/to/tag, with delay, interval timers, maxage, and `<filter>` children). A matching message dispatches the component's **crank** — an exported `int8 Fn(PsyAPI*)` (C++) or Python code. Cranks are **one-shot** (run once per trigger, thread returns to pool) or **continuous** (loop on `api->shouldContinue()` holding a thread).

### Contexts
Hierarchical dot-notation activation trees (`SoB.Alive.Awake`). Default context is `Psyclone.Ready`, always active. `<context name="X">` blocks inside a module scope triggers/cranks/posts to a context; `<post context="X"/>` switches contexts (deactivating sibling branches, ancestors stay active); `<trigger context="X"/>` fires on the switch (internally a `CTRL_CONTEXT_CHANGE` subscription). Out-of-context cranks see `shouldContinue()==false`, and posting returns `POST_OUTOFCONTEXT (-3)`. — all **IMPLEMENTED** (including simultaneous multi-context roots).

### Signals
The fast lane: no subscription matching or routing logic — every `<signal>` subscriber triggers shortly after a signal is sent. Independent of contexts; typical for simulation time-steps. Only `name` + `type` are parsed on `<signal>` — deliberately minimal. — **IMPLEMENTED (minimal)**.

### Whiteboards
Store messages (by type subscription or direct `to=` addressing), auto-indexed on creation time, optionally on content keys (`key="count" keytype="integer"`). Consumers pull with declared `<retrieve>`s executed via `api->retrieve(...)` / ranged variants. PsyProbe shows stored messages live with filters. `root=` persistence: **STUB** — contents are in-memory; bound with `maxcount`/`maxsize`.

### Catalogs
Modules that additionally answer **synchronous queries**: caller runs `queryCatalog(...)` and blocks (with timeout); catalog receives a `CTRL_QUERY` message and answers with `queryReply(id, status, data|DataMessage)`. Built-ins (PsySystem library): **FileCatalog**, **DataCatalog**, **ReplayCatalog** (record/replay), **RequestStore** / **MessageDataCatalog** (serve live messages over HTTP). Custom catalogs are just crank functions with state + query handling. — **IMPLEMENTED**.

### Distributed operation
- **Nodes** join computers into *one* system (shared types, contexts, synced clocks).
- **Remote queries** connect *separate* systems: `<query source="CCMMaster" host="other" port="10010"/>` — same code, different spec; remote queries carry `INTERSYSTEM_IDENTIFICATION/ADDRESS/PORT/SOURCENAME` entries. — **IMPLEMENTED** (InterSystemManager).
- **Migration/load-balancing:** `migration="yes"` and `priority` are **parsed-only STUB** — runtime migration machinery is not fully acted on.

### PsyProbe
Built-in HTTP introspection server on the main port (default 10000): system communication matrices, contexts, subscriptions (with manual `[test]` fire buttons), logs, dataflow graph; per-component Performance/Activity/Subscriptions/Log/Data tabs; whiteboard Stored Messages; custom tabs via `addPsyProbeCustomView`; HTTP data APIs (`/api/getcomponentdata`, `/api/query`). — **IMPLEMENTED**.

---

## 3. The Complete PsySpec XML Reference (code-verified)

Source of truth: the parser in `Psyclone/src/Node.cpp` (+ `Networking.cpp`, `ServiceManager.cpp`). Statuses: **IMPLEMENTED** — parsed and functionally wired; **PARSED-ONLY (STUB)** — attributes read into structs but never (or only partially) acted on; **ROADMAP** — log-only branch or documentation-only, no real behaviour.

### 3.1 Status summary table

| Element | Status | Notes |
|---|---|---|
| `<psyspec>` | IMPLEMENTED | Root tag; SSL policy attrs (`allowselfsigned`, `cafile`, `capath`) new |
| `<include>` | IMPLEMENTED | XML file inclusion |
| `<variable>` | IMPLEMENTED | `%var%` substitution + **cmdline `key=value` override** |
| `<node>` | IMPLEMENTED | Multi-node topology; `<library>` child per node |
| `<library>` | IMPLEMENTED | DLL/.so crank-library registration |
| `<space>` | IMPLEMENTED | Only `type="external"` handled at top level; regular spaces implicit via `space=` attr |
| `<interface>` | IMPLEMENTED (auth/encryption stubbed) | New SSL verification attrs |
| `<service>` | PARTIAL | Registered + component created, but **custom services have a NULL receiver** |
| `<psyprobe>` | IMPLEMENTED | port/location/alias/post |
| `<module>` | IMPLEMENTED | Core component; per-module verbose/debug/logfile now wired |
| `<whiteboard>` | IMPLEMENTED | `root=` persistence parsed-only |
| `<catalog>` | IMPLEMENTED | File/Data/Replay + CCM/TimeSeries families; Media/Google types are stubs |
| `<stream>` | PARSED-ONLY (STUB) | Component created; backing function empty |
| `<feed>` | ROADMAP | Log-only, never a component |
| `<supervisor>` | ROADMAP | Log-only |
| `<monitoring>` / `<performance>` | ROADMAP | Explicit "not yet implemented" log |
| `<group>` | ROADMAP | Log-only |
| `<trigger>` | IMPLEMENTED | Incl. filters, interval, context triggers |
| `<filter>` | IMPLEMENTED | 7 comparison types + maxage |
| `<triggers>` | PARSED-ONLY (STUB) | Empty parser branch |
| `<triggergroup>` | IMPLEMENTED | Declarative multi-trigger join (all/N-of-M, time windows, snapshot, etc.) — see §3.21 |
| `<retrieve>` | IMPLEMENTED | Whiteboard/catalog fetch specs |
| `<query>` | IMPLEMENTED | Free-form catalog queries, incl. remote host/port |
| `<retrieves>` / `<posts>` / `<signals>` | PARSED-ONLY (STUB) | Empty branches |
| `<crank>` | IMPLEMENTED | function / script / inline / external |
| `<post>` | IMPLEMENTED | Message posting spec |
| `<signal>` | IMPLEMENTED (minimal) | name + type only |
| `<context>` (in module) | IMPLEMENTED | Groups triggers under a named context |
| `<parameter>` | IMPLEMENTED | Typed values incl. ranges/lists/random |
| `<setup>` / `<key>` / `<customview>` | IMPLEMENTED | Component children |
| `<recording>` / `<playback>` (per component) | PARSED-ONLY (STUB) | Parameters stored, not acted on (use ReplayCatalog instead) |
| `<executable>` | PARSED-ONLY (STUB) | cmdline/consoleoutput/autorestart stored, **never launched** (T1.4 in progress) |
| `<alias>` (in psyprobe) | IMPLEMENTED | Loop-index quirk: only the first alias is reliably read |
| `<authentication>` (in interface) | PARSED-ONLY (STUB) | `Authentication::fromXML` body is empty |
| `<phase>` | ROADMAP | No parsing code at all |
| `system=` trigger attribute | ROADMAP | Documented in NewPsySpec only; no handler in code. Use `context=` instead |

### 3.2 `<psyspec>` (root) — IMPLEMENTED

| Attribute | Meaning | Default |
|---|---|---|
| `allowselfsigned` | Global SSL client verification policy. `yes`/`true`/`1` accepts self-signed certs for all SSL client connections. Per-interface value overrides either way | **secure — verify peer** |
| `cafile` | CA PEM file used for SSL client verification instead of the OS trust store | OS trust store |
| `capath` | Directory of CA certs (c_rehash format), same mechanism | OS trust store |

Root-level `verbose`/`debug`/`logfile` attributes (shown in the old manual) are **ROADMAP — not wired**; use the command line for system-wide levels and per-component attributes for targeted logging.

Children: `<include>`, `<variable>`, `<node>`, `<library>`, `<space>`, `<interface>`, `<service>`, `<psyprobe>`, `<supervisor>`, `<monitoring>`, `<performance>`, `<feed>`, `<whiteboard>`, `<stream>`, `<catalog>`, `<module>`, `<group>`.

### 3.3 `<include file="frag.xml"/>` — IMPLEMENTED
Splices the referenced file's contents inside `<psyspec>`, then reparses. Errors reported with line/column. Includes and variables are processed before everything else.

### 3.4 `<variable name=".." value=".."/>` — IMPLEMENTED (+ cmdline override, SHIPPED T1.5)
Defines literal `%name%` text substitution over the whole spec (names, types, paths — the result must stay valid XML). Both attributes required. A `name=value` argument on the Psyclone **command line overrides the spec default** (command line wins). Reserved params (`spec=`, `port=`, `html=`, `verbose=`, `debug=`, `verbose-<topic>=`, `debug-<topic>=`) keep their meanings and are not treated as variables.

### 3.5 `<node>` — IMPLEMENTED

| Attribute | Meaning | Default |
|---|---|---|
| `name` | Node name; `Main` (or no/local addr) reconfigures the local node | — |
| `addr` / `address` | IP/host of the node (`addr` preferred) | local |
| `type` | Node type string (`adhoc`/`java` types NOT implemented) | — |
| `port` | Port; a differing port on the local node adds Main/PsyProbe/Console interfaces there | 0 |
| `timeout` | Connect timeout (ms) | 0 |

Child: `<library name=".." library=".." path=".."/>` — libraries loaded only on that node.

### 3.6 `<library name=".." library=".." path=".."/>` — IMPLEMENTED
`name`: logical name used in `lib::crank` references; `library`: file name (`Examples` → `Examples.dll` / `libExamples.so`; debug builds first try `ExamplesDebug.dll` / `libExamplesDebug.so`); `path`: optional directory; `node` attr restricts to a node. Libraries compile against CMSDK alone.

### 3.7 `<space name=".." type="external"/>` — IMPLEMENTED (external only at top level)
At top level only `type="external"` does anything (creates a ProcessMap entry waiting for an external process to connect). Regular spaces are created implicitly per component via `space="Name"` on the component (default `Root`). `node` attr restricts to a node.

### 3.8 `<interface>` — IMPLEMENTED (SSL verify attrs new; `<authentication>` STUB)

```xml
<interface name="MyWebInterface" node="Node0" port="1234" protocol="HTTP"
           service="MyWebService" timeout="3000" default="yes" />
```

| Attribute | Meaning | Default |
|---|---|---|
| `name` | **Required** (parse fails without) | — |
| `service` | Target service name (**required**) | — |
| `node` | Node the interface lives on | any/local |
| `port` | TCP/UDP port (**required**) | — |
| `timeout` | Timeout ms (protocol autodetect window) | unset |
| `default` | `yes` = default interface on a shared port | no |
| `protocol` | `Message` \| `HTTP` \| `Telnet` \| `RAW` (anything else = parse failure) | unset |
| `ip` | `UDP` selects UDP; otherwise TCP | TCP |
| `allowselfsigned` | Per-interface SSL verification override; wins over global either way | inherit global |
| `cafile` / `capath` | Per-interface CA file/dir override | inherit global |

Child `<authentication>`: **PARSED-ONLY STUB** (parser body is empty, returns true). Interface encryption attrs (SSL/AES256 from NewPsySpec docs) likewise stubbed. Multiple interfaces can share a port; the protocol is sniffed from the first bytes; after the timeout the `default="yes"` interface takes the connection.

### 3.9 `<service>` — PARTIAL
Registered (`name`, `rootdir`, `subdir`, `timeout`) and a component is created for its subscription, so `<trigger>`/`<post>`/`<crank>` children parse. **Caveat: custom web/telnet/message services register a NULL receiver — inbound requests are discarded (STUB).** Built-in PsyProbe and Console services work normally. Disable a built-in: `<interface protocol="HTTP" service="none"/>`.

### 3.10 `<psyprobe>` — IMPLEMENTED

| Attribute | Meaning | Default |
|---|---|---|
| `defaultport` | any value other than `no`… in practice: `no` removes PsyProbe from the main port | shares main port |
| `port` | Adds a dedicated PsyProbe HTTP interface | — |
| `location` | Web root directory (overrides autodetect and `html=`) | built-in autodetect |

Children:
- `<alias name=".." location=".." default=".."/>` — URL alias mapping a prefix to a local directory with optional default file. **Known quirk:** with multiple aliases the parse loop reuses the wrong index — only the first alias is reliably read; prefer one alias per spec.
- `<post>` — a clickable manual-test post in the PsyProbe UI (same attributes as a module `<post>`, including XML content).

### 3.11 Component common attributes (module / whiteboard / catalog / stream / service / feed)

All parsed by `Node::componentSpecFromXML`; tag name selects the component type. (`<feed>` never reaches this path from configNode — its parsing here is dead code.)

| Attribute | Meaning | Default | Status |
|---|---|---|---|
| `name` | Component name (**required**; parse returns NULL without) | — | IMPLEMENTED |
| `tag` | Appends `_tag` to the name; `*` uses the component's own name as tag | none | IMPLEMENTED |
| `node` | Node assignment; modules also accept `*` = every node | config node | IMPLEMENTED |
| `space` | Process space; Python-crank modules auto-get their own space | Root | IMPLEMENTED |
| `selftrigger` | `allow` \| `warn` \| anything else = never trigger on own posts | warn | IMPLEMENTED |
| `logfile` | Per-module log file (wired into central logging) | node log | **IMPLEMENTED (T1.5)** |
| `verbose` / `debug` | Per-module verbosity/debug levels (0–9) | node levels | **IMPLEMENTED (T1.5)** |
| `migration` | `yes` = migration-allowed flag; runtime migration not acted on | no | PARSED-ONLY STUB |
| `priority` | uint8 stored in setup; scheduling effect not acted on | 0 | PARSED-ONLY STUB |

`<module count=N>` (tag-instancing) is **NOT implemented**. `<module type="external">` marks an external-process component (connects via a space; may carry an `<executable>` child — a STUB, see below).

Extra attributes for whiteboard/stream/catalog/service/feed:

| Attribute | Meaning | Default |
|---|---|---|
| `function` / `type` | Backing crank `Lib::Func`; bare names get `PsySystem::` prefix. Defaults: whiteboard→`PsySystem::Whiteboard`, stream→`PsySystem::Stream`; catalog/service/feed with neither = error | see left |
| `root` | Storage root dir (whiteboard `root=` persistence is **parsed-only STUB**) | — |
| `subdir` / `ext` | Subdirectory / file extension | — |
| `operation` | Default operation | — |
| `key` + `keytype` | Adds an index key (repeatable via `<key>` children) | time index |
| `timeout` / `interval` | ms | 0 |
| `maxcount` / `maxsize` | Max entries / max bytes | 0 |
| `rotate` | `yes`/`no` rotation (ReplayCatalog playback loop) | no |

### 3.12 Component children

| Child | Status | Meaning |
|---|---|---|
| `<recording root filesize overwrite>` | PARSED-ONLY STUB | Parameters stored; not acted on. Use a ReplayCatalog |
| `<playback root interval>` | PARSED-ONLY STUB | Same |
| `<executable consoleoutput autorestart>cmdline</executable>` | PARSED-ONLY STUB | Command line stored, **never launched** (auto-start is T1.4 ROADMAP, in progress). Start external processes yourself |
| `<customview name template>` (or inline content) | IMPLEMENTED | Registers a PsyProbe custom tab |
| `<setup file="..">` or inline XML/text | IMPLEMENTED | Free-form config blob; read back via `getParameterString("componentsetup")` |
| `<key name type>` | IMPLEMENTED | Extra index keys |
| `<parameter name type value interval>` | IMPLEMENTED | Typed parameters (rich value syntax, §3.13) |
| `<context name="..">` | IMPLEMENTED | Groups child trigger/post/retrieve/query/crank/signal under a named context (default: `Psyclone.Ready`) |
| any other child | — | Parsed as trigger-family elements in the default context |

### 3.13 `<parameter>` — IMPLEMENTED

`type` = `String` (default) | `Integer` | `Float`/`Double`. Value syntax:

| Value syntax | Meaning |
|---|---|
| `42` | Plain value |
| `[0:110]` | Range; initial value = lowest |
| `[0:110]=55` | Range with explicit initial value |
| `[0:110]=*` | Range with random initial value |
| `[0:110:2]` | Stepped range (step 2); step on a plain numeric via `interval="2" value="10"` |
| `[-2.5;1.5;1.8]` / `['a';'b';'c']` | Value lists (numeric / string) |

From code: `getParameterString` / `getParameterInt` (+ float getter), `hasParameter`, `setParameter`, `resetParameter`, `tweakParameter(name, ±steps)`, `getParameterDataType` → `PARAM_STRING` 1, `PARAM_INTEGER` 2, `PARAM_FLOAT` 3, `PARAM_STRING_COLL` 4, `PARAM_INTEGER_COLL` 5, `PARAM_FLOAT_COLL` 6. All parameters are queryable and tweakable live from PsyProbe.

### 3.14 `<trigger>` — IMPLEMENTED

| Attribute | Meaning | Default |
|---|---|---|
| `name` | **Required; must be unique** (duplicate = error) | — |
| `type` | PsyType to match (exact or wildcard) | — |
| `from` / `to` | Only messages from / addressed to this component | any |
| `interval` | ms — timer trigger. **The timer only starts after the module is first triggered by a real message** (kick-start with `Psyclone.Ready`) | 0 |
| `delay` / `after` | ms delay before firing (`after` is an alias, read only when `delay` absent) | 0 |
| `context` | Fire on context change to this context (type forced to `CTRL_CONTEXT_CHANGE`) | — |
| `maxage` | ms — becomes a MAXAGE filter (skip stale messages) | ∞ |
| `tag` | Match tag; `*` = component's own name | any |

Validity: `name` plus at least one of valid `type`/`from`/`to`. Trigger `priority`, `history`, and `system=` attributes are **NOT implemented** (no handlers in code).

### 3.15 `<filter>` (child of trigger) — IMPLEMENTED

`<filter type=".." key=".." value=".."/>`:

| Filter type | Comparison |
|---|---|
| `haskey` | The entry exists (key only, no value; matches any entry type — string, number or binary) |
| `equals` | String equality — case-sensitive, **exact strcmp, no wildcards** |
| `notequals` | String inequality |
| `equalsnumeric` | Float equality, converting from string/int as needed (float64, epsilon compare) |
| `notequalsnumeric` | Numeric inequality |
| `greaterthan` / `lessthan` | Numeric > / < (values equal within epsilon also pass, i.e. effectively >= / <=) |
| `maxage` (trigger *attribute*) | Message younger than `maxage` milliseconds; the eighth filter op, stored internally as a FilterSpec |

Worked example covering all 8 ops (every filter must pass; `maxage` is a trigger attribute, not a `<filter>` child):

```xml
<trigger name="input" type="msg.1" maxage="5000">
  <filter type="haskey" key="SomeKey"/>
  <filter type="equals" key="camera" value="front-left"/>
  <filter type="notequals" key="state" value="disabled"/>
  <filter type="equalsnumeric" key="mode" value="3"/>
  <filter type="notequalsnumeric" key="channel" value="0"/>
  <filter type="greaterthan" key="temp" value="80"/>
  <filter type="lessthan" key="pressure" value="2.5"/>
</trigger>
```

**Warning:** unrecognised filter types are **silently ignored** — a typo in `type=` means the filter never rejects anything.

### 3.16 `<retrieve>` — IMPLEMENTED
On-demand fetch from a whiteboard/catalog, executed from crank code by name.

| Attribute | Meaning | Default |
|---|---|---|
| `name` | **Required, unique** | — |
| `type` | PsyType filter | any |
| `from` / `to` | Original sender/addressee filter | any |
| `source` | Component to retrieve from (**required**) | — |
| `maxcount` | Max messages | 0 = all |
| `maxage` | Age window, stored ×1000 (exact wall-clock unit under review — API call takes µs) | ∞ |
| `tag` | Tag filter; `*` = own name | any |
| `key` / `keytype` | Index key; keytype `string` \| `integer` \| `float`/`double` \| `time` (invalid keytype drops the spec) | time |
| `start` / `end` | Range bounds, interpreted per keytype | open |

### 3.17 `<query>` — IMPLEMENTED
Free-form query to a component (catalogs dispatch on Query/Operation fields). Attributes: `name` (**required, unique**), `type` (a plain string, not a PsyType), `source` (**required**), `subdir`, `ext`, `binary` (`yes`), `maxcount`, `maxage` (stored ×1000; unit under review), `key`, `value`, `operation`, and for **remote (inter-system) queries**: `host` (DNS-resolved at parse time) and `port`.

### 3.18 `<crank>` — IMPLEMENTED
Forms:

```xml
<crank name="c1" function="lib::function"/>            <!-- C/C++ library crank; no lib:: = built-in Internal:: crank -->
<crank name="c2" language="python3" script="file.py"/> <!-- external script file -->
<crank name="c3" language="python3">inline code</crank><!-- inline script (auto-unindented) -->
<crank name="c4"/>                                     <!-- external crank slot; a process connects and fills it -->
```

Duplicate crank names across components are rejected. A module with no crank and no function gets the built-in `Internal::Simple` default (passthrough) crank. Java cranks are **NOT implemented**. `language="python2"` is accepted but legacy (Python 2 is EOL) — use `python3`.

### 3.19 `<post>` — IMPLEMENTED

| Attribute | Meaning | Default |
|---|---|---|
| `name` | **Required, unique** | — |
| `type` | PsyType of the posted message | — |
| `context` | Post a context *change* to this context | — |
| `to` | Direct copy to a named component regardless of subscriptions (also still delivered to normal subscribers) | broadcast |
| `ttl` | Seconds (stored ×1000 ms) — keeps unmatched messages available | 0 = no expiry |
| `guaranteed` | `no` = best-effort (MESSAGE_NON_GUARANTEED) | guaranteed |
| `tag` | `*` = own name | none |

Message content comes from child XML entries such as `<content key=".." type=".." value=".."/>`. Valid if `name` and (`type` or `context`). Post `time="trigger"` is **NOT implemented**.

### 3.20 `<signal name=".." type=".."/>` — IMPLEMENTED (minimal)
Both attributes required; nothing else parsed.

### 3.21 `<triggergroup>` declarative join — IMPLEMENTED
Joins several triggers into ONE crank activation. Member messages sharing the same **tag** are buffered node-side (join engine) until the group completes; untagged messages group under the literal empty-string tag `""` and never mix with real tags. One activation per completed set (wakeups ≈ joins).

```xml
<triggergroup name="ts" count="2" maxage="2000">
  <trigger name="t1" type="bla1"/>
  <trigger name="t2" type="bla2"/>
  <trigger name="t3" type="bla3" optional="true" maxage="200"/>
</triggergroup>
```

Attributes: `name` (**required**, unique per module; duplicate name → group discarded + level-1 warn; missing name/maxage → warn+skip), `maxage` ms (**required**, no default; partial set evicted when older), `count` (default `all`; `count=N` = any N-of-M, later arrivals discarded), `min=N` (fire at N then **re-fire** as the set grows), `within=` ms (post-time window, `getSendTime`), `time=` ms (content-time window, `getCreatedTime`; `time="0"` = identical content time), `mode="snapshot"` (fire on the primary trigger, attach latest of each other member — no wait, no tag match), `slide` (with `within=`: re-emit fresh joined set as members update while the window holds), `debounce=` ms (fire at most once per N ms with latest set), `dup="latest"` (default, latest-wins) / `"keep"` (queue all), `order="strict"` (fire only on declared arrival order), `timeout="<ms>"` + `ontimeout="fire|discard"` (expiry with incomplete set: default `fire` = partial-fire + level-1 warn).

Child `<trigger>` additions: `optional="true"` (not required; in the map if present), per-member `maxage` (stale member doesn't count toward completion), `primary="true"` (snapshot primary; defaults to first child).

SDK: `PsyAPI::waitForNewMessageGroup(uint32 ms, std::string& groupName)` returns `std::map<std::string, DataMessage*>` keyed by **trigger name** (absent/optional members not in the map); `getCurrentTriggerName()` returns the **group** name; delivered as `CTRL_TRIGGER_GROUP` (type id 10307). Messages owned by the API, freed on next `waitForNewMessageGroup()`/`waitForNewMessage()`. Single-message cranks untouched.

### 3.21b Stub grouping elements
`<triggers>`, `<retrieves>`, `<posts>`, `<signals>` hit parser branches whose entire body is a comment — recognised, **silently do nothing** (PARSED-ONLY STUB).

### 3.22 ROADMAP-only elements
`<feed>`, `<supervisor>`, `<group>` — log-only branches. `<monitoring>`/`<performance>` — explicit "not yet implemented" log. `<phase>` — **no parsing code at all**. `system=` trigger attribute — documentation-only; the implemented equivalent for context events is `<trigger context="X"/>`; there is **no** implemented mechanism for the generic `system="key:value"` form.

### 3.23 Cross-cutting parsing rules
- Elements without `node=` run on the config node; with `node=` only on that node.
- Variables and includes are expanded before anything else; the spec is reparsed after each expansion.
- Three built-in Psy types/contexts registered at config: `PsyControl`, `Psyclone.Ready` (type + default context), `Psyclone.Shutdown`.

---

## 4. Writing Cranks (C++ and Python)

A crank is the processing function of a component. In C++ it is an exported C function taking a single `PsyAPI*` — the module's whole world. In Python the full CMSDK is exposed via SWIG (`cmsdk` module), so `PsyAPI` and `DataMessage` work as in C++.

### 4.1 The C++ crank pattern (continuous)

```cpp
// header
namespace cmlabs { extern "C" {
    DllExport int8 MyCrankFunction(PsyAPI* api);
} }

// implementation
int8 MyCrankFunction(PsyAPI* api) {
    while (api->shouldContinue()) {
        DataMessage* msg = api->waitForNewMessage(100, "Input");  // 100 ms timeout, trigger name
        if (!msg) continue;                                       // timeout: do periodic work here
        api->logPrint(1, "Got a message...");
        DataMessage* outMsg = new DataMessage();
        outMsg->setInt("Count", 1);
        outMsg->setString("Status", "ok");
        api->postOutputMessage("Output", outMsg);                 // post name, spec binds the type
    }
    return 0;
}
```

The `DllExport` classifier is **required** for visibility to Psyclone. CMSDK objects live in namespace **`cmlabs`** (the old PDF's claim of a "cmsdk" C++ namespace is wrong; `cmsdk` is the *Python* module name).

### 4.2 The Python crank equivalents

**Script file** (`<crank name="c" language="python3" script="process.py"/>`) — entry point is `PsyCrank(apilink)`; the `cmsdk` import is automatic:

```python
def PsyCrank(apilink):
    api = cmsdk.PsyAPI.fromPython(apilink)
    while api.shouldContinue():
        msg = api.waitForNewMessage(100, "Input")
        if msg is None: continue
        api.logPrint(1, "Got a message...")
        outMsg = cmsdk.DataMessage()
        outMsg.setInt("Count", 1)
        outMsg.setString("Status", "ok")
        api.postOutputMessage("Output", outMsg)
    return 0
```

**Inline code** — an `api` PsyAPI object is auto-created and libraries auto-loaded; keep code cleanly unindented:

```xml
<crank name="p1" language="python3"><![CDATA[
while api.shouldContinue():
    msg = api.waitForNewMessage(20)
    if msg is None: continue
    count = msg.getInt("Count")
    outMsg = cmsdk.DataMessage()
    outMsg.setInt("Count", count + 1)
    api.postOutputMessage("Output", outMsg)
]]></crank>
```

Python runtime requirements: matching interpreter (major version + bitness), the CMSDK binary (`_cmsdk.pyd` / `_cmsdk.so`) plus interface file (`cmsdk3.py`; debug variants `_cmsdkdebug.*`, `cmsdk3debug.py`), searched in (a) the script's directory, (b) the Psyclone binary's directory, (c) a module parameter `<parameter name="libpath" type="String" value="../CMSDK/bin/..."/>`. Python-crank modules automatically get their own process space.

### 4.3 C++ ↔ Python API mapping cheatsheet

| Concept | C++ | Python |
|---|---|---|
| API handle | `int8 Fn(PsyAPI* api)` (exported, DllExport) | inline: `api` auto-provided; file: `def PsyCrank(apilink): api = cmsdk.PsyAPI.fromPython(apilink)` |
| Loop condition | `api->shouldContinue()` | `api.shouldContinue()` |
| Wait for message | `api->waitForNewMessage(100, "Trigger")` → `DataMessage*` or NULL | `api.waitForNewMessage(100, "Trigger")` → msg or `None` |
| New message | `new DataMessage()` | `cmsdk.DataMessage()` |
| Entries | `msg->setInt/setString/setFloat/setData/setMessage(...)`, `msg->getInt("k")` etc. | `msg.setInt/...` identical, no buffer sizes needed |
| Post | `api->postOutputMessage("Out", msg)` → int (negative = POST_* error) | `api.postOutputMessage("Out", msg)` |
| Log | `api->logPrint(level, "text %d", x)` | `api.logPrint(level, "text %d" % x)` |
| Retrieve | `std::list<DataMessage*> msgs; uint8 st = api->retrieve(msgs, "r1");` | `msgs = api.retrieve("r1")` (returns a list) |
| Ranged retrieve | `api->retrieveIntegerParam(msgs, "r2", start, end, maxcount, maxageUS, timeoutMS)` — also Float/String/Time variants | `msgs = api.retrieveIntegerParam("r2", start, end, maxcount, maxageUS, timeoutMS)` |
| Query (string) | `api->queryCatalog(&result, size, "Name", "query", "operation", data, datasize, timeout)` | `result = api.queryCatalog("Name", "query", operation, data, timeout)` |
| Query (message) | `api->queryCatalog(&resultMsg, "Name", msg, timeout)` | `resultMsg = api.queryCatalog("Name", msg, timeout)` |
| Answer a query (catalog side) | `api->queryReply(id, PsyAPI::QUERY_SUCCESS, replyMsg)` | `api.queryReply(id, cmsdk.PsyAPI.QUERY_SUCCESS, replyMsg)` |
| Private data | `api->setPrivateData("Name", buf, len[, "mimetype"])`; `api->getPrivateDataCopy("Name", size)` | `api.setPrivateData("Name", s[, "mimetype"])`; `api.getPrivateDataCopy("Name")` |
| Parameters | `api->getParameterString("componentsetup")`, `getParameterInt`, `tweakParameter` | same method names on `api` |
| PsyProbe custom tab | `api->addPsyProbeCustomView("Title", "elements/x.html")` | `api.addPsyProbeCustomView(...)` |
| Runtime subscriptions | `api->addSubscription(xmlFragment)` | `api.addSubscription(...)` |
| Context introspection (external modules) | `getCurrentTriggerContext()`, `contextToText(ctx)` | same |
| External space | `PsySpace* s = new PsySpace("X"); s->connect(sysid); s->start(); PsyAPI* api = s->getCrankAPI("Crank");` | `s = cmsdk.PsySpace("X"); s.connect(sysid); s.start(); api = s.getCrankAPI("Crank")` |

### 4.4 One-shot vs continuous
- **One-shot:** the crank runs once per trigger and returns; its thread goes back to the pool. Ideal for stateless processing; re-invoked per message.
- **Continuous:** `while (api->shouldContinue())` with a `waitForNewMessage` timeout; the crank keeps its OS thread — good for heavy init or big in-memory state; many continuous cranks = many threads.
- Treat `shouldContinue() == false` as normal (shutdown or context switch): save state and exit cleanly.

### 4.5 Private (persistent) data
Any component can persist binary blobs (state for one-shot modules, survives space restarts and migration). With a mimetype the data also appears in PsyProbe's Data tab and over HTTP:

```cpp
api->setPrivateData("My Report", html.c_str(), html.length(), "text/html");
```

### 4.6 Custom catalogs = cranks with query handling

```cpp
while (api->shouldContinue()) {
    // no trigger-name filter: handle whatever this catalog is subscribed to
    DataMessage* inMsg = api->waitForNewMessage(100);
    if (!inMsg) continue;
    if (inMsg->getType() == PsyAPI::CTRL_QUERY) {
        uint32 id = (uint32) inMsg->getReference();
        // string-query params arrive as string entries: Subdir, Ext, Binary, Operation
        DataMessage* reply = new DataMessage();
        reply->setString("Answer", "42");
        api->queryReply(id, PsyAPI::QUERY_SUCCESS, reply);
    } else {
        // ordinary trigger message: ingest, post, etc.
    }
}
```

```python
while api.shouldContinue():
    inMsg = api.waitForNewMessage(100)
    if inMsg is None: continue
    if inMsg.getType() == cmsdk.PsyAPI.CTRL_QUERY:
        qid = inMsg.getReference()
        reply = cmsdk.DataMessage()
        reply.setString("Answer", "42")
        api.queryReply(qid, cmsdk.PsyAPI.QUERY_SUCCESS, reply)
```

Three `queryReply` forms: status only; status + raw buffer (data, size, count); status + DataMessage. Declare with `<catalog name="X" type="MyLib::MyCatalog">`; web queries from PsyProbe carry an `HTTP_OPERATION` entry and expect a `JSON` string entry in the reply.

### 4.7 DataMessage entries, arrays, maps

```cpp
msg->setString("Utterance", "hello");
msg->setInt("Count", 42);
msg->setFloat("Confidence", 0.93);
msg->setData("Frame", pixelBuffer, bufferSize);   // binary blob
DataMessage* inner = new DataMessage(); inner->setInt("x", 10);
msg->setMessage("Position", inner);               // nested message

msg->setInt(1, "myarray", 2);                     // array form: myarray[1] = 2
std::map<int64,int64> arr = msg->getIntArray("myarray");
msg->setInt("in", "mymap", 1);                    // map form: mymap["in"] = 1
std::map<std::string,int64> m = msg->getIntMap("mymap");
```

Array and map forms exist for all entry types (`getFloatArray`, `setStringMap`, …). In Python the same calls take/return dicts and need no buffer sizes.

---

## 5. System Messages (code-catalogued)

**Important: Psyclone has no `SYS_*` message constants in shipped code** — a grep across Psyclone and CMSDK finds none. All shipped system messaging is the **`CTRL_*`** family below. `SYS_BUILDER_*` messages exist only in the roadmap design (§11). System control messages are ordinary `DataMessage`s in reserved type ranges: 10001–10004 (low-level), 10200–10306 + 10297–10299 (PsyAPI control), 10998/10999 (intersystem), plus the internal `CTRL_NODE_*` family.

Status legend: ✅ implemented (sender + consumer in code) · ⚠️ partial · ❌ declared only.

| Message | ID | Purpose | Status |
|---|---|---|---|
| `CTRL_TEST` | 10001 | Test type | ✅ (tests) |
| `CTRL_LOGPRINT` | 10002 | Forwards a log line from an external process/space to node logging | ✅ |
| `CTRL_PING` / `CTRL_PING_REPLY` | 10003/10004 | Liveness ping between nodes/consoles | ✅ |
| `CTRL_SYSTEM_READY` | 10200 | "System is up" broadcast; the type behind every component's implicit `DefaultComponentTrigger` and the `Psyclone.Ready` default context — this is why components wake on start | ✅ |
| `CTRL_PROCESS_INITIALISE` | 10201 | Initialise an external process/space | ✅ |
| `CTRL_PROCESS_GREETING` | 10202 | Process announces itself after connect | ✅ |
| `CTRL_PROCESS_SHUTDOWN` | 10203 | Ask a process/space to shut down | ✅ |
| `CTRL_CONTEXT_CHANGE` | 10297 | Context-switch notification; `<trigger context="X">` compiles to this type; PsyProbe can inject one from the UI | ✅ |
| `CTRL_SYSTEM_SHUTDOWN` | 10298 | Whole-system shutdown, sent to all nodes and processes | ✅ |
| `CTRL_SYSTEM_SHUTTINGDOWN` | 10299 | "Shutdown in progress" — declared and type-registered, but no live sender found; effectively reserved | ⚠️ |
| `CTRL_TRIGGER` | 10300 | Wraps a fired trigger delivered to a crank/space | ✅ |
| `CTRL_QUERY` / `CTRL_QUERY_REPLY` | 10301/10302 | On-demand query round-trip (`<query>` / API query) — what catalog cranks match on | ✅ |
| `CTRL_PULLCOMPONENTDATA` | 10303 | Space pulls component data/parameters from the node | ✅ |
| `CTRL_CREATECUSTOMPAGE` | 10304 | Ask PsyProbe to create a custom page | ✅ |
| `CTRL_ADDSUBSCRIPTION` | 10305 | Runtime subscription add from an API client (XML same as module spec) — used by e.g. TaskManagerCatalog's dynamic registration | ✅ |
| `CTRL_RETRIEVESYSTEMIDS` | 10306 | Client retrieves type/context/component ID maps | ✅ |
| `CTRL_INTERSYSTEM_QUERY` / `_REPLY` | 10998/10999 | Query between separate Psyclone systems (InterSystemManager) | ✅ |

**Node-to-node sync family (`CTRL_NODE_*`)** — internal multi-node protocol: `WELCOME`, `JOIN`, `TIMESYNC(_REPLY)`, `NODEMAP(_REQUEST)`, `STATUSMAP`, `PERFSYNC`, `SYNC_REQ/REPLY`, `CONFIG(±)`, `SYNC_ID / CONFIRM_ID / CANCEL_ID (±)`, `CONTEXT_ACTIVE`, `SIGNAL`, `QUERY(±)`, `SYNC_COMPONENT(±)`, `SYNC_SUBSCRIPTION(±)`, `CTRL_PSYPROBE_CREATECUSTOMPAGE(±)`. All ✅ implemented as a family (multi-node networking is working core); `CTRL_NODE_CONTEXT_ACTIVE` propagates context activation across nodes.

**Builder system messages (ROADMAP — do not use):** `SYS_BUILDER_CREATE`, `SYS_BUILDER_LOAD`/`SYS_BUILDER_EXECUTE`, `SYS_BUILDER_DELETE`, `SYS_BUILDER_MOVE`, `SYS_BUILDER_REGISTER_UPDATE`, results as `SYS_BUILDER_RESULT`. These belong to the planned node-local builder (§11) and do not exist in shipped code.

Context-related summary: `<trigger context="X">` ✅; `<post context="X">` ✅; `<context name="X">` blocks ✅; PsyProbe context injection ✅; `system="context..."` attribute syntax ❌ ROADMAP.

---

## 6. Running & Configuring Psyclone

### 6.1 Command line

```bash
./Psyclone spec=pingpong.xml                     # run a spec
./Psyclone port=11000                            # satellite node (no spec), custom port
./Psyclone spec=robot.xml port=9000 html=/opt/psyclone/html verbose=2 debug=0
```

| Parameter | Meaning | Default |
|---|---|---|
| `spec=<file>` | PsySpec XML to load | none (satellite mode) |
| `port=<n>` | Main network port (inter-system comms + PsyProbe URL) | 10000 |
| `html=<dir>` | PsyProbe HTML directory | autodetected `./html` |
| `verbose=<0-9>` / `debug=<0-9>` | System-wide log levels | 1 / 1 |
| `verbose-<topic>=` / `debug-<topic>=` | Per-topic levels (lower-case topic) | inherit |
| **any `key=value`** | **SHIPPED (T1.5):** overrides the PsySpec `<variable>` of that name; command line wins over the spec default | — |

```bash
# Same spec, production values — cmdline overrides <variable> defaults
./Psyclone spec=robot.xml port=9000 nodename=rig RecordingDir=/data/rec
```

Log topics: System, Network, Memory, Process, Sync, Node, Space, Component, Subscriptions, Triggers, Timing — e.g. `verbose-subscriptions=7`.

### 6.2 Per-module logging — SHIPPED (T1.5)

```xml
<module name="FaceFinder" verbose="7" debug="5" logfile="./log/facefinder.log">
```

Per-component `verbose`/`debug`/`logfile` now take real effect (historically parsed-only) on all component kinds. Root-level `<psySpec verbose=..>` is still **ROADMAP** — use the command line for system-wide levels.

### 6.3 Satellite mode & distributed startup
Start Psyclone with no `spec=` (optionally `port=N`) on each extra computer; the master spec claims them with `<node>` entries and places components with `node=`.

### 6.4 Building from source
Five projects: **CMSDK** (core library), **Psyclone** (main executable), **System** (whiteboards/catalogs/built-ins DLL/SO), **Examples**, **External Modules**. Linux: `make` / `make debug` / `make ssl` / `make ssldebug`; Python bindings: `make python3`. Windows: the Visual Studio solution with Release/Debug (+ SSL) configurations. The reference toolchain versions in older docs (VS2015, OpenSSL 1.0.2g, Python 2.7/3.5, GCC 5) are dated — use current compilers, OpenSSL, and Python 3; Python 2 is EOL/legacy.

### 6.5 Testing

```bash
./Psyclone test=cmsdk            # full unit-test suite (aliases: test=all, test=sdk)
./Psyclone test=list             # list all registered tests
./Psyclone test=network_sslverify fork=0 verbose=1   # single test, in-process (debugger-friendly)
./Psyclone spec=Examples/psytest.xml                 # PsyTest: whole-system function+performance test (~35 s)
```

Options: `fork=0` (in-process), `verbose=1`, `out=`/`outdir=`, `compare=<old.json>` (perf regression deltas). Notable tests: `psy_configvars` (cmdline variable override), `psy_modulelogging` (per-module logging), `network_sslverify` / `network_sslhostca` (SSL verification — require the SSL build). PsyTest stages: Intro → TimeTest → PingTest → PingTestUDP → Signals → Contexts → Catalogs → Done, reporting throughput and tail latency (`p50/p90/p95/p99/p99.9` in µs); tune via cmdline variable overrides (`PingCount=`, `PayloadSize=` …).

---

## 7. Security (SSL) — SHIPPED (T1.3)

Current behaviour (this **obsoletes** older docs that implied "SSL just works with self-signed certs"):

- SSL client connections **verify the peer certificate by default** (`SSL_VERIFY_PEER` against the OS trust store; previously encrypted-but-unverified).
- **Hostname verification** is enforced (`SSL_set1_host`) — issue certs with correct hostnames/SANs.
- **`allowselfsigned` — default FALSE** — is the only escape hatch. Scopes, most-specific-wins precedence **connection > manager > interface > global**:

| Scope | Where |
|---|---|
| Global | `<psySpec allowselfsigned="yes">` |
| Per interface | `<interface ... allowselfsigned="no">` (overrides global *either way*) |
| Per manager | connection-manager level (API) |
| Per connection | individual connection (API) |

Accepted true values: `yes`/`true`/`1`. `allowselfsigned="yes"` = encryption **without authentication** (MITM-able) — dev/loopback only.

- **Custom CA:** `cafile` (PEM bundle) / `capath` (hashed dir) on `<psySpec>` or per `<interface>` → `SSL_CTX_load_verify_locations`. Full chain + hostname verification still apply — real security with self-minted certs, no purchase needed.

```xml
<psySpec cafile="/etc/psyclone/ca/rootCA.pem">
  <interface name="Public" port="8443" protocol="HTTP"
             service="MyWebService" allowselfsigned="no" />
</psySpec>
```

Hardening: run the SSL build (`make ssl` / "Release SSL") across untrusted networks; leave `allowselfsigned` unset; disable unused Telnet Console / PsyProbe interfaces on exposed ports; firewall the main port. Remember interface `<authentication>` and encryption attributes are **parsed-only STUBs** — not security controls.

---

## 8. PsyProbe (web introspection) — IMPLEMENTED

Open `http://localhost:10000/` (main port) or a dedicated `<psyprobe port=..>` interface.

### 8.1 Standard views

| View / tab | Shows |
|---|---|
| System Component / Node Communication | Traffic amount/speed matrices |
| System Contexts | Active/inactive contexts, expandable to triggers/posts |
| System Subscriptions | All subscriptions by message type; every trigger/post has a **[test]** button that fires it with an empty message |
| System Log / Dataflow | Aggregated `logPrint` output with free-text filter; graphical component/flow map with filters |
| Node Performance / Info | I/O, queues, CPU, memory over last 1/10/30 s; spaces per node |
| Component Performance / Activity / Subscriptions / Log / Data | Per-component stats; **last 10 in + 10 out messages live** (hover = full content, click = pin); private data published with a mimetype |
| Whiteboard Stored Messages | Live store with text/count/size/age filters |

### 8.2 Configuration (`<psyprobe>`)

```xml
<psyprobe location="otherdir/html" port="8080" defaultport="no">
  <alias name="robot" location="/somedir/html" default="index.html" />
  <post name="TestMove" type="Robot.Command.Move.Forward" />
</psyprobe>
```

`location` overrides the autodetected web root and `html=`; `port` adds a dedicated HTTP interface; `defaultport="no"` removes PsyProbe from the main port. `<alias>` maps `http://host:port/robot/...` to any local directory (whole custom operator UIs ship this way) — but note the multiple-alias parsing quirk: only the first alias is reliably read. `<post>` children are clickable manual-test posts in the UI.

### 8.3 The four extension paths (increasing effort)
1. **Custom subsite / static page** — files in the web root or behind an `<alias>`; fetch data with AJAX.
2. **RequestStore Catalog** — declare `<store>` entries in the spec; pages poll `GET /api/query?from=RequestStore&query=<storename>`. No native code. Store attrs: `name` (URL name, required), `trigger` (required), `key` (a user entry, or header fields `time` (µs since year 0), `timetext`, `type`, `size`, `content`; omitted ⇒ whole message rendered per mimetype), `keytype` (reserved — STUB), `datatype` (`raw` = raw pixels; message must contain width/height), `mimetype` (required; `application/json`, `text/html`, `text/xml` or short forms `json`/`xml`/`text`), `maxkeep` (default 1), `maxfrequency`. The related **MessageDataCatalog** uses the same store table to expose whole messages/binary blobs (camera frames etc.).
3. **Private data over HTTP** — component: `api->setPrivateData("My Data", buf, len, "application/json")`; browser: `GET /api/getcomponentdata?compid=15&name=My%20Data&format=json` (formats: `text`/`xml`/`json`/`html`/`binary`).
4. **Custom component tab** — `api->addPsyProbeCustomView("Title", "elements/mytab.html")` or a `<customview>` spec child. A template is an HTML fragment with a script defining `var TemplateCompID`, `var TemplateCompName`, `loadTemplate($target, compID)` (once) and `updateTemplate($target, compID)` (every update cycle). Copy `elements/whiteboardmessages.html` as the canonical example. Catalogs can also serve live queries to their own tabs via `/api/query?from=<name>&query=...` (web calls carry `HTTP_OPERATION`; answer with a `JSON` string entry).

---

## 9. Advanced Real-World Example: CoCoMaps

CoCoMaps — a CMLabs multi-robot human-robot-interaction system (TurtleBots + humans, speech dialogue, turn-taking, ROS navigation) — is the best worked example of extending Psyclone with **custom catalogs**. Each robot runs its own Psyclone system; the systems federate through a shared world model. Key lesson: **a custom catalog is just an exported `int8 Fn(PsyAPI*)` crank with state and CTRL_QUERY handling**, declared with `<catalog type="...">`; its `<setup>` XML arrives as the `componentsetup` parameter.

### 9.1 CCMCatalog — the Collaborative Cognitive Map
The shared world model: maps (floor plans), identities (robots/humans with portraits/faces), live objects, timestamped typed observations (Location, String utterances…), vantage/named navigation points, and distributed **roles** and **tasks** (Exclusive/Shared, with assignment history and timeouts). One master serves many systems; the query loop dispatches on a `Query` string entry: data ingest (`CreateObject`, `AddObservations`, …), reads (`map`, `objects`, range/value observation queries — also served as JSON to PsyProbe HTTP calls), navigation (`GetNamedPoint`, `SetVantagePoint`, …), federation (`CCMHeartbeat`, `post` relay), roles (`takerole`/`leaverole`/`giverole` → posts `RoleAssigned`/`RoleLeft`/`RoleGiven`) and tasks (`createtask`…`completetask` → posts `TaskCreated`/`TaskAssigned`/`TaskTimeout`/…). Replies via `api->queryReply(inMsg->getReference(), status, replyMsg)`. Registers a jCanvas map view: `api->addPsyProbeCustomView("Cognitive Map", "elements/ccmcatalog.html")`.

```xml
<catalog name="CCMMaster" type="CCMCatalog">
  <parameter name="SystemID" type="Integer" value="%SystemID%" />
  <parameter name="StorageDir" type="String" value="%DataDir%/CCMData" />
  <trigger name="ObjectInfo" type="Object.Information" />
  <post name="RoleAssigned" type="Role.Assigned" />
  <post name="TaskCreated"  type="Task.Created" />
  <setup>
    <map name="main" view="10" invertx="yes" />
    <identity id="3" type="human" name="Thor List" nickname="Thor" />
    <roles>
      <role name="Searcher" type="Shared" />
      <role name="Communicator" type="Exclusive" />
    </roles>
  </setup>
</catalog>
```

### 9.2 CCMProxyCatalog — transparent federation
Slave systems run a proxy **under the same name** as the master CCM, routing queries remotely — local modules never know the world model is on another machine:

```xml
<catalog name="CCMMaster" type="CCMProxyCatalog">
  <parameter name="SystemID" type="Integer" value="%SystemID%" />
  <query name="Master" source="CCMMaster" host="%MasterAddress%" port="%MasterPort%" />
  <!-- same post list as the master, so events re-emit locally -->
</catalog>
```

### 9.3 CCMCollector — messages → observations
A bridge catalog whose `<setup>` holds `<obs name obstype trigger>` entries with `<mapping entry key/>` children: on each trigger it extracts fields from the message and sends one `Query=AddObservations` bundle to the CCM via `queryCatalog`.

### 9.4 TaskManagerCatalog — hierarchical task scripting
Tasks (arbitrary nesting) defined in a **JSON file** (`TaskFile` parameter); each task has per-event message triggers/posts (`fail, success, complete, stop, timeout, start, next, previous, restart, startnow`), timeouts and delays. Standout feature: **dynamic subscription registration** — after parsing the JSON it builds a subscription XML fragment at runtime and registers it with `api->addSubscription(...)` (self-triggering posts dropped with a warning). Publishes its live task tree as private JSON (`api->setPrivateData("JSON", ..., "application/json")`) rendered by a vis.js Network custom tab (`elements/hierarchicaldiagram.html`).

### 9.5 TimeSeriesCatalog — typed time series + live charting
`<setup>` declares `<dataset name datatype color shoulders shading maxkeep maxfrequency/>` and `<store dataset trigger key|value timekey durationkey datatype/>` bindings; incoming trigger messages append capped samples; `Query=<dataset>` (+ `sinceid/sincetime/maxcount/maxage/minage`) returns JSON; `List=all` returns metadata. A SmoothieChart custom tab streams it live — CoCoMaps charted turn-taking dynamics (who has the floor, speech on/off, pitch) in real time.

### 9.6 ControlPanel — interactive screen/menu catalog
A JSON-defined hierarchy of screens/options (optionally PIN-protected) that robots or humans navigate via queries (`OptionSelect`, `BackNavigation`, `EnterPIN`, `HTML`, …) returning a `PanelResponse` enum (SUCCESS/UNKNOWN/BUSY/NOACCESS/ENTERPIN/NONE/FAILED). Its companion **ControlPanelController** crank bridges pub/sub → query: for each trigger it queries the panel and posts `<TriggerName>Success` / `<TriggerName>Failed` — a reusable convention-based request/response bridging pattern. The catalog renders its own screen HTML; the PsyProbe tab just hosts it.

### 9.7 System-shape lessons
- Federation by **same-name proxy catalogs + remote `<query host= port=>`** (an application-level alternative to multi-node specs).
- Roles gate behaviour: mode messages (`dialog.on/off`, …) or explicit `<context>` blocks switch whole subsystems.
- A `MessagesOfInterest` wildcard whiteboard as a non-intrusive debugging tap; a `ReplayCatalog` (`maxsize="1000000000"`, ~30 types) records demos for offline replay (`*_static_*.xml` variants).
- PsyProbe doubles as the operator UI: `<alias>` subsites, clickable `<post>`s, MessageDataCatalog serving camera frames.
- `<include file="system.inc"/>` + `%variables%` keep one spec family portable across master/slave/virtual/replay variants.
- A Python-crank module (turn-taking model) and a GUI in its own external space show polyglot, crash-isolated integration.

---

## 10. Recording & Replay — the shipped mechanism

Recording is done by **catalogs in the PsySpec** — primarily the **ReplayCatalog** (IMPLEMENTED). The per-component `<recording>`/`<playback>` child tags are a separate, **parsed-only STUB** mechanism.

**Record** (declare triggers):

```xml
<catalog name="Replay1" type="ReplayCatalog" root="./replay1"
         maxsize="10240000" maxcount="2000">
  <trigger name="Video"  type="robot.sensor.video" />
  <trigger name="Bumper" type="robot.sensor.bumper" />
</catalog>
```

Keeps the newest 2000 messages / ~10 MB (first cap hit wins). **Playback** (declare posts over the same root; names must match the recording's trigger names, types may be remapped):

```xml
<catalog name="Replay1" type="ReplayCatalog" root="./replay1">
  <post name="Video"  type="robot.sensor.video" />
  <post name="Bumper" type="robot.sensor.bumper" />
</catalog>
```

Messages replay at their recorded intervals; `interval="1000"` overrides pacing; `rotate="yes"` loops. Record in one system, copy the root directory, replay in another — downstream modules cannot tell replayed from live. Minimal working pair: `Examples/messagerecord.xml` / `Examples/messageplayback.xml`. Other built-in catalogs: **FileCatalog** (directory tree as a queryable store; `ReadOnly` parameter; query with `subdir`/`ext`/`binary` scoping and `read`/`write` operations), **DataCatalog** (key/value store in one file, flushed at `interval`), **RequestStore**/**MessageDataCatalog** (§8.3).

---

## 11. Roadmap Features — NOT SHIPPED (never claim these work)

| Feature | Spec surface today | Status |
|---|---|---|
| **Supervisors** (global; sim-/performance-/ad-hoc-manager specialisations, standard PsyProbe entry) | `<supervisor>` parsed, log-only | ROADMAP (T1.1, sim-manager first, in progress) |
| **Node-local builder** — single local executor per node for module actions (create/load/execute/delete/move/registration), driven by `SYS_BUILDER_*` messages; basic non-LLM tier auto-present on every node | — | ROADMAP |
| **Builder LLM tier / LLM sim-manager** (build Python or C++ from code or description via `<llm>` connections) | — | ROADMAP (later phase) |
| **Non-LLM sim-manager** — test-harness supervisor: build module set, run, check XML success criteria, publish lifecycle events (`created/started/status/success/fail`), terminate by message, multi-node | — | ROADMAP |
| **`<llm>` named connections** — platform primitive for LLM access (Gemini/Bedrock/OpenAI/Anthropic; `global="true"` master-proxying; `apikeyref` secrets never in XML; native CMSDK transport incl. SSE; token/cost tracking + PsyProbe LLM section) | — (XML surface reserved) | ROADMAP (Phase 2 forward design — none of it is built) |
| **`<executable>` external auto-start** (T1.4) — launch the declared command line when its external space is created (`%nodeid%` substitution, PID tracking, `consoleoutput`/`autorestart` honoured, node-level `<executable name= cmdline=>`) | Parsed, **never launched** | ROADMAP — in progress; treat as roadmap until merged |

**Shipped-code STUBs (parsed-only; recognised but inert):** `<stream>` (empty backing function), `<feed>` (log-only, never a component), `<phase>` (no parsing code at all), `<authentication>` in interfaces (empty parser body) + interface encryption attrs, per-component `<recording>`/`<playback>`, `<executable>`, `<triggers>`/`<retrieves>`/`<posts>`/`<signals>` grouping tags, custom `<service>` receivers (NULL — requests discarded), whiteboard `root=` persistence, `migration`/`priority`, `system=` trigger attribute, RequestStore `keytype`, catalog Media/Google types.

---

## 12. Common Gotchas & Obsolete Facts

**Obsolete claims (do not repeat):**
- "SSL just works with self-signed certs" — **OBSOLETE**. Verification (peer + hostname) is on by default; self-signed fails unless `allowselfsigned="yes"` or a custom CA is configured (§7).
- "Per-module verbose/debug/logfile are parsed but ignored" — **OBSOLETE**; they are fully wired (T1.5).
- "Only spec/port/html/verbose/debug on the command line" — **OBSOLETE**; any `key=value` is a variable override (T1.5).
- Toolchain versions VS2015 / OpenSSL 1.0.2g / Python 2.7/3.5 / GCC 5 are dated reference minimums; Python 2 is EOL/legacy.
- "Mac OSX on the roadmap" and "Java support being added" — do not claim without verifying the current build.
- Old-manual errata: `getParameterInt` used to fetch a Float parameter in one example; the C++ namespace is **`cmlabs`**, not "cmsdk" (that's the Python module); the RequestStore section was duplicated verbatim; a whiteboard example referenced WB1 where WB2 was defined.

**Gotchas:**
- **Interval triggers need a first real trigger** — `interval=` timers start only after the module has been triggered once (kick-start with `Psyclone.Ready`).
- **`equals` filter is exact strcmp** — case-sensitive, no wildcards; and **misspelled filter types are silently ignored** (the filter never rejects).
- **Shared-memory struct changes need `make clean`** — stale objects with the old layout cause "impossible" crashes across binaries.
- **POST result codes:** `postOutputMessage` returns the count posted; negative = `POST_FAILED` (-1), `POST_NOSPEC` (-2, posted before any trigger arrived), `POST_OUTOFCONTEXT` (-3, context switched away). A silently-dead module is often `POST_OUTOFCONTEXT`; external modules must check `getCurrentTriggerContext()` themselves.
- **Debug builds load `<Lib>Debug` first** — a stale debug library shadows a fresh release build.
- **Unmatched messages vanish** unless posted with `ttl`; verify subscriptions in PsyProbe before assuming loss. A `maxage` filter deliberately skips stale messages under load.
- **PsyProbe `<alias>` index quirk** — only the first alias is reliably read.
- **Delete rules:** the caller owns buffers returned by string `queryCatalog` (`delete []` in C++); never delete a `PsyAPI` obtained from a `PsySpace` (the space owns it).
- Duplicate trigger names (per component) and duplicate crank names (system-wide) are rejected at parse time.

---

## 13. Complete Worked Examples

### 13.1 Ping-pong (the canonical first system)

Two modules bat a message back and forth. With the built-in `Ping` crank, no user code is needed:

```xml
<psySpec>
  <module name="Ping">
    <trigger name="Ready" type="Psyclone.Ready" />  <!-- start the rally at system boot -->
    <trigger name="Ball"  type="ball.1" />
    <crank   name="Ping"  function="Ping" />         <!-- built-in crank; no library needed -->
    <post    name="Ball"  type="ball.2" />
  </module>
  <module name="Pong">
    <trigger name="Ball" type="ball.2" />
    <crank   name="Pong" function="Ping" />
    <post    name="Ball" type="ball.1" />
    <post    name="Done" type="Psyclone.Shutdown" /> <!-- after 100,000 messages -->
  </module>
</psySpec>
```

Run: `Psyclone spec=pingpong.xml`. Startup posts `Psyclone.Ready` → Ping posts `ball.2` → Pong posts `ball.1` → … for 100,000 messages (throughput stats every 10,000), then Pong posts `Psyclone.Shutdown`. Note the modules never mention each other — only message types.

**The same system with your own C++ crank** (both modules can share one function; the trigger/post *names* differ per module in the spec, the code only uses names):

```cpp
// library "Examples" → Examples.dll / libExamples.so; register with
// <library name="MyExamples" library="Examples"/> and function="MyExamples::PingPong"
namespace cmlabs { extern "C" { DllExport int8 PingPong(PsyAPI* api); } }

int8 PingPong(PsyAPI* api) {
    while (api->shouldContinue()) {
        DataMessage* msg = api->waitForNewMessage(100);      // any of this module's triggers
        if (!msg) continue;
        int64 count = msg->getInt("Count");                  // 0 if absent (Psyclone.Ready kick-off)
        DataMessage* out = new DataMessage();
        out->setInt("Count", count + 1);
        int posted = api->postOutputMessage("Ball", out);    // post NAME; spec decides the type
        if (posted < 0) api->logPrint(1, "post failed: %d", posted);
    }
    return 0;
}
```

**And the Python crank version**, inline in the spec (an `api` object is auto-provided):

```xml
<module name="Ping">
  <trigger name="Ready" type="Psyclone.Ready" />
  <trigger name="Ball"  type="ball.1" />
  <crank name="PingPy" language="python3"><![CDATA[
while api.shouldContinue():
    msg = api.waitForNewMessage(100)
    if msg is None: continue
    count = msg.getInt("Count")
    out = cmsdk.DataMessage()
    out.setInt("Count", count + 1)
    api.postOutputMessage("Ball", out)
]]></crank>
  <post name="Ball" type="ball.2" />
</module>
```

(Reminder: Python-crank modules automatically run in their own process space.)

### 13.2 A richer system: whiteboard + catalog + filter + context switch

A sensor feeds detections; a whiteboard stores them keyed on confidence; an analyser retrieves high-confidence ones on a timer; a FileCatalog persists reports; a supervisor-style module flips the system between `Mode.Active` and `Mode.Idle` contexts.

```xml
<psySpec>
  <variable name="DataDir" value="./data" />   <!-- override: Psyclone spec=sys.xml DataDir=/mnt/rig -->

  <!-- 1. Whiteboard: stores detections, indexed by time (automatic) and by confidence -->
  <whiteboard name="Detections" key="Confidence" keytype="float" maxcount="5000">
    <trigger name="Any" type="percept.detection.*" />       <!-- wildcard subscription -->
  </whiteboard>

  <!-- 2. File catalog: persistent report store -->
  <catalog name="Reports" type="FileCatalog" root="%DataDir%">
    <parameter name="ReadOnly" type="String" value="no" />
  </catalog>

  <!-- 3. Sensor module: only strong, fresh, well-formed detections pass the trigger -->
  <module name="Analyser" verbose="3" logfile="%DataDir%/analyser.log">
    <context name="Mode.Active">                             <!-- live only in Mode.Active -->
      <trigger name="Kick" type="Psyclone.Ready" />          <!-- kick-start so interval runs -->
      <trigger name="Tick" type="Regular.Post" interval="2000" />
      <trigger name="Strong" type="percept.detection.human" maxage="500">
        <filter type="greaterthan" key="Confidence" value="0.8" />
        <filter type="haskey"      key="Identity" />
      </trigger>
      <retrieve name="recent" source="Detections" key="Confidence" keytype="float"
                start="0.8" end="1.0" maxcount="20" maxage="60" />
      <query name="Reports" source="Reports" ext="txt" />
      <crank name="Analyse" language="python3"><![CDATA[
while api.shouldContinue():
    msg = api.waitForNewMessage(100)
    if msg is None: continue
    msgs = api.retrieve("recent")               # last-minute detections with Confidence in [0.8, 1.0]
    report = "%d strong detections" % len(msgs)
    api.queryCatalog("Reports", "summary", "write", report)   # writes ./data/summary.txt
    out = cmsdk.DataMessage()
    out.setString("Report", report)
    api.postOutputMessage("Summary", out)
]]></crank>
      <post name="Summary" type="report.summary" ttl="30" />
    </context>
  </module>

  <!-- 4. Mode controller: passthrough module (no crank) switching contexts -->
  <module name="ModeSwitch">
    <trigger name="GoIdle"   type="cmd.mode.idle" />
    <trigger name="GoActive" type="cmd.mode.active" />
    <post name="Idle"   context="Mode.Idle" />     <!-- context-change posts -->
    <post name="Active" context="Mode.Active" />
  </module>
</psySpec>
```

What this demonstrates: wildcard whiteboard subscription; a float keyed index queried by range from a `<retrieve>`; trigger `<filter>`s (numeric + haskey) and `maxage`; the interval-needs-a-kick rule; per-module logging; a FileCatalog written through the string query API; `ttl` keeping the summary available; and a passthrough module whose posts carry `context=` so a single incoming command reroutes the whole system (posting `Mode.Idle` deactivates the `Mode.Active` block — the Analyser's crank sees `shouldContinue()==false` and exits cleanly; posts from it would return `POST_OUTOFCONTEXT`).

---

## 14. Quick Reference Appendix

**PsyType wildcards:** dot-notation, left-to-right specificity; `*` matches one segment position pattern as in `input.audio.*` (any sub-type) and `input.*.raw` (any middle segment). Exact match otherwise. Trigger/post *names* are code-facing; *types* are spec-facing.

**POST_* result codes** (`postOutputMessage` returns messages posted; negative = error):

| Code | Value | Meaning |
|---|---|---|
| `POST_FAILED` | -1 | General failure |
| `POST_NOSPEC` | -2 | Posted before any trigger arrived (no post spec resolved) |
| `POST_OUTOFCONTEXT` | -3 | Active context changed; crank no longer live |

**QUERY_* status codes** (shared by retrieves and queries):

| Code | Value | Meaning |
|---|---|---|
| `QUERY_FAILED` | 1 | General failure |
| `QUERY_TIMEOUT` | 2 | No reply within the timeout |
| `QUERY_NAME_UNKNOWN` | 3 | No retrieve/query with that name |
| `QUERY_COMPONENT_UNKNOWN` | 4 | Source component does not exist |
| `QUERY_QUERYFAILED` | 5 | The component rejected the query |
| `QUERY_SUCCESS` | 6 | Results delivered |
| `QUERY_NOT_AVAILABLE` | 7 | Component not currently available |
| `QUERY_NOT_REACHABLE` | 8 | Component could not be reached |

**Filter types:** `equals` (exact strcmp), `notequals`, `equalsnumeric`, `notequalsnumeric`, `greaterthan`, `lessthan` (all key+value), `haskey` (key only). Unknown types silently ignored.

**Parameter value syntax:** `42` · `[0:110]` · `[0:110]=55` · `[0:110]=*` · `[0:110:2]` · `[-2.5;1.5;1.8]` · `['a';'b';'c']`; parameter types String/Integer/Float; data-type ids PARAM_STRING 1 … PARAM_FLOAT_COLL 6.

**Keytypes** (whiteboard/retrieve indexes): `string` | `integer` | `float`/`double` | `time` (default; invalid keytype drops the retrieve spec). Age windows: `<trigger maxage=>` is a stale-message skip filter; the ranged retrieve **API** calls (`retrieveIntegerParam` etc., args `start, end, maxcount, maxage, timeout`) take `maxage` in **microseconds**. The XML `<retrieve maxage=>`/`<query maxage=>` attribute is parsed as value ×1000; its exact wall-clock unit is being confirmed against the engine build, so for a precise age window prefer the API call with an explicit microsecond value.

**Intersystem query entries** stamped on remote queries: `INTERSYSTEM_IDENTIFICATION` (string), `INTERSYSTEM_ADDRESS` (int, uint32 IP), `INTERSYSTEM_PORT` (int), `INTERSYSTEM_SOURCENAME` (string).

**Built-in types/contexts:** `PsyControl`, `Psyclone.Ready` (default context, always active), `Psyclone.Shutdown`. Every component gets an implicit `DefaultComponentTrigger` on `CTRL_SYSTEM_READY` — that is why components wake at system start.

**HTTP APIs:** `GET /api/getcomponentdata?compid=N&name=X&format=text|xml|json|html|binary`; `GET /api/query?from=<catalog>&query=<name>[&...]`.

**Defaults worth memorising:** main port 10000; verbose/debug 1/1; context `Psyclone.Ready`; space `Root`; `selftrigger="warn"`; posts guaranteed; SSL verify-peer with `allowselfsigned` **false**.

---

*Generated for the Psyclone 2 distribution (T1.7 documentation). Statuses verified against the source tree in July 2026; when a ROADMAP item ships, re-verify before relying on this document's status labels.*