Operations and diagnostics

A working page, a working provider request, and a working native integration are different acceptance results. Diagnose the failed boundary before changing another subsystem.

This page contains source-derived operating guidance. It is not a service inventory or a claim that every upstream capability runs on the deployed host.

Recorded integration context

The integration owner reports Node v22.23.1 with Corepack pnpm 11.24.0 as the deployment/runtime choice for the application now served at airi.loca.zone. The host did not switch Node versions.

The repository records different Node guidance. See setup for that distinction and private page (sign-in required) for the evidence labels.

Provider credentials, model inference, microphone behavior, native devices, and external bots were not exercised during wiki authoring. Deployment and browser acceptance remain separate evidence.

Authenticated loading investigation — 2026-09-27

The pxl browser session reached the real app, help guide, and provider settings through Authelia. An anonymous redirect was not used as app-health evidence.

The initial root page was blank at widths 390 and 1280. Provider settings and the preserved help guide rendered in the same session. The console identified a blocked embedded WebAssembly request: the wlipsync single-file entry fetched a data: URL. The deployed connect-src policy permits same-origin, HTTPS, and secure WebSocket requests, not data: fetches. This is a loading defect separate from provider authentication.

The source repair uses the supported split wlipsync entry. It emits same-origin WebAssembly and worklet assets, compiles on first use, and registers the worklet once per audio context. Live2D uses the same wrapper. The build and package typecheck passed; the app was redeployed without changing nginx, Authelia, or the preserved help guide. These build results do not establish an OMP inference response.

A real Chrome audio smoke passed with three worklet nodes across two audio contexts. Oscillator input produced lip-sync volume and weights; silence reduced volume below 0.02. The emitted WASM and worklet loaded from the same origin, with no browser errors. A negative control confirmed that the old embedded data: WASM entry is still blocked. This proves the local audio-processing contract, not microphone capture, audible device output, or provider speech.

At the earlier 2026-09-27 authenticated checkpoint, fresh onboarding exposed a second boundary: Live2D model data and textures used browser-created blob: URLs, and the then-observed policy rejected them. The stage showed a model-render error. This is dated rendering evidence, not a statement of the current headers.

Later anonymous HEAD receipts at 2026-09-27T23:52:25.346Z and 2026-09-28T05:20:23Z observed blob: allowances in both connect-src and img-src on the sign-in redirect. Those headers do not prove the authenticated effective policy or successful avatar/Live2D rendering; subsequent captured stages remained blank. This mission made no nginx/CSP change and makes no current ownership claim from the earlier Auth Gatekeeper hold. Full model rendering remains unverified and needs a separately admitted authenticated browser proof, not an assumed policy repair.

The optional web server channel uses an explicit saved endpoint or deployment endpoint. An unconfigured web session does not open a localhost socket. Native runtimes retain their local-server default. Closing the channel cancels pending waiters; an old waiter cannot reopen it or replace a newer authenticated connection. Local context processing does not wait for this optional connection.

omp model bridges (tvrn, airi)

The project-owned omp-model-bridge.py services read modelRoles.default from each project’s .omp/config.yml; a judge role alone is not a primary model. When the default is absent, startup raises project_primary_model_missing. With the recorded restart policy this caused a 5–6 s restart loop and journal flood; both services had exceeded 200,000 restarts before repair.

The applied configuration is block YAML in the project’s existing file, preserving its other settings:

modelRoles:
  default: typesafe/jev-latest:auto
RepairApplied date and restartOperator-local receipt (not a public download)
D-633: tvrn-omp.service2026-11-02, 17:29:15 CET/home/loca/.flowmaster-d181/d633/receipt.json
D-637: airi-omp.service2026-11-03, 02:43:39 CET/home/loca/.flowmaster-d181/d637/receipt.json

Each receipt records an active/running bridge, NRestarts=0 unchanged across a 120 s post-restart window, and a Unix-socket GET /v1/models listing typesafe/jev-latest:auto. These are dated startup/catalog observations, not successful inference, credentials, entitlement or a current browser acceptance test. The 120 s journal counts are not a measured hour of logging.

Configuration custody: AIRI does not track .omp/config.yml (git status showed ?? .omp/config.yml); D-633 also records the Tavern file as untracked. A repo checkout or deployment alone therefore does not carry these fixes. Retain the receipt-backed configuration/preimage evidence; this wiki update does not change either app repository.

Pending restart-policy hardening — wake word bound omp bridges: the recorded /etc/systemd/system/tvrn-omp.service and /etc/systemd/system/airi-omp.service use Restart=on-failure and RestartSec=5 without an explicit start-rate bound. The follow-up is to add StartLimitIntervalSec=300 and StartLimitBurst=5 in each unit’s [Unit] section. This is pending, not an applied change; neither unit is modified by wiki sync.

journald retention

D-636 applied /etc/systemd/journald.conf.d/50-flowmaster-cap.conf on 2026-11-03 (journald restarted at 02:40:24 CET): SystemMaxUse=2G, SystemMaxFileSize=256M, MaxRetentionSec=14day, RateLimitIntervalSec=30s, RateLimitBurst=20000. Operator-local receipt: /home/loca/.flowmaster-d181/d636/receipt.json. Journal use was 1G before and after; this cap is retention/rate protection, not attribution of the separate root-disk growth or proof that the bridge restart-policy follow-up is complete.

Dated D193 standing state — 2026-09-28

The integration owner’s c3 evidence records the pinned airi-omp bridge installed and started at 00:40Z, PID 2532684. Its Unix-socket model listing returned HTTP 200 for exact model zai/glm-5.3-flash; the socket mode was 0600. This readiness probe performed no generation. A running bridge and a listed model do not establish credentials, entitlement, or authorized zero-incremental-cost use.

Earlier D193 observation: the first Vite stage built successfully during 00:00–00:04Z and was promoted at 00:44Z. Frontend PID 2564611 served index SHA-256 d7329406ad667f35ca468235d4884ee1507aedf92587c3de9d078e8f9f297756. Real authenticated Chrome showed no OMP provider at widths 390 and 1280, and /settings/providers/chat/omp showed Provider definition not found. Although native OMP source existed at 55086274e, packages/provider-inference/dist/index.mjs was stale and lacked registration. This is the earlier defect, not the current compiled state.

The admitted dependency-repair controller acquired its lock at 01:52:21.411506Z with nice level 15. It preserved the prior provider dist, successfully built the existing provider-inference package with tsdown, and verified OMP through actual ESM exports getDefinedProvider('omp') and listProviders. A fresh Vite artifact was built and promoted; the frontend alone restarted as PID 3151889. The new live index SHA-256 is 35257f7d3c801dd625462e5e9a720fdea2bbc4103a9990899ee7c208141787fc; the prior artifact was retained.

The controller’s readiness budget expired at 01:56:05Z, before the preview journal announced its listener at 01:56:06Z. It ended with frontend_readiness_failed, exit 1, and released the lock at 01:56:05.211985Z. The parent’s independent loopback GET at 01:57:18.749Z returned HTTP 200, 9,315 bytes, with the new promoted index hash. The compiled registration defect is repaired and the artifact is served; this does not convert the controller’s terminal failure into a pass or prove browser behavior.

The post-repair real UI showed native OMP and selected provider omp with model zai/glm-5.3-flash. At 2026-09-28T04:52:06.830Z, the one observed composer POST requested stream: true with zero tools and returned HTTP 403 JSON {"error":{"message":"Same-origin requests only"}}. The actual DOM displayed a Core System error, with no assistant tokens. Network observation began immediately before composer submission, not during provider setup: this is not a total browser-lifetime or native physical request count. The preceding “Health check failed” badge is configuration-state presentation, not a network health response. Inspected native OMP source has no validators, disables required validation, and has no automatic completion health probes/retries; selection uses a catalog GET, whose bridge handler starts no child. This does not retroactively prove deployed-bundle equivalence or historical request totals.

The user prompt and 403 error persisted across a real reload. This proves error-state persistence, not generated-output persistence or successful streaming. Screenshots at widths 390 and 1280 were saved and reviewed: narrow-screen text was pale and the avatar stage blank. This is not polished-UI acceptance.

The exact owned conversation was deleted through normal native history UI; the marker was then absent from the DOM and all IndexedDB stores. Chrome closed through the owned protocol and exited 0. The continuation controller released at 04:54:58.502320Z; the original preload SHA-256 70287703aaac695932fe4d426feb481f2c715ede8ec4c18ada6ea678779c61a4 was restored.

No diagnostic sidecar was produced; its absence is not successful native-startup evidence. According to source, middleware rejected the composer request before bridge forwarding. Source investigation found the enabled nginx vhost rewriting Host to 127.0.0.1, while the original middleware derived its expected public origin from Host. The exact historical header branch is an inference because the old request headers were not captured.

The approved project-owned correction in apps/stage-web/vite-omp-preview.ts pins the exact deployment origin https://airi.loca.zone for this fixed-socket deployment, not generic development origins. Fetch-Metadata, absent-Origin behavior and all other guards remain unchanged; forwarded headers are not trusted. The admitted origin runner’s focused actual-middleware contract test exited 0. Frontend restart PID 555449 activated the correction without a Vite rebuild; served index SHA-256 remained 35257f7d3c801dd625462e5e9a720fdea2bbc4103a9990899ee7c208141787fc. All eight live text/plain guard cases returned their expected 415/403 results without native dispatch. No badge repair or forced-valid state was made.

For this subsequent browser session, passive network capture covered pre-navigation through cleanup, 05:13:27.460Z–05:17:40.124Z. It recorded one catalog GET 200 and exactly one completion POST, at 2026-09-28T05:14:56.915Z, for zai/glm-5.3-flash with stream: true and zero tools. The initial blank page before capture attachment and final teardown after CDP disconnection were not recorded; this is not an entire spawned-browser-lifetime count. The response was HTTP 502 startup_network_not_admitted; no assistant tokens, retry, fallback, or 429 occurred in the captured attempt. The approved diagnostic sidecar for child 575606, timestamp 05:15:08.050Z, recorded loopback_catalog/get, inference active false, requests 0 and dispatches 0. That directly bounds this AIRI child’s failure before native inference dispatch; it is not a finding about Tavern or successful native startup.

The prompt and 502 error persisted on actual reload, not generated output. The unique owned conversation was deleted through native history UI; its marker was absent from the DOM and all IndexedDB stores. Chrome PID 556654 closed through the acknowledged protocol and exited 0. The origin runner released at 05:17:42.728999Z and restored preload SHA-256 70287703aaac695932fe4d426feb481f2c715ede8ec4c18ada6ea678779c61a4. The exact sidecar bytes were preserved with SHA-256 8d6227a066d34579561654cef63b6ea712e89543f6bcfd4678bbb37e1f842db8 before the sidecar was removed. Origin repair is activated and proven; native inference remains blocked by loopback catalog startup denial. Supported-control analysis is read-only; no further request or policy widening is admitted.

Completed read-only startup analysis found that the child overlay omits the exact native loopback discovery IDs ollama, llama.cpp, lm-studio, litellm and vllm. Supported exact exclusions can remove those probes, but the existing CLI still starts selected-provider online-if-uncached catalog discovery. The inspected zai cache expired at 05:15:49.156Z, after the observed 05:15:08.050Z loopback denial: this later cache prerequisite is source/state analysis, not another observed failure. No supported persistent CLI offline-startup switch was established. Public SDK registry injection and genuine ACP embedding exist in source, so ACP itself is not a disqualifier; however, an installed compatible SDK dependency graph, remaining runtime refresh/preconnect behavior and preserved native guard/tools-zero contracts were not established as a complete permitted path. No speculative exclusion repair, SDK implementation, cache alteration or further request was performed. The terminal result remains blocked under the admitted constraints, not a universal SDK-impossibility claim.

There was no retry, fallback, or 429 in the observed composer attempt. No generated-response, streaming, entitlement, or native-startup success is established. The no-tools/MCP/ambient-sources, no-retry/fallback and first-429 durable-stop boundaries remain in force.

Tavern’s separate evidence records one app request returning HTTP 502 startup_network_not_admitted, with no reply. Its native physical request phase/count remain unresolved; later AIRI evidence cannot resolve that earlier Tavern request.

The September D193 source update was not published at that checkpoint because the shared publisher was held. That historical hold is not a current admission instruction. Its operator-local evidence remains on the private provenance page (sign-in required), not offered as public downloads; the November bridge maintenance above does not reopen or resolve the native-inference boundary.

Dated c2 history — 2026-09-27 evidence round

The earlier staged build exited 1 after 603.76 seconds at heavy-lock admission timeout. At that checkpoint no candidate had been staged or promoted, the bootstrap/guard/bridge/unit candidates were uninstalled, and the prior dist was unchanged. Those are historical observations, not the current deployment state. The original c2 source/proof record, including fake-ACP and installed-native fixture limits, is preserved on the private provenance page. None of those fixtures established real provider entitlement or generation.

Diagnose the first failed boundary

flowchart TD
    Failure[Unexpected behavior] --> Shell{Does the application load?}
    Shell -->|No| Assets[Inspect build assets and browser errors]
    Shell -->|Yes| Scope{Showcase or full runtime?}
    Scope -->|Showcase| Guide[No character session implied]
    Scope -->|Full runtime| Text{Does a text request work?}
    Text -->|No| Provider[Inspect provider, model, endpoint, and credentials]
    Text -->|Yes| Feature[Isolate hearing, speech, display, or external service]
    Feature --> Evidence[Observe the actual failed operation]
SymptomFirst inspectionSource owner
Provider appears selected but chat failsEndpoint, selected model, credentials, and the real request errorConsciousness and provider stores
Chat works but no voice playsSpeech provider, model, voice, and device outputSpeech store and audio pipelines
No transcriptionMicrophone permission, available input, provider capability, and transcription modelHearing store
Imported model does not appearSelected asset, renderer error, graphics support, and required filesDisplay-model store and renderer package
Native speech is absentApplication surface, OS, and native availabilityApple Speech provider
Integration settings exist but nothing connectsSeparate process, credentials, destination, and server-channel connectionSelected integration

The Hearing source distinguishes permission, device, input, and provider failures. Those categories are more useful than treating every failure as an inference error.

A source configured flag is a configuration predicate. It is not a substitute for a request or device check.

Protect state before recovery

The provider configuration store uses localStorage. The display-model store uses localforage for imported assets. Provider configuration also has remote operations.

Do not assume that browser storage is encrypted, backed up, or identical to account-side state. Keep original assets and recovery information outside the browser.

Storage clearing can remove user data

Preserve required configuration and original assets before clearing site data. A cache-recovery action can also remove locally stored state.

The web Vite configuration documents an existing service worker as a possible cause of stale development behavior. A previous production or preview registration can survive later development sessions.

Before destructive recovery, inspect the actual origin, loaded assets, and service-worker registration. The deployed site and a separate upstream development origin do not share one universal browser state.

Keep service checks separate

The local backend Compose file defines readiness checks for API, Auth, PostgreSQL, and Redis. A container start does not establish service readiness.

The backend guide gives the API ownership of shared migrations. It does not give that responsibility to Auth. The local edge rejects the API’s internal routes.

The server channel is a separate runtime. Its connection result does not establish hosted account, billing, or database readiness. See private page (sign-in required).

Maintain the source guide

  1. Record the new upstream revision before changing a source-based claim.
  2. Read changed manifests and the owning implementation.
  3. Update affected pages and their immutable source links together.
  4. Preserve the distinction between source support and runtime evidence.
  5. Record the exact surface and operation for each acceptance result.

The root build:web command is a source build procedure. A successful build alone does not prove keyboard behavior, mobile layout, audio, or provider access.

Source context

Navigation: index · setup · workflows · private page (sign-in required)

current_url_state

  • Documentation URL: AIRI operations, on the public wiki root; private provenance stays behind its existing sign-in boundary.
  • Runtime state: D-633 (2026-11-02) and D-637 (2026-11-03) record healthy Unix-socket model listings after the configuration fix; D-636 (2026-11-03) records the journald cap. These are receipt observations, not fresh runtime probes by wiki sync.
  • Open state: bound omp bridges restart-policy hardening is pending. No new app/inference/browser acceptance or service restart is claimed by this documentation update.