Architecture

MindRoom's architecture consists of several key components working together.

Overview

┌─────────────────────────────────────────────────────────┐
│                   Matrix Homeserver                      │
│              (Synapse, Conduit, etc.)                    │
└──────────────────────┬──────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────┐
│              MultiAgentOrchestrator                      │
│  ┌─────────────────────────────────────────────────┐    │
│  │                   Matrix Client                  │    │
│  │         (nio, sync loops, presence)             │    │
│  └─────────────────────────────────────────────────┘    │
│                                                          │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────┐    │
│  │ Router  │  │ Agent 1 │  │ Agent 2 │  │  Team   │    │
│  └────┬────┘  └────┬────┘  └────┬────┘  └────┬────┘    │
│       │            │            │            │          │
│  ┌────▼────────────▼────────────▼────────────▼────┐    │
│  │              Agno Runtime                       │    │
│  │         (LLM calls, tool execution)            │    │
│  └─────────────────────────────────────────────────┘    │
│                                                          │
│  ┌─────────────────────────────────────────────────┐    │
│  │                Memory System                     │    │
│  │  (Mem0, file, or none; agent/team scopes)       │    │
│  └─────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────┘

Components

Matrix Integration - How MindRoom connects to Matrix
Agent Orchestration - How agents are managed
Bot Runtime - The inbound turn pipeline and its module boundaries
Migration and Compatibility Boundaries - Current owners for historical formats, dependency migrations, and retained compatibility
Matrix Event-Journal Security - Which decrypted plaintext is durable, who owns it, and what removes it
Matrix Event-Journal Contracts - What the journal guarantees, and the homeserver behaviour you would otherwise rediscover by debugging

Key Internal Modules

Module	Purpose
`knowledge/collection_lifetime.py`	Compatible publication selection, shared reader locks, and exclusive collection reclamation
`orchestrator.py`	MultiAgentOrchestrator — boots entities, manages sync loops, hot-reload
`orchestration/`	Extracted orchestrator helpers (sync loops, config diffing, room invitations)
`orchestration/config_lifecycle.py`	Debounced config-reload lifecycle: queueing, response drain, and update-plan dispatch
`runtime_state.py`	Shared runtime readiness state for health/ready endpoints
`runtime_resolution.py`	Authoritative runtime resolution for agent materialization
`team_exact_members.py`	Runtime resolution for team member materialization
`model_loading.py`	Authoritative model instantiation and provider-specific loader selection
`ai_runtime.py`	Agent-run input preparation and queued-notice hooks
`agent_storage.py`	Agent session and learning SQLite storage construction helpers
`session_storage_preflight.py`	Required session-column checks and retained archives for incompatible owned session stores
`agent_descriptions.py`	Shared agent description rendering for routing and delegation
`agent_policy.py`	Derives canonical execution policies from authored agent config
`workspaces.py`	Agent workspace scaffolding, template seeding, context file resolution
`tool_system/google_workspaces.py`	Workspace-specific Google OAuth provider construction and tool registration
`bot.py`	AgentBot and TeamBot runtime shells for Matrix lifecycle and sync callbacks
`matrix/journal_ingress.py`	The boundary where Matrix events become durable facts; nio provenance decides actionable vs context-only
`event_journal/`	Durable ownership of admitted Matrix events, conversation projection, and delivery outbox
`journal_dispatch.py`	Fan admitted journal events out to typed Matrix callbacks and settle the ones that finish
`pending_event_worker.py`	Decides when pending journal work runs, and wakes itself again whenever a pass stops early
`turn_controller.py`	TurnController — owns one inbound turn from ingress to recorded outcome
`ingress_validation.py`	Ingress boundary validation: trust, effective requester, handled-id dedup, router-echo drop, command detection
`inbound_turn_normalizer.py`	Raw input shaping (text, voice, sidecars, media) into canonical turn inputs
`conversation_resolver.py`	Conversation identity, thread history, and ingress envelope assembly
`ingress_lanes.py`	Per-(room, sender) receipt-order FIFO delivering resolving ingress (voice/STT readiness) to conversations
`coalescing.py`	Live message coalescing gate; ordinary text dispatches immediately, adaptive text waits for its quiet window, and media waits for attachments and a trailing caption
`text_ingress_dispatch.py`	Text ingress dispatch path used by TurnController
`turn_policy.py`	Pure turn policy: decide ignore, route, or respond for inbound turns
`participation.py`	Framework-independent participation state: one immutable decision, concurrent checks, and approval-preserving settlement
`agno_participation.py`	Agno participation adapter: prepared request checks, primary-run isolation, metrics, and scoped model interception
`provider_tool_policy.py`	Task-local restriction enforced by provider adapters before native tools can execute
`groq_model.py`	Groq adapter enforcing provider tool restrictions for Compound systems
`config/participation.py`	Opt-in room participation settings: designated agent, bounded pause, and decision instructions
`command_turn_executor.py`	Command execution and durable command/config mutation journals
`reaction_dispatch.py`	Durable semantic routing for Matrix reactions
`user_stop_reconciliation.py`	STOP ordering, response cancellation, and terminal turn reconciliation
`visible_response_reconciliation.py`	Visible Matrix response recovery, adoption, and replay reconciliation
`turn_store.py`	Unified durable turn access (wraps the handled-turn ledger)
`handled_turns.py`	Disk-backed handled-turn ledger preventing duplicate responses
`response_runner.py`	Response lifecycle execution (locking, streaming vs non-streaming, cancellation, detached inbox responses, shutdown drains)
`response_turn.py`	Shared blocking and streaming response-turn drivers, including retries and dynamic-tool continuation
`response_attempt.py`	Executes one visible response attempt with stop tracking
`response_terminal.py`	Classifies pending-visible failures and terminal stream outcomes
`response_lifecycle.py`	Shared response lifecycle helpers and queued-notice state
`approval_tools.py`	Rebuilds required saved-approval tools from current configuration and scoped MCP catalogs for agent and team continuation
`execution_preparation.py`	Request-scoped execution preparation for prompts and persisted replay
`response_payload_preparation.py`	Execution-side, under-lock assembly of one response's payload from immutable ingress inputs
`delivery_gateway.py`	Visible Matrix delivery for already-generated responses (send, edit, finalize)
`custom_tools/chat_ui.py`	Runtime-bound MindRoom Chat UI action requests with canonical Matrix conversation and sender identity
`custom_tools/matrix_message_idempotency.py`	Bounded durable keyed Matrix sends: preparation, receipts, retention, replay, and current authorization checks
`tools/chat_ui.py`	Tool-catalog registration and discovery metadata for Chat UI actions
`visible_voice_echo.py`	Immediate router voice-placeholder delivery, replacement ordering, and deduplication
`post_response_effects.py`	Shared post-response effects after Matrix delivery
`routing.py`	Intelligent agent or team selection when no entity is mentioned
`streaming.py`	Streaming state machine and progressive response state
`media_inputs.py`	Shared media-input container passed across bot, teams, and AI layers
`provider_media_fallback.py`	Retries provider requests without rejected inline media and remembers unsupported kinds per model route for the process lifetime
`file_memory_knowledge.py`	Shared resolution for agent file-memory semantic knowledge overlays
`memory_scope_ids.py`	Cycle-free canonical agent memory scope identifiers
`avatar_generation.py`	Generates and manages avatar assets for agents, rooms, and spaces
`topic_generator.py`	AI-generated room topics
`background_tasks.py`	Non-blocking async task management with GC protection
`desktop/session.py`	Owns the desktop device's durable NIO session and storage binding
`desktop/transport.py`	Polls owned to-device work and acknowledges only after durable command admission
`desktop/command_journal.py`	Persists command admission, execution outcomes, and pending responses
`desktop/bridge.py`	Enforces current local authority and coordinates serial execution and response delivery
`desktop/observations.py`	Bounds observation references by requester, agent, session, application, and age
`desktop/displays.py`	Maps verified logical display bounds to capture pixel scale
`desktop/input.py`	Defines the allowed application-local keyboard and scroll inputs
`desktop/macos_input.py`	Sends bounded Quartz pointer input in global logical coordinates
`desktop/macos_capture.py`	Captures verified windows and displays through ScreenCaptureKit
`desktop/native_config.py`	Validates and persists private native-helper configuration
`desktop/native_protocol.py`	Parses and bounds requests on the local NDJSON channel
`desktop/native_host.py`	Owns helper setup, runtime lifecycle, local control, and stdio dispatch
`desktop/native_entry.py`	Starts the packaged native desktop helper

Storage upgrade boundaries

Historical formats stay with their storage or lifecycle owners, while current callers consume canonical identities and paths. legacy_private_storage_aliases.py owns historical requester spellings and verified aliases; only startup migration and private_storage_paths.py can import it. Worker mount planning and sandbox path validation use private_storage_paths.py, while private_instance_identity_store.py validates current identities.

oauth/legacy_credentials.py owns publication-field normalization and lossless historical requester bindings. Only oauth/credential_store.py can import it; the store retains schema, scope validation, current credential state, transaction locks, retries, and commit ownership. OAuth credentials stored only in legacy JSON files require reconnection; those files and their obsolete sidecars remain untouched.

Existing lifecycle adapters remain at their focused entry points: legacy_private_storage.py at startup, config/legacy_access.py during config loading, and Nio journal and crypto adapters when their stores open. session_storage_preflight.py checks owned session databases before opening them and archives session directories whose tables lack required columns; see Session Storage Recovery. Tach visibility rules keep compatibility internals behind their owning boundaries.

Data Flow

Message arrives from the Matrix homeserver and is committed by matrix/journal_ingress.py before nio is told it was accepted; journal_dispatch.py then hands it through bot.py to turn_controller.py, which owns the turn from ingress to recorded outcome
Input is validated, normalized, and resolved: ingress_validation.py checks trust and the effective requester, deduplicates handled event ids, and drops trusted router echoes; inbound_turn_normalizer.py shapes raw text, voice, and media into canonical turn inputs, and conversation_resolver.py resolves thread identity and history; !commands are control inputs that dispatch directly here instead of entering coalescing
Messages are ordered and coalesced: ingress_lanes.py delivers each sender's messages in receipt order (late-ready voice/STT waits in the lane), and coalescing.py batches each sender's live conversation burst. Ordinary text completes an utterance and dispatches immediately; adaptive text waits its configured quiet period. Later adaptive text cannot extend an earlier immediate text's wait. A live batch ending in media waits for more attachments or a trailing caption. Follow-up backlogs queued behind an active response flush as one combined turn at idle; conversations never wait on each other.
The turn is planned: turn_policy.py decides to ignore, route, or respond; a direct responder is resolved when one eligible agent or team remains, otherwise the router selects among candidates
Selected entity processes the message via response_runner.py and the Agno runtime, executing tools as needed
Response is delivered through delivery_gateway.py, which owns Matrix send/edit/finalization while streaming.py owns progressive response state
The turn is recorded in the durable handled-turn ledger (turn_store.py / handled_turns.py) so restarts do not double-reply
Memory updates asynchronously in background

See Bot Runtime for the module boundaries and the ongoing simplification roadmap.