Personal operations system · live agent stack
Jarvis Multi-Agent Operations
A multi-agent operations system that routes natural-language requests through one master agent to email, calendar, and meeting specialists, with provider tools, shared context, durable workflow execution, and chat and voice entry surfaces.
Jarvis is a private operational system because it connects to personal communication and calendar data. The architecture shown here is verified from the running workflow and services; credentials and message content remain private.
1
master routing agent
3
specialist operational agents
3+
provider domains behind bounded tools
2
chat and voice entry surfaces
The system
From operational problem to reliable workflow.
Problem
Personal operations span systems with different permissions, data shapes, and failure modes. A single unconstrained prompt becomes difficult to reason about when it can read email, change calendars, inspect meetings, and act through communication tools.
Approach
Jarvis uses a master agent as the conversational front door and routing layer. Email, Calendar, and Meeting agents each own a narrower tool set and execution contract. n8n provides durable workflow orchestration, provider integrations expose bounded actions, and memory carries useful context without flattening every domain into one agent.
Outcome
The system operates as a private daily workflow layer with chat and a live voice service, specialized Gmail, Google Calendar, and Fireflies paths, and an architecture that can add tools without giving every agent unrestricted access to every account.
Architecture
System architecture
Jarvis is an orchestration system, not one all-powerful chatbot. The master agent interprets an operator request, selects the appropriate specialist, and returns a consolidated answer. Specialist agents own narrower Gmail, Calendar, and meeting-intelligence actions. This reduces prompt and permission sprawl while keeping a single conversational interface.
Interaction surfaces
Accept operator requests through chat and a live voice service, normalize them into the central workflow, and return human-readable results.
Chat trigger · Call Jarvis voice service
Master Agent
Understands intent, chooses a specialist, passes the minimum useful context, and assembles the final response without directly owning every provider action.
n8n AI Agent · routing prompt · conversation memory
Email Agent
Handles bounded inbox search, thread retrieval, triage, drafting, and approved email actions through Gmail tools.
Gmail integration · scoped tool actions
Calendar Agent
Reads availability, investigates calendar state, and performs explicitly requested scheduling actions through Google Calendar.
Google Calendar integration · date/time normalization
Meeting Agent
Retrieves Fireflies meeting records and transcripts and converts them into summaries, decisions, and follow-up context.
Fireflies integration · transcript processing
Workflow and runtime layer
Persists orchestration, invokes tools, carries results between agents, exposes execution state, and runs on a private Docker/Linux host.
n8n · Docker · Linux · OpenClaw/Hermes profiles
Execution model
End-to-end execution
- 01
Receive the operator request
A chat or voice surface converts the request into a normalized message for the Jarvis workflow.
- 02
Interpret and route
The Master Agent identifies whether the job belongs to email, calendar, meeting intelligence, or a combination and selects the required specialist contract.
- 03
Invoke bounded tools
The chosen specialist calls only its configured provider tools and returns structured results or a clear failure to the master workflow.
- 04
Coordinate multi-domain work
For requests such as preparing for a meeting, the master can combine calendar context, prior email, and meeting records without collapsing their credentials into one tool surface.
- 05
Return and remember
Jarvis composes a concise result for the original surface and retains useful conversational state for follow-up requests.
- 06
Inspect operational failures
n8n execution history and service state provide a concrete place to diagnose provider authentication, schema, tool, or routing failures.
Implementation
What Harshitha implemented
- Created the central message-trigger-to-Master-Agent workflow and specialist routing contracts.
- Separated Email, Calendar, and Meeting agents so each receives a narrower prompt, provider connection, and set of available actions.
- Connected Gmail for inbox work, Google Calendar for scheduling context, and Fireflies for meeting intelligence.
- Carried tool results back through the master workflow so multi-domain requests can be composed into one operator-facing response.
- Deployed and operates the n8n runtime on a private Docker host with service-level checks and recoverable workflow definitions.
- Added a live Call Jarvis service as a voice entry point while retaining the same downstream orchestration boundary.
- Maintains OpenClaw and Hermes profiles as part of the broader agent runtime environment and experiments with new tools against the private stack before exposing them elsewhere.
Contribution summary
- Designed and operates the master-agent plus specialist-agent topology for real email, calendar, and meeting work.
- Mapped each specialist to bounded provider tools and kept routing responsibility separate from domain execution.
- Built the n8n workflow layer that carries messages, tool requests, results, memory, and error behavior across the system.
- Added and operates a live voice entry service alongside chat while preserving the same central agent and downstream tool boundaries.
- Uses the running system as an operational test bed for permissions, prompt contracts, integrations, recovery, and agent observability.
Failure design
Reliability engineering
R01
Risk
One prompt receives unrestricted access to every personal system.
Control
Specialist agents expose domain-specific tools while the Master Agent owns routing and response composition.
R02
Risk
A provider schema or authentication change fails silently.
Control
n8n execution history, explicit tool nodes, and service checks expose the failing integration and its returned payload.
R03
Risk
A voice request bypasses the established agent controls.
Control
Voice is treated as an input adapter to the central workflow, not as a second independent operations brain.
R04
Risk
Context from one domain is over-shared with another.
Control
The master passes task-relevant context to the selected specialist instead of granting every domain the full conversation and all tools.
R05
Risk
A container or workflow process restarts mid-operation.
Control
The private Docker runtime, n8n persistence, and execution records separate service recovery from provider-side action review.
Tradeoffs
Technical decisions
Keep one master agent and multiple specialists
A single entry point is convenient for the operator; narrower execution agents are easier to prompt, permission, test, and debug.
Preserve n8n as the orchestration layer
The existing workflow makes routing, provider calls, payload shapes, and failures inspectable without hiding all behavior inside model code.
Add voice as an adapter, not a redesign
Keeping the Master Agent as the brain prevents chat and voice from diverging into two systems with different tools and safety behavior.
Run the stack privately
Email, calendar, and meeting data require a controlled host and explicit credential boundary rather than a public demo environment.
Credibility
Evidence and scope
Verified workflow topology
The inspected Jarvis workflow routes a chat trigger into one Master Agent with Email, Calendar, and Meeting specialists backed by Gmail, Google Calendar, Fireflies, and memory.
Running private services
The current host runs the n8n stack and an active Call Jarvis service alongside OpenClaw/Hermes runtime profiles.
Deliberate private boundary
No public demo is linked because the system holds personal communication, scheduling, and meeting context.
Attribution boundary
Harshitha designed and operates the Jarvis agent topology, workflow behavior, provider integrations, and live private runtime described here. The verified current voice surface is the Call Jarvis service; proposed third-party voice connectors are not presented as deployed production components.
Technology