Mimir

A privacy-first personal AI assistant — Telegram interface, local LLMs, production-grade observability.

---

Mimir is a local-first conversational AI assistant built in Go. It runs on your machine, uses Ollama for LLM inference, stores everything in PostgreSQL, and communicates through Telegram. No conversation data leaves your machine.

Beyond basic chat, Mimir handles real-world tasks through a tool-calling layer: querying your Google Calendar, analyzing bank statements imported from CSV exports, and proactively sending financial summaries on a schedule. Long-term conversational memory is provided by a RAG pipeline backed by pgvector — semantic search, BM25 keyword matching, and recency scoring combined into a single hybrid retrieval step.

The project is built as a modular monolith with strict package boundaries (no import cycles), complete observability (Prometheus metrics, structured logs shipped to Loki, OpenTelemetry traces), and tested interfaces throughout.

---

Table of Contents

• Architecture
• Features
• Project Structure
• Quick Start
• Capabilities
  • Conversation Memory (RAG)
  • Tool Calling via MCP
  • Finance Tracking
  • Proactive Finance Agent
• Configuration
• Observability
• Testing
• Security Model
• Roadmap

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        Mimir (Go Binary)                        │
│                                                                 │
│  ┌────────────────┐    ┌──────────────────────────────────────┐ │
│  │ Telegram Poller│───▶│            Orchestrator              │ │
│  │  (long poll)   │◀───│  RAG Pipeline · MCP Manager · LLM    │ │
│  └────────────────┘    └───────────────┬──────────────────────┘ │
│                                        │                        │
│            ┌───────────────────────────┼─────────────────────┐  │
│            ▼                           ▼                     ▼  │
│  ┌─────────────────┐   ┌───────────────────────┐  ┌────────────┐│
│  │  Ollama (LLM)   │   │  PostgreSQL+pgvector  │  │ MCP Manager││
│  │  Tool Calling   │   │  messages · embeddings│  │ Calendar   ││
│  └─────────────────┘   │  finance transactions │  │ Finance MCP││
│                        └───────────────────────┘  └────────────┘│
│                                   ▲                             │
│  ┌─────────────────┐   ┌──────────┴─────────┐                   │
│  │   Scheduler     │   │   Finance Syncer   │                   │
│  │ Weekly/Monthly  │   │  CSV → DB · Alerts │                   │
│  │   LLM Digest    │   │  (~/finanças/)     │                   │
│  └─────────────────┘   └────────────────────┘                   │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│  Observability Stack (Docker Compose)                           │
│  App :9090 ──▶ Prometheus :9091 ──▶ Grafana :3000               │
│  slog JSON ──▶ Promtail ──▶ Loki :3100 ──▶ Grafana              │
└─────────────────────────────────────────────────────────────────┘

Design Principles

Principle	Implementation
No inbound connections	Telegram long polling only — no public ports
LLM isolation	Ollama runs locally with no internet access
Deterministic tool execution	LLM decides which tool; handler executes it deterministically
No cyclic imports	Strict dependency flow: main → orchestrator → {llm, rag, mcp, telegram}
Graceful degradation	Circuit breakers on Ollama (3 failures) and Telegram (5 failures)
No raw personal data in logs	Correlation IDs (chat_id-message_id) on every log entry

---

Features

Feature	Status	Notes
Telegram chat interface	✅	Long polling, allowlist enforcement, circuit breaker
Conversation persistence	✅	PostgreSQL, configurable history window
Ollama LLM integration	✅	Tool calling verified at startup; model-swappable
RAG long-term memory	✅	Hybrid search: pgvector + FTS + recency scoring
Embedding backfill utility	✅	cmd/backfill-embeddings — processes historical messages
Google Calendar (MCP)	✅	List, search, create events via OAuth 2.0
Finance tracking	✅	Itaú CC, Itaú bank, Nubank, PicPay CSV parsers
Finance categorizer	✅	3-stage: user overrides → rules → LLM fallback with DB caching
Finance MCP tools	✅	query_spending, list_transactions, fix_transaction_category
Proactive finance agent	✅	Weekly/monthly summaries pushed to Telegram
Full observability	✅	Prometheus, Grafana, Loki, OpenTelemetry, pre-provisioned dashboards
Multi-provider LLM	🚧	OpenAI, Anthropic, Gemini via config.yaml (in development)
Google Drive RAG	📋	Planned — document search via daily sync

---

Project Structure

mimir/
├── cmd/
│   ├── assistant/              # Main binary — wires and starts all components
│   ├── backfill-embeddings/    # One-shot: generate embeddings for existing messages
│   └── finance-mcp/            # MCP server binary — exposes finance tools over stdio
│
├── internal/
│   ├── config/                 # Env var loading and validation
│   ├── telegram/               # Bot client + long-poll loop
│   ├── orchestrator/           # Full message lifecycle (receive → LLM → reply)
│   │
│   ├── llm/                    # ChatProvider interface + Ollama implementation
│   │   └── ollama/             # HTTP client, circuit breaker, tool call parsing
│   │
│   ├── rag/                    # RAG pipeline: search, context assembly, token budgeting
│   ├── embeddings/             # Embedding generation (Ollama) + retry wrapper
│   ├── chunker/                # Recursive character splitter for long messages
│   │
│   ├── store/                  # PostgreSQL access layer
│   │   └── migrations/         # 003 SQL migrations (chat, embeddings, finance)
│   │
│   ├── mcp/                    # JSON-RPC 2.0 stdio client (MCP protocol)
│   │   └── tools/              # calendar.go — Google Calendar tool wrapper
│   │
│   ├── mcpmanager/             # MCP subprocess lifecycle + tool routing
│   │   ├── manager.go          # Spawn/teardown MCP subprocesses
│   │   ├── tools.go            # Expose tool schemas to the LLM
│   │   └── executor.go         # Route tool calls to the right handler
│   │
│   ├── finance/                # Bank statement pipeline
│   │   ├── model.go            # Domain types (Transaction, Account, Category)
│   │   ├── categorizer.go      # 3-stage categorizer (overrides → rules → LLM)
│   │   ├── syncer.go           # File watcher + import loop
│   │   └── parser/             # CSV parsers: itau_cc, itau_bank, nubank, picpay
│   │
│   ├── scheduler/              # Cron-like job runner
│   │   └── jobs/               # finance_agent.go (weekly + monthly digests)
│   │
│   └── observability/          # slog logger, Prometheus metrics, OTEL tracing
│
├── observability/              # Prometheus, Grafana, Loki, Promtail configs
│   └── grafana/provisioning/  # Auto-provisioned datasources + dashboards
│
├── docs/                       # Architecture documentation
├── docker-compose.yml
├── Makefile
└── .env.example

Database Schema

Three SQL migrations, applied automatically at startup:

Migration	Tables	Purpose
001_initial.sql	chat_sessions, messages	Conversation persistence
002_add_embeddings.sql	message_embeddings	pgvector (768-dim), HNSW index; messages.content_tsv for FTS (GIN index)
003_financial_tables.sql	financial_transactions, finance_merchant_categories	Finance pipeline with raw_hash deduplication

---

Quick Start

Prerequisites

Tool	Purpose
Go 1.24.3+	Build and run the binary
Docker	PostgreSQL + observability stack
Ollama	Local LLM runtime
Telegram bot token	Create via @BotFather
Your Telegram user ID	Get from @userinfobot

1. Clone and configure

git clone https://github.com/fandangolas/mimir.git
cd mimir
cp .env.example .env
# Edit .env: set TELEGRAM_BOT_TOKEN, DATABASE_URL, ALLOWED_USER_IDS

2. Pull a model with tool calling support

ollama pull llama3.1
# Alternatives: qwen2.5, llama3.2, mistral-nemo

Mimir verifies tool calling support at startup and exits with a clear error if the model doesn't support it.

3. Start infrastructure

make up
# Postgres :5432 · Prometheus :9091 · Grafana :3000 · Loki :3100

4. Run

make run
# Or: make build && ./bin/assistant

5. Verify

• Grafana: http://localhost:3000 (admin / admin) — open the Mimir dashboard
• Send a message to your Telegram bot

---

Capabilities

Conversation Memory (RAG)

By default, Mimir holds a sliding window of the last 20 messages. Enable RAG to give it true long-term memory backed by vector similarity search.

How it works:

1. Every message is embedded asynchronously (non-blocking) after it is stored
2. On each query, the user's message is embedded and a hybrid search runs:
   • Semantic (pgvector cosine similarity, weight 0.5) — finds conceptually similar messages
   • Keyword (PostgreSQL FTS with tsvector, weight 0.3) — exact term matching
   • Recency (exponential decay over 30 days, weight 0.2) — recent messages score higher
3. Retrieved messages are deduplicated against the recent-window slice
4. A context token budget caps total context length before sending to the LLM

Enabling:

# Pull the embedding model
ollama pull nomic-embed-text

# Backfill existing chat history (one-time)
make build && ./bin/backfill-embeddings

# Enable in .env
RAG_ENABLED=true

make run

Storage: ~200 MB for 5 years of daily conversations (768-dim float32 vectors, HNSW index).

See docs/rag-architecture.md for implementation details.

---

Tool Calling via MCP

Tools are exposed to the LLM through the Model Context Protocol — a JSON-RPC 2.0 protocol over stdio. Each tool server runs as a subprocess; the mcpmanager handles process lifecycle, and the orchestrator routes tool call results back into the LLM conversation loop.

Orchestrator
  └── ChatWithTools() → LLM responds with tool_calls[]
       └── mcpmanager.ExecuteToolCall()
            ├── list_calendar_events  → calendar subprocess
            ├── create_calendar_event → calendar subprocess
            ├── query_spending        → finance-mcp subprocess
            └── list_transactions     → finance-mcp subprocess
       └── Tool results appended to messages → LLM called again
           (up to 5 rounds to prevent infinite loops)

Google Calendar setup: docs/google-calendar-setup.md

MCP_ENABLED=true
GOOGLE_OAUTH_CREDENTIALS=/path/to/gcp-oauth.keys.json

Available calendar tools: list_calendar_events, create_calendar_event, search_calendar_events, get_current_time.

---

Finance Tracking

Mimir imports bank statement CSV files exported from Brazilian banks (Itaú CC, Itaú bank account, Nubank, PicPay). The syncer runs on a configurable interval, parses each file, categorizes transactions, and upserts them into PostgreSQL using a raw_hash for idempotent imports.

Categorization (3-stage pipeline):

1. User overrides — finance_merchant_categories rows with source='user', applied retroactively to all past and future transactions matching the keyword
2. Rule-based — 50+ patterns covering Brazilian merchants (Uber, iFood, Guanabara, PetLove, etc.)
3. LLM fallback — Uncategorized merchants are sent to Ollama with a Portuguese prompt; results cached in DB with source='llm'

Finance MCP tools (available to the LLM):

Tool	Description
query_spending	Spending totals grouped by category for a date range
list_transactions	Individual transactions with optional filters
fix_transaction_category	Save a merchant override + retroactively update all matching rows
get_spending_alerts	Categories with spending >20% above 3-month average

The system prompt is pre-populated with concrete date ranges (today, this month, last month, last 3/6 months) so the model never has to guess the current date or call get_current_time before running a finance query.

FINANCE_ENABLED=true
FINANCE_DIR=~/Documents/finanças  # Directory containing exported CSV files
FINANCE_SYNC_INTERVAL_HOURS=6

---

Proactive Finance Agent

When FINANCE_AGENT_ENABLED=true, Mimir pushes unprompted financial summaries to your Telegram:

• Weekly (Monday 09:00): Current week's spending vs. 3-month category averages; anomalies flagged; late fees listed
• Monthly (1st of month, 08:00): Previous month vs. month before, per-category delta and %, top 5 one-off expenses, loan repayments

Data is assembled from the DB, formatted into plain text, and passed to the LLM with a Portuguese counselor prompt. The LLM writes a 6-sentence narrative — no structured output required, which avoids issues with quantized models that struggle with constrained generation.

Late-fee transactions (multa, juros) trigger an immediate Telegram alert when detected during any sync run, outside the scheduled jobs.

FINANCE_AGENT_ENABLED=true
FINANCE_AGENT_CHAT_ID=0  # 0 = first user in ALLOWED_USER_IDS

---

Configuration

All configuration is loaded from .env (via godotenv) or system environment variables.

Required

Variable	Description
TELEGRAM_BOT_TOKEN	Bot token from @BotFather
ALLOWED_USER_IDS	Comma-separated Telegram user IDs allowed to interact
DATABASE_URL	PostgreSQL connection string

LLM

Variable	Default	Description
OLLAMA_BASE_URL	http://localhost:11434	Ollama API endpoint
OLLAMA_MODEL	llama3.1	Model name — must support native tool calling
CONVERSATION_WINDOW	20	Recent messages sent to LLM in each request

RAG

Variable	Default	Description
RAG_ENABLED	false	Enable hybrid retrieval for long-term memory
EMBEDDING_MODEL	nomic-embed-text	Ollama embedding model (768 dimensions)
RAG_RETRIEVED_COUNT	5	Messages retrieved from history via hybrid search
RAG_RECENT_COUNT	10	Most recent messages always included
RAG_MIN_SIMILARITY	0.5	Minimum cosine similarity for vector results
RAG_WEIGHT_SEMANTIC	0.5	Semantic score weight (must sum to 1.0 with keyword + recency)
RAG_WEIGHT_KEYWORD	0.3	FTS score weight
RAG_WEIGHT_RECENCY	0.2	Recency decay weight
RAG_RECENCY_DECAY_DAYS	30	Recency half-life in days

MCP / Google Calendar

Variable	Default	Description
MCP_ENABLED	false	Enable MCP tool calling (Calendar + Finance tools)
GOOGLE_OAUTH_CREDENTIALS	—	Path to Google Cloud OAuth client credentials

Finance

Variable	Default	Description
FINANCE_ENABLED	false	Enable finance syncer and MCP tools
FINANCE_DIR	~/Documents/finanças	Directory containing bank CSV exports
FINANCE_SYNC_INTERVAL_HOURS	6	How often to scan for new files
FINANCE_AGENT_ENABLED	false	Enable weekly/monthly Telegram digests
FINANCE_AGENT_CHAT_ID	0	Target chat ID (0 = first in ALLOWED_USER_IDS)

Application

Variable	Default	Description
LOG_LEVEL	info	debug, info, warn, error
LOG_DIR	logs	Daily log files tailed by Promtail
METRICS_ADDR	:9090	Prometheus scrape endpoint

---

Observability

Prometheus Metrics

Metric	Labels	Description
assistant_messages_received_total	—	All incoming Telegram messages
assistant_messages_processed_total	status	Processed messages (success / error)
assistant_llm_duration_seconds	—	Histogram of LLM response latency
assistant_telegram_send_errors_total	—	Failed Telegram sends
assistant_unauthorized_messages_total	—	Messages dropped by allowlist
assistant_rag_searches_total	result	RAG queries (success / error / fallback)
assistant_rag_search_duration_seconds	—	Hybrid search latency histogram
assistant_rag_context_tokens	—	Tokens in assembled context per request
assistant_embedding_generation_duration_seconds	—	Embedding latency histogram
assistant_embedding_errors_total	—	Failed embedding calls

Grafana

Open http://localhost:3000 (admin / admin). The Mimir dashboard is auto-provisioned and includes:

• Message throughput and error rates
• LLM latency heatmap (p50/p95/p99)
• RAG search quality and latency (when enabled)
• Embedding generation throughput

Loki / Structured Logging

Every log line is JSON via slog. Query in Grafana Explore:

# All logs
{job="mimir"}

# Filter by chat session
{job="mimir"} | json | correlation_id = "123456-789"

# Errors only
{job="mimir"} | json | level = "ERROR"

Each log entry carries correlation_id (chat_id-message_id), chat_id, and contextual fields for the current operation.

OpenTelemetry

Traces are emitted for the orchestrator's handle span and LLM calls. The exporter is stdout by default; swap to an OTLP endpoint for persistent traces.

---

Testing

make test          # go test ./...
make lint          # golangci-lint run

Test coverage by package:

Package	What's tested
config	Env var parsing, required field validation
llm/ollama	Tool calling model detection, circuit breaker
embeddings	Embedding generation, retry/backoff logic
chunker	Recursive character splitting, boundary conditions
rag	Context assembly, token budgeting, deduplication
mcp	JSON-RPC 2.0 encoding/decoding, subprocess protocol
mcp/tools	Calendar tool parameter translation
orchestrator	Full message flow with mock ChatProvider

The ChatProvider interface makes the orchestrator fully testable without Ollama or a real LLM — tests pass a mock that returns controlled responses and records all calls for assertion.

---

Security Model

Access control: Only Telegram users listed in ALLOWED_USER_IDS can interact. Unauthorized messages are silently dropped, logged, and counted in Prometheus (assistant_unauthorized_messages_total). No error is returned to the sender.

Data flow: Telegram uses long polling — Mimir initiates all outbound connections. No inbound connections, no public ports, no webhooks.

LLM trust boundary: The LLM decides which tool to call and with which arguments. Tool handlers validate and execute deterministically. The LLM cannot construct queries directly or access data outside the tool interface. The worst outcome of a misbehaving model is a bad sentence or a wrong tool argument — never unauthorized data access or an irreversible action.

Credentials: OAuth tokens for Google services are stored encrypted at rest. API keys and tokens are loaded from environment only — never logged.

Finance data: Bank transaction data never leaves the machine. The finance MCP binary runs as a subprocess, communicates over stdio, and has read-only database access by convention.

---

Available Make Targets

make build   # Build bin/assistant and bin/finance-mcp
make run     # Build and run the assistant
make test    # Run all tests
make lint    # golangci-lint
make up      # docker compose up -d (Postgres + observability)
make down    # docker compose down

---

Roadmap

Phase	Status	Deliverables
Core (P1-P2)	✅ Complete	Telegram bot, Ollama integration, conversation persistence, full observability stack
RAG	✅ Complete	pgvector hybrid search, HNSW indexes, embedding backfill, token budgeting
Integrations	✅ Complete	Google Calendar via MCP, finance tracking (4 banks), proactive finance agent
Multi-LLM	🚧 In Progress	OpenAI, Anthropic, Gemini providers behind a common interface; config.yaml for provider selection
Google Drive RAG	📋 Planned	Index Drive documents for retrieval alongside conversation history
Cloud deployment	📋 Planned	Always-on hosting on a home server or private VPS

---

Why "Mimir"?

In Norse mythology, Mimir was renowned for his knowledge and wisdom — Odin's counselor and the keeper of the well of wisdom beneath the World Tree. In God of War (2018), he's Kratos's constant companion: a disembodied head carried everywhere, offering guidance and insight.

The name fits a local assistant that lives on your machine, remembers your conversations over time, and knows your calendar, spending, and habits — all without sharing any of it with a cloud service.

---

License

MIT License. See LICENSE for details.

---

Built With

• Ollama — Local LLM runtime
• pgvector — Vector similarity search for PostgreSQL
• Prometheus + Grafana — Metrics and dashboards
• Loki + Promtail — Log aggregation
• OpenTelemetry — Distributed tracing
• gobreaker — Circuit breaker pattern
• pgx/v5 — PostgreSQL driver
• Telegram Bot API — Chat interface