Architecture and services
A set of services behind one entry point. The API gateway owns the business logic, and specialist services handle data, documents and sandboxed work.
It runs as containers with Docker Compose.
Overview#
Figure 1. Everything enters through one proxy. Only the API gateway talks to the data stores and the document service.
Core services#
| Service | Role |
|---|---|
| Web console | The screens people use (Next.js) |
| API gateway | All business logic: sign-in, connectors, canonicalization, APIs, MCP, approvals, evaluation and AI features (Spring Boot) |
| Reverse proxy | The only public entry point (nginx) |
Data plane#
| Store | Holds |
|---|---|
| PostgreSQL | Users, workspaces, settings, audit, and unified records with lineage. Used in production; local development uses a single-file database. |
| Apache Jena Fuseki | The ontology and each workspace's knowledge fabric |
| Neo4j | The record store in production, and access policies |
| Lakehouse (DuckDB and Spark) | An alternative record store in Parquet files. It files records by system and object, not by workspace. |
| MinIO | Uploaded files and document originals |
| Apache Kafka | Change events for live updates. A single server can use an in-memory bus instead. |
Document intelligence#
| Service | Role |
|---|---|
| Document service | Reads, splits, indexes and answers from documents (Python) |
| Qdrant | Search index of document passages |
| FalkorDB | Graph of codes and terms that documents define or mention |
| Model server (our own GPU) | PaddleOCR-VL, Qwen3 reranker and Qwen3 embedder |
See Document intelligence for how they work together.
Sandboxed workers#
| Service | Role |
|---|---|
| Functions runner | Runs admin-written functions in isolated sandboxes |
| Insight worker (opt-in) | Computes numerical insights from approved plans. Runs beside the gateway. |
| Artifact exporter (optional) | Turns generated reports and slides into PDF or PowerPoint |
External services#
| Service | Used for | Needed when |
|---|---|---|
| Source systems | Reading and writing business data | A system is connected |
| Keycloak | Single sign-on | Single sign-on is turned on |
| Language model provider | AI features and document answers | AI features are used; can be self-hosted |
| OpenAI embeddings | Search vectors for documents | The default; self-hosted embeddings avoid it |
| GPU model server | OCR and reranking | Documents are added or searched; OCR falls back to Docling |
| Varnam component registry | Building Pulse and Metaspace screens | Those features are used |
| Microsoft Graph | Email sent by agents | Agent email is set up |
Design principles#
- One way in. Only the proxy is public. Every data store is reachable only from the server itself.
- The gateway owns the data. The console never talks to a data store directly.
- Degrade, don't fail. If an optional service is down, the rest keeps working and the affected feature says so. For example, if the document service is down, records and mappings keep working.
- Heavy work stays apart. Spark, model inference and user code run outside the gateway.