Architecture
How the system works under the hood.
This page explains the product architecture as implemented in the current codebase: how knowledge is ingested, how retrieval and streaming chat work, where human approvals and escalation fit in, and how tenant isolation is enforced.
System diagram
The product has three main layers: ingestion, runtime retrieval/chat, and control surfaces for approvals, escalation, auditing, and outbound integrations.
Prepare knowledge
Ingestion layer
Answer with sources
Retrieval + chat layer
Keep humans in the loop
Control layer
Tenant boundary
Every document, chunk, session, message, escalation, and webhook is associated with an org id.
Model boundary
Embeddings are generated with OpenAI, while the main streaming answer flow uses Anthropic in the current implementation.
Operational boundary
Sensitive or uncertain cases are routed through approval or escalation instead of being treated as unrestricted automation.
1. Knowledge ingestion and RAG preparation
The retrieval system starts by converting support knowledge into chunked, embedded records that can be searched per workspace. This is what makes the assistant grounded in customer-specific documents instead of relying on generic model memory.
- PDF and text uploads go through the document upload API, are parsed, chunked, embedded, and written into organization-scoped document and chunk tables.
- Public URLs can be fetched and converted into plain text before the same chunking and embedding pipeline runs.
- The dashboard also exposes help-doc style imports for supported tools such as Notion, Confluence, and other connected sources where those integrations are configured.
2. PDF, URL, and help-doc ingestion flow
The ingestion pipeline is intentionally simple: acquire content, normalize it to text, split it into coherent chunks, embed those chunks, and store them under the organization that owns the workspace.
PDF / URL / help-doc source -> text extraction -> semantic chunking with overlap -> embedding generation -> org-scoped chunk storage -> retrieval-ready knowledge base
3. Embeddings and retrieval layer
Embeddings are generated with OpenAI in the current implementation. Retrieval uses a Supabase RPC vector search against chunk embeddings, filtered by organization id so tenants do not search across each other's data.
- Single-query retrieval exists in the base RAG utility, but the main chat flow uses a richer agent retrieval path.
- The agent retrieval layer decomposes complex questions into 1-3 sub-queries, generates hypothetical answers for better search alignment, embeds those results, then performs parallel vector searches.
- Results are deduplicated and re-ranked by similarity before being turned into a grounded system prompt and user-visible source list.
4. Streaming chat flow
The runtime chat path is built as a streaming API rather than a request-then-block flow, which helps the interface feel responsive and also exposes more of the system reasoning state back to the UI.
- The chat API authenticates the workspace, checks monthly message quotas, and creates or resumes a chat session.
- Long conversation history can be compressed before retrieval to keep runtime context smaller and focused.
- The retrieval layer decomposes multi-part questions, generates HyDE-style hypothetical answers, embeds those search queries, and performs parallel vector search against organization-scoped chunks.
- The best chunks are deduplicated, ranked by similarity, converted into a system prompt, and sent into the streaming model response flow.
- The client receives a live Server-Sent Events stream with thinking steps, sources, confidence, token deltas, suggestions, and completion state.
5. Human approval workflow and safety controls
The product positioning is intentionally narrower than “fully autonomous support.” The implementation and public docs are centered on guardrailed automation with human review where risk is higher.
- Support teams define which support workflows are safe to automate and which should be escalated or require human review.
- Sensitive actions such as refunds, account changes, or billing updates are described publicly as human-approved flows unless they are explicitly automated in product code.
- Low-confidence retrieval results are labeled as such and are intended to push the experience toward cautious answers or human escalation instead of fabricated certainty.
6. Escalation logic
When the workflow should leave the automated path, the system creates a handoff record and notifies humans through the configured support stack.
- Escalations create organization-scoped records tied to the current chat session when available.
- Escalation notifications can fan out to configured destinations such as email, Slack, Zendesk, HubSpot, or Salesforce depending on the workspace settings.
- The escalation view in the dashboard gives humans the customer context, the conversation history, and the current status of the handoff.
7. Audit logs and operational visibility
This repo is stronger than a typical demo because it does not stop at generating answers. It also stores operational traces around what was answered, why, and what happened next.
- Audit records are stored in a dedicated audit log with organization id, user information when available, action, resource, metadata, and timestamp.
- Chat messages also persist sources, confidence, and sub-query metadata, which makes the runtime behavior more explainable than a plain black-box chat transcript.
- Agent analysis workflows such as gap analysis and FAQ generation are also persisted as structured records for later review.
8. Multi-tenant organization model
Support AI is architected as an organization-scoped SaaS. Workspace isolation is one of the core patterns repeated across the app and API surface.
- Organizations are the core tenant boundary. User profiles, subscriptions, document records, chat sessions, messages, escalations, audit records, settings, keys, and webhooks all attach to an org id.
- Authenticated dashboard and API routes resolve the current user through Supabase auth and then look up their workspace membership in user_profiles and org_members.
- Role checks are applied at the route layer so owner, admin, and member permissions can differ across settings, billing, webhooks, and audit surfaces.
9. API and webhook flow
The product exposes authenticated APIs for chat, documents, analytics, webhooks, and operational surfaces. Webhooks provide a clean way to push Support AI events into external systems.
- Outbound webhook endpoints are stored per organization and managed through authenticated dashboard APIs.
- Webhook test delivery signs payloads with an HMAC secret and sends event metadata plus JSON data to the customer endpoint.
- This makes it possible to plug Support AI events into external automation, alerting, or internal systems without coupling the core app to a single helpdesk.
10. What this means for buyers and engineering managers
The architecture is meant to be understandable: ingestion builds a workspace-specific knowledge base, retrieval grounds answers in that knowledge, streaming chat exposes runtime state, and control surfaces keep humans involved where needed.
For founders, that means faster rollout and a clearer trust story. For engineering managers, it means the system has visible tenant boundaries, explicit runtime flow, and concrete operational hooks instead of just a polished UI.