AgentOps AI
Connect questions to tools and organization documents.
Shows how an internal assistant can combine controlled tools, organization-scoped data, and inspectable execution history.
The problem
Internal questions can involve document text, exact arithmetic, and operational figures. A language model alone is not an authoritative source for any of those records.
The approach
A FastAPI service coordinates a bounded agent loop with calculator, predefined analytics, and document-retrieval tools. PostgreSQL stores conversations and runs; pgvector supports document retrieval; React streams progress and answers.
- Organization registration, authentication, and distinct admin/user roles
- User chat with stored history and server-sent events
- API-based document upload, chunking, and embeddings
- Admin run timelines and citations restricted to retrieved documents
Follow the flow.
Select a stage to explore its role. This diagram explains the documented design; it is not live infrastructure.
Authenticated request
The server loads the caller and their organization from the token-backed user record.
The decisions behind it.
Enforce organization boundaries on the server
The organization is derived from the authenticated user, not from an identifier supplied by the model or request body.
Constrain operational queries
The analytics tool exposes a closed set of operations. The calculator parses expressions instead of asking the model to perform arithmetic in prose.
Record agent behavior
Persisted runs and tool calls support inspection. Redis stores rate-limit counters, not agent state or a second copy of application data.
What the evidence supports.
The repository documents mocked model tests, API/service/database tests, frontend tests, and a mocked Playwright chat flow. CI runs backend checks, frontend tests/build, and a backend image build; it does not deploy the application.
Current boundaries
- Document upload and user creation are API-only; the documents page remains a placeholder.
- Admin and user roles are disjoint, and admins currently land on a chat page the API will reject.
- Only OpenRouter is implemented; sample commerce rows and a deployment pipeline are not included.
What comes next.
- Finish role-aware navigation and document-management UI.
- Evaluate retrieval quality and answer grounding with a documented test set before presenting accuracy claims.