Designing a private document assistant

The context

An organisation in the healthcare sector wanted to make its IT team's internal knowledge easier to consult, hosted within its own environment.

The work

Designed a document assistant combining local models, vector search and document filtering based on Active Directory permissions. Supervised development, testing and technical validation.

The API uses Python and FastAPI, with a React interface, Qdrant for vector search and locally served Mistral models. The interaction history retains metadata for three months.

The result

The first part, covering IT knowledge, has been delivered. The project remains in development; the envisaged medical uses were not part of this delivery.

Environment: Python, FastAPI, React, Qdrant, PostgreSQL, vLLM, Mistral, Docker, Active Directory.