Handling Concurrent Document State

We maintain document state through a centralized registry. This ensures that when a user interacts with one file, the system does not lose track of others in the session. Keeping state consistent across varied input types is the primary challenge for a study assistant. We approach this by decoupling the raw document content from its vectorized embedding. The raw text resides in a local cache for quick retrieval, while the vector store handles semantic queries across the entire corpus. When a user queries across multiple documents, the search service performs a weighted aggregation. This prevents data from a single document from dominating the result set. We assign a metadata tag to every chunk. This tag identifies the source file, allowing the assistant to cite specific documents even during broad synthesis tasks. If two documents share similar content, our deduplication logic checks the hash of the text chunks before they enter the vector database. We prioritize keeping the context window clean over increasing search granularity. This trade-off reduces the likelihood of the model hallucinating content by conflating disparate sources. The registry also monitors the active window of the assistant. It prunes chunks that have fallen out of the top-k relevance window as the conversation progresses. This dynamic pruning happens on the client side to avoid unnecessary round-trips to the server. The client keeps a local shadow copy of the document registry. This ensures the interface remains responsive during document uploads or removals. If a user deletes a file, the client sends a reconciliation command to the backend. The backend updates the vector store indices and clears the corresponding cache entries. Consistency is not binary, so we treat it as a continuous reconciliation process. Our current architecture handles up to twenty simultaneous documents per session. Beyond this, we throttle indexing to prevent memory spikes in the browser environment. We prefer a stable, slower experience over one that crashes due to heap exhaustion. The integration of document metadata allows us to build features like source-specific grounding. When an answer is generated, the backend returns a list of source references alongside the text. This allows users to verify claims by clicking through to the specific passage. We avoid storing user documents in persistent storage beyond the duration of a session unless the user explicitly saves the project. This limits our liability and simplifies the state cleanup process. We handle state conflicts by assigning a version identifier to every document node. When a state update arrives, the client compares the version ID. If the local version is newer, the incoming update is discarded. This prevents out-of-order responses from corrupting the document registry. Our current focus is refining the chunking strategy to better handle mixed-format documents. Different file types require different parsing logic before reaching the vector store. We are moving toward a unified parsing pipeline that normalizes input before serialization. This will reduce the complexity of our document registry management.