AI Document Q&A Agent with Pinecone & OpenAI

Unlock Instant Document Insights with this AI Agent

This AI Agent transforms your documents into an interactive conversational experience. It's designed for anyone who needs to quickly extract information or understand complex documents without tedious manual reading. Simply provide a document (e.g., from Google Drive), and this agent will process it, enabling you to ask questions and receive precise, AI-generated answers complete with source citations.

At its core, this agent implements a Retrieval Augmented Generation (RAG) pipeline. It first ingests your document, breaks it down into manageable chunks using a text splitter, and then generates vector embeddings for each chunk using OpenAI's powerful models. These embeddings are stored in a Pinecone vector database, creating a searchable index of your document's content. When you ask a question via the chat interface, the agent converts your query into an embedding, searches Pinecone for the most relevant document chunks, and then feeds these chunks along with your original question to an OpenAI LLM to generate a coherent and contextually accurate answer. The agent also intelligently extracts and formats citations, pointing you back to the specific parts of the document used to formulate the response.

Key Features & Benefits

• Intelligent Document Ingestion: Fetches documents from sources like Google Drive and prepares them for AI processing.
• Advanced Text Chunking & Embedding: Utilizes recursive character text splitting and OpenAI embeddings to accurately represent document content for semantic search.
• Efficient Vector Storage & Retrieval: Leverages Pinecone for high-speed, scalable vector search, ensuring quick access to relevant information.
• AI-Powered Conversational Q&A: Employs OpenAI's chat models (e.g., GPT series) to understand natural language queries and generate human-like answers based on document context.
• Precise Source Citations: Automatically generates citations for answers, linking back to the specific chunk and metadata (file name, line numbers) from the source document for transparency and verification.
• Interactive Chat Interface: Simple n8n chat trigger allows for easy interaction and querying of your indexed documents.
• RAG Pipeline Automation: Fully automates the complex RAG process, making sophisticated document Q&A accessible.

• An n8n instance (Cloud or self-hosted).
• OpenAI API Key with access to an embedding model (e.g., text-embedding-ada-002) and a chat model (e.g., gpt-3.5-turbo, gpt-4).
• Pinecone API Key and an existing Pinecone index. The index MUST be created with 1536 dimensions to match OpenAI's text-embedding-ada-002 model.
• Google Drive credentials and a publicly accessible file URL if using the default Google Drive setup.

1. Download the n8n workflow JSON file.
2. Import the workflow into your n8n instance.
3. Configure Credentials: a. Set up OpenAI credentials in the 'Embeddings OpenAI', 'Embeddings OpenAI2', and 'OpenAI Chat Model' nodes. b. Set up Pinecone credentials in the 'Add to Pinecone vector store' and 'Get top chunks matching query' nodes. Select your pre-configured Pinecone index (1536 dimensions). c. Configure Google Drive credentials in the 'Download file' node if you plan to fetch files from Google Drive.
4. **Document Indexing (Run Once Per Document): ** a. In the 'Set file URL in Google Drive' node, update the value field with the direct URL to your document (e.g., the Bitcoin whitepaper URL provided is a good test case: https://drive.google.com/file/d/11Koq9q53nkk0F5Y8eZgaWJUVR03I4-MM/view). Ensure the link is publicly accessible or adjust the 'Download file' node for private files. b. Activate the workflow if it's not already active. c. Manually trigger the workflow by clicking the 'Execute Workflow' button on the 'When clicking "Execute Workflow"' node. This will download, process, chunk, embed, and store your document in Pinecone. This step only needs to be done once per new document.
5. Chat with Your Document: a. Once indexing is complete, open the 'Chat Trigger' node. b. Click the 'Chat' button that appears in the node's parameters panel. c. A chat interface will open. Type your question about the indexed document (e.g., "Which email provider does the creator of Bitcoin use?" if you used the sample document). d. The AI Agent will retrieve relevant information and provide an answer with citations.