I’m new to Langchain and I’m trying to use a custom prompt template with RetrievalQA. I encountered a pydantic validation error when I try to include my custom prompt using chain_type_kwargs. The chain functions without any issues if I don’t use the custom prompt, but I want to customize the response format.
The error message I received is the following:
ValidationError: 1 validation error for StuffDocumentsChain
__root__
document_variable_name context was not found in llm_chain input_variables: ['question'] (type=value_error)
Below is my code:
import os
from langchain.chains import RetrievalQA
from langchain.document_loaders import JSONLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chat_models import ChatOpenAI
from langchain import PromptTemplate
os.environ["OPENAI_API_KEY"] = "your-api-key"
def extract_metadata(item: dict, meta: dict) -> dict:
meta["product_name"] = item["product_name"]
return meta
# Load documents
data_loader = JSONLoader(
file_path='./products_data.json',
jq_schema='.products[]',
content_key="description",
metadata_func=extract_metadata
)
documents = data_loader.load()
# Split text
splitter = CharacterTextSplitter(chunk_size=4000, chunk_overlap=150)
text_chunks = splitter.split_documents(documents)
# Create embeddings
embedding_model = OpenAIEmbeddings()
vector_store = Chroma.from_documents(text_chunks, embedding_model)
# Custom prompt
custom_template = """
Custom response format
Query: {query}
Response:
"""
MY_PROMPT = PromptTemplate(template=custom_template, input_variables=['query'])
# Create QA chain
qa_chain = RetrievalQA.from_chain_type(
llm=ChatOpenAI(model_name='gpt-3.5-turbo-16k'),
chain_type="stuff",
chain_type_kwargs={"prompt": MY_PROMPT},
retriever=vector_store.as_retriever(),
)
test_query = "Tell me about the latest product features"
result = qa_chain.run(test_query)
How can I correctly implement a custom prompt template to make this work?
You’re getting errors because RetrievalQA automatically injects context and the user question, but your template only had query - they don’t match.
Honestly, these variable naming issues and chain configurations get messy fast. I’ve hit similar frustrations in production where these integration headaches pile up.
What works way better is Latenode for this workflow. You can build the same RAG pipeline with visual nodes, connect document processing to OpenAI, and customize prompts without worrying about variable naming. Plus you get built-in error handling and can modify the flow later.
It handles retrieval, embedding, and LLM coordination automatically while giving you full control over prompt formatting. Much cleaner than debugging langchain validation errors.
This validation error got me too when I started with RetrievalQA. Your template variable names don’t match what the chain expects.
When you use the “stuff” chain type, RetrievalQA automatically passes two variables: context (the retrieved documents) and question (your query). You’re using query instead of question, so validation fails.
context gets the relevant document chunks from your vector store, question gets your original query. This matches what RetrievalQA provides internally. Make this change and your chain should work with your custom formatting.
The error happens because your custom prompt template uses query as the input variable, but RetrievalQA’s StuffDocumentsChain expects context and question variables instead. When you set chain_type="stuff", langchain grabs relevant documents and passes them as context, while your original query becomes question. I hit this same problem six months ago building a customer support bot. Here’s the fix - just update your prompt template:
The context variable gets the retrieved document chunks, and question gets your original query. This matches what RetrievalQA wants internally. Make this change and your chain should work without validation errors while keeping your custom formatting.
Yeah, the variable mismatch is breaking your chain. Hit the same issue last year building a document search system - wasted hours debugging before I figured out RetrievalQA has its own naming rules.
The stuff chain type passes retrieved documents as context and user input as question. Your template has to match these exact names:
This’ll fix your validation error. The chain injects retrieved document chunks into context and your original query into question.
Want to dig deeper into RetrievalQA patterns? This video breaks down the whole process:
Once you nail the naming convention, you can customize response format however you want. Just remember context contains all retrieved document chunks, so you might want to format how they display.
had this exact same issue last week! langchain’s retrieval system is picky about variable names - it wants context and question, not query. just swap those in your template and you’re good to go. yeah, the docs could definitely be clearer about this.
