How to integrate custom JSON dataset with LangChain ChatOpenAI and ConversationChain for contextual responses

Other Questions langchain
1

I’m working with a custom JSON dataset that I generate from an Excel file stored in AWS. I need to use this data as context for my AI conversations while also maintaining conversation history through LangChain.

Currently, I can get responses based on my JSON data using the direct OpenAI API approach:

import boto3
import pandas as pd
import openai
from io import BytesIO

# Fetch Excel file from AWS S3
s3_client = boto3.client('s3')
file_obj = s3_client.get_object(Bucket='my-data-bucket', Key='employees.xlsx')

file_content = file_obj['Body'].read()
excel_data = pd.read_excel(BytesIO(file_content), sheet_name='employee_info')

json_records = excel_data.to_dict("records")

ai_response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[{
        "role": "system", 
        "content": f"{system_message}. \n\nData context: {json_records}. \n\nResponse:"
    }],
    temperature=0.7,
    max_tokens=400
)

This works fine for single questions, but I need conversation memory. When I try to implement LangChain’s ConversationChain, I can’t figure out how to pass my JSON dataset as context:

from langchain.chat_models import ChatOpenAI
from langchain.chains import ConversationChain

chat_model = ChatOpenAI(temperature=0.7, openai_api_key=my_api_key, model="gpt-4")
conversation_chain = ConversationChain(
    llm=chat_model,
    prompt=my_prompt_template,
    verbose=True,
    memory=conversation_memory,
    chain_type_kwargs=additional_params
)
ai_answer = conversation_chain.predict(input=user_question)

How can I incorporate my JSON dataset into the LangChain setup so that the AI can reference this data throughout the conversation while maintaining chat history?

2

The real problem isn’t just getting data into your prompt - you need a workflow that automatically handles Excel processing, data updates, and conversation management.

I built something similar for an employee query system. Instead of hardcoding everything, I created an automation flow that pulls from S3, processes the Excel file, and feeds fresh data to the conversation chain whenever needed.

Set it up as an automated pipeline where S3 changes trigger data processing. Your conversation chain always gets the latest context without you manually managing refreshes or dealing with stale information.

For LangChain integration, embed your JSON data directly into a custom memory class that keeps both conversation history and your dataset context. Every exchange gets access to your employee data without losing chat history.

The automation approach scales to multiple datasets, lets you schedule regular updates, and even add data validation steps. Way cleaner than managing all these moving pieces manually in your code.

3

To integrate your JSON dataset with LangChain, utilize a memory system alongside a custom prompt. Start by creating a ConversationBufferMemory that can track the conversation history. Modify your prompt to include your JSON data, ensuring both the dataset context and chat history are represented. When invoking the predict() method on your ConversationChain, pass the json_records alongside the memory. If your dataset is extensive, consider using ConversationSummaryMemory to avoid token limits while preserving context.

4

Skip ConversationChain and use LangChain’s RetrievalQA instead. Injecting JSON directly into prompts will hit token limits fast - especially with hundreds or thousands of employee records.

I converted my JSON to a vector store using FAISS or Chroma. Load the employee data, create embeddings, then use RetrievalQAWithSourcesChain. It automatically grabs relevant records based on the user’s question and keeps conversation context.

Retrieval’s way more efficient since it only pulls relevant employee records per query instead of dumping everything into each prompt. Combine it with ConversationBufferMemory to get data context and chat history without maxing out tokens.

For S3, just rebuild the vector store when the Excel file updates. Way more scalable than stuffing everything into system messages.

5

just modify your prompt template to include the json data as context. create a custom prompttemplate that injects your dataset into the system message, then pass it to conversationchain. try something like prompttemplate.from_template("context: {context}\n\nconversation:\n{history}\nuser: {input}\nai:") and format the context parameter with your json_records when you initialize it.