I built a chat application with React frontend and Flask backend. The bot gets evaluated through Langsmith but I need to pull those evaluation results into my web app dashboard.
Is there a way to fetch chatbot analytics data from Langsmith using REST endpoints? I want to show things like response accuracy and conversation quality on my UI.
I checked their docs but couldn’t locate API info for getting performance data out of their system.
check if langsmith has webhooks instead of constantly polling their api. we set up webhook endpoints in our flask app to automatically receive eval results when runs finish. way more efficient than repeated rest calls, and you get real-time dashboard updates without hammering their servers.
I added Langsmith metrics to a production dashboard 6 months ago. The official API docs are buried, but you can find the endpoints by checking their Python client source on GitHub - same REST endpoints they use internally. Hit /api/v1/projects/{project_id}/stats for aggregate metrics and POST to /api/v1/runs/query for detailed evaluation data. Stats gives you overview stuff like average response time and success rates. The runs query lets you filter by date ranges and evaluation types. Couple things to watch out for: Their timestamp filtering only uses UTC, so convert your local times first. Some metrics don’t populate until evaluation runs finish, so you’ll get empty responses if you query too fast after conversations end. For auth, just use Bearer token in headers with your API key. Response format stays pretty consistent once you figure out their schema.
Had this exact problem building internal dashboards. Langsmith API works but it’s more hassle than it’s worth.
Rate limiting’s all over the place and those nested JSON responses are brutal to parse. You’re also stuck maintaining auth logic and error handling.
Switched to Latenode for this. Set it to auto-pull Langsmith data on schedule, clean it up, and push straight to your dashboard database. Takes maybe 10 minutes to set up.
Best part? You can add other data sources later without touching Flask code. I’m now pulling from three AI platforms plus our customer feedback system through the same setup.
React components just hit your own endpoints instead of wrestling with Langsmith’s weird responses. Much cleaner.
Yeah, Langsmith has REST APIs but their docs are all over the place. Don’t look in the main API reference - check the “Datasets & Evaluations” section instead.
Ran into this same problem last year building analytics for our customer service bot. Here’s what you need:
/api/v1/runs for evaluation run data/api/v1/feedback for feedback scores/api/v1/sessions for conversation metricsGrab your API key from project settings for auth. You’ll get back accuracy scores, latency metrics, and custom evaluation results.
Heads up - they have undocumented rate limits that’ll bite you. Had to add exponential backoff when pulling large datasets. Also cache everything on your Flask backend since their API is pretty slow.
Their JSON comes back super nested, so flatten it before sending to React.