I’m working on a chatbot project with a React frontend and Flask backend. I’ve been using Langsmith to track my bot’s performance metrics, but now I need to pull that data into my own dashboard. I want to show things like response accuracy, conversation ratings, and other evaluation stats directly in my app instead of having to check Langsmith separately.
I’ve been looking through their documentation but haven’t found clear examples of how to access their data programmatically. Does anyone know if Langsmith offers API endpoints for retrieving performance data? I’m specifically interested in getting evaluation results and metrics that I can then display in custom charts on my frontend.
Pulling data manually from APIs gets messy fast. Every time Langsmith updates their endpoints or changes response formats, you’re stuck fixing broken code again.
Just automate the whole data flow instead of building polling and caching logic yourself. Set it up once and forget it - handles API calls, transforms data, and feeds your database automatically.
I did this for our chatbot dashboard. Pulls evaluation metrics from Langsmith hourly, converts the JSON to what my React components need, and dumps it straight into PostgreSQL. Haven’t touched it since setup.
Add smart filtering to grab only the evaluation types you want. Way better than pulling everything and sorting later.
The real win is failure notifications. When Langsmith’s API dies or you hit rate limits, you’ll know right away instead of waiting for users to complain about stale data.
Think about how often you need fresh data before you build this. I made something similar last year and started with real-time API calls whenever the dashboard loaded. Huge mistake - users hated the slow loading times. Webhooks worked way better. I set up a webhook listener in Flask so Langsmith pushes evaluation results straight to my endpoint when runs finish. You get near real-time updates without constantly hammering their API. Just store the incoming data in your database and serve it instantly to React. The webhook setup also lets you crunch numbers server-side first. I calculate rolling averages and trend stuff on the backend instead of making browsers do heavy lifting. Performance boost was huge, especially on mobile. One heads up: validate those webhook payloads properly. Langsmith sends run metadata that’ll break your parsing if you’re not ready for it.
Ran into some gotchas with Langsmith’s evaluation endpoints that’ll save you time. The /evaluations endpoint gives aggregate stats but no individual run context - you’ll need that for detailed analytics. Had to cross-reference with runs data to get anything useful.
Watch out for timestamp filtering too. They use ISO format but timezone handling gets wonky with Flask. I just store UTC timestamps and convert them client-side in React.
Batching requests by evaluation type works way better than time ranges. Much more efficient for custom charts. Pull accuracy metrics separately from user ratings and combine however works for your dashboard.
Docs are pretty sparse but support responds fast if you get stuck on specific endpoints.
Yeah, Langsmith has REST API endpoints for this. I’ve built similar analytics pipelines - pretty straightforward once you handle auth.
Use their Runs API to pull evaluation data. Hit /runs with query params for your project and time range. You’ll get accuracy scores, latency, feedback ratings, and custom eval results.
For auth, grab an API key from your Langsmith dashboard settings. Add it to request headers as x-api-key.
Here’s what I did on a similar project: scheduled job in Flask pulls data hourly and caches in Redis. My React frontend hits my own API instead of Langsmith directly every dashboard load.
Response is JSON so it’s easy to parse for charts. Handle pagination if you’re pulling large datasets.
Heads up - their rate limits are conservative. Don’t hammer it during development.
Been there a few months ago - exact same headache. Nobody’s mentioned this yet, but Langsmith’s API dumps way more data than you need. Use their query params to filter at the API level instead of pulling everything then filtering client-side.
For React, build a simple Flask caching layer first. Pull data every few hours and store what matters. Your charts load instantly and you dodge rate limits.
Figure out which evaluation metrics you actually want to show. Langsmith tracks everything, but users care about maybe 3-4 key numbers. I wasted tons of time on complex charts nobody used.
Heads up - custom evaluations use different endpoint structure and metadata format. Saves you from wondering why your parsing randomly breaks.
Been there multiple times. APIs work but you’ll quickly hate dealing with auth tokens, rate limits, and caching.
Skip the custom Flask polling and automate the whole pipeline instead. Set up a workflow that runs on schedule, grabs your Langsmith metrics, formats them for React, and dumps everything into your database or straight to the frontend. No more token refresh headaches or rate limit crashes.
Did this for our chatbot analytics last month - 15 minutes to set up and it handles everything. Way cleaner than maintaining API code.
Bonus: add alerts when metrics tank or syncing breaks.
honestly, the langsmith python client beats raw REST calls every time. just pip install langsmith and use client.get_runs(). it handles auth automatically and takes care of pagination for you. i’ve been using it for months - zero issues.