Parsing improperly formatted JSON from AI-generated responses in JavaScript

Other Questions OpenAI
1

I’m working with AI-generated JSON responses and running into issues when trying to parse them in JavaScript. The AI doesn’t escape special characters correctly. It’s using single backslashes instead of double backslashes. Here’s what I’m dealing with:

const problematicJSON = `{
  "prompt": "What time is your flight to 'Boston'?\nDon't forget to include the gate number."
}`

This should actually look like:

const correctJSON = `{
  "prompt": "What time is your flight to \'Boston\'?\nDon't forget to include the gate number."
}`

Does anyone know how to fix the problematic JSON or if there’s a JavaScript library that can handle parsing it as is? I’ve tried using JSON.parse() but it throws an error due to the incorrect formatting. Any suggestions would be really helpful!

2

yo danielr, ive run into this too. have u tried the json-repair library? it’s pretty sweet for fixing broken JSON. just npm install json-repair and use it like:

const jsonRepair = require('json-repair');
const fixed = jsonRepair(problematicJSON);
const parsed = JSON.parse(fixed);

it handles a bunch of common json errors. give it a shot!

3

hey danielr, i’ve faced similar issues. try using a regex replacement before parsing:

const fixedJSON = problematicJSON.replace(/\\'/g, "'");
const parsed = JSON.parse(fixedJSON);

this should replace single backslashes before quotes with actual quotes. hope it helps!

4

I’ve dealt with this exact issue in my AI projects. One trick that’s worked wonders for me is using a custom parser. Here’s a quick and dirty solution I whipped up:

function parseAIJSON(jsonString) {
  return JSON.parse(jsonString.replace(/\\(?=[^\"\\])/g, '\\\\'));
}

const parsed = parseAIJSON(problematicJSON);

This regex looks for single backslashes not followed by quotes or another backslash and doubles them up. It’s not perfect, but it’s handled most cases I’ve thrown at it.

Another approach is to use eval(), but be careful - it can be a security risk if you’re not 100% sure about the input:

const parsed = eval('(' + problematicJSON + ')');

Just remember, these are quick fixes. For production, you might want to look into more robust solutions or talk to your AI provider about improving their JSON output.

5

I’ve encountered this issue in my work with AI-generated responses as well. One approach that’s proved effective is using a custom JSON parser like hjson. It’s designed to be more human-friendly and tolerant of formatting issues.

Here’s how you can implement it:

const Hjson = require('hjson');
const parsedData = Hjson.parse(problematicJSON);

Hjson can handle single quotes, unescaped characters, and even comments in JSON. It’s particularly useful when dealing with AI outputs that may not strictly adhere to JSON syntax rules.

Just remember to convert back to standard JSON if you need to use the data elsewhere in your application. This approach has saved me considerable time and headaches when working with imperfect JSON outputs.

6

I’ve encountered this problem before when working with AI-generated responses. One solution that worked for me was using the JSON5 library. It’s more lenient than standard JSON and can handle single quotes and unescaped characters.

First, install JSON5 with npm:

npm install json5

Then use it in your code:

const JSON5 = require('json5');
const parsedData = JSON5.parse(problematicJSON);

This should parse your AI-generated JSON without errors. Just remember to stringify it back to standard JSON if you need to send it elsewhere. It’s a robust solution that’s saved me countless hours of debugging.