enhanced json parsing

[iEgit]

reopened from #4125

This PR improves the JSON parsing logic in StructuredOutputParser.ts to handle malformed JSON responses from LLMs more effectively.

Changes:
• Fixed JSON extraction logic:
• Previously, JSON extraction relied on splitting text using code block markers (`json).
• Now, it uses a regex (/(?:^[^{[])|(?:[^}\]]$)/g) to remove any leading or trailing non-JSON content, ensuring a more robust parsing approach.

Why this change?
• The previous approach failed in some edge cases where JSON responses did not follow the expected format.
• The new regex-based approach ensures that only valid JSON content is passed to jsonrepair, reducing the likelihood of parsing errors.

Testing:
• Manually tested with different malformed JSON outputs to verify correct parsing.
• Ensured existing functionality remains intact.

[HenryHengZJ]

Do you have some examples where the old approach is not able to parse, but solvable with the newer approach?

[iEgit]

yes, basically it breaks if llm outputs something like this:

sure, here's you json output:

{
  "result": "string"

in our case it was even worse - we provided few shot examples and it repeated part of the prompt and only then provided the json response