Structured output & json mode

CLAUDE.md

@@ -0,0 +1,23 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Build & Test Commands
+- Build: `bundle exec rake build`
+- Install dependencies: `bundle install`
+- Run all tests: `bundle exec rspec`
+- Run specific test: `bundle exec rspec spec/ruby_llm/chat_spec.rb`
+- Run specific test by description: `bundle exec rspec -e "description"`
+- Re-record VCR cassettes: `bundle exec rake vcr:record[all]` or `bundle exec rake vcr:record[openai,anthropic]`
+- Check style: `bundle exec rubocop`
+- Auto-fix style: `bundle exec rubocop -A`
+
+## Code Style Guidelines
+- Follow [Standard Ruby](https://github.com/testdouble/standard) style
+- Use frozen_string_literal comment at the top of each file
+- Follow model naming conventions from CONTRIBUTING.md when adding providers
+- Use RSpec for tests with descriptive test names that form clean VCR cassettes
+- Handle errors with specific error classes from RubyLLM::Error
+- Use method keyword arguments with Ruby 3+ syntax
+- Document public APIs with YARD comments
+- Maintain backward compatibility for minor version changes

README.md

@@ -36,6 +36,7 @@ RubyLLM fixes all that. One beautiful API for everything. One consistent format.
 - 🖼️ **Image generation** with DALL-E and other providers
 - 📊 **Embeddings** for vector search and semantic analysis
 - 🔧 **Tools** that let AI use your Ruby code
+- 📝 **Structured Output** with JSON schema validation
 - 🚂 **Rails integration** to persist chats and messages with ActiveRecord
 - 🌊 **Streaming** responses with proper Ruby patterns
 
@@ -83,6 +84,28 @@ class Weather < RubyLLM::Tool
 end
 
 chat.with_tool(Weather).ask "What's the weather in Berlin? (52.5200, 13.4050)"
+
+# Get structured output with JSON schema validation
+schema = {
+  type: "object",
+  properties: {
+    name: { type: "string" },
+    age: { type: "integer" },
+    interests: { 
+      type: "array", 
+      items: { type: "string" }
+    }
+  },
+  required: ["name", "age", "interests"]
+}
+
+# Returns a validated Hash instead of plain text
+user_data = chat.with_output_schema(schema).ask("Create a profile for a Ruby developer")
+
+# Access the structured data using hash keys
+puts "Name: #{user_data.content['name']}"              # => "Jane Smith"
+puts "Age: #{user_data.content['age']}"                # => 32
+puts "Interests: #{user_data.content['interests'].join(', ')}"  # => "Ruby, Rails, API design"
 ```
 
 ## Installation

docs/_data/navigation.yml

@@ -19,6 +19,8 @@
       url: /guides/image-generation
     - title: Embeddings
       url: /guides/embeddings
+    - title: Structured Output
+      url: /guides/structured-output
     - title: Error Handling
       url: /guides/error-handling
     - title: Models

docs/guides/index.md

@@ -33,6 +33,9 @@ Learn how to generate images using DALL-E and other providers.
 ### [Embeddings]({% link guides/embeddings.md %})
 Explore how to create vector embeddings for semantic search and other applications.
 
+### [Structured Output]({% link guides/structured-output.md %})
+Learn how to use JSON schemas to get validated structured data from LLMs.
+
 ### [Error Handling]({% link guides/error-handling.md %})
 Master the techniques for robust error handling in AI applications.
 

docs/guides/rails.md

@@ -25,6 +25,7 @@ After reading this guide, you will know:
 *   How to set up ActiveRecord models for persisting chats and messages.
 *   How to use `acts_as_chat` and `acts_as_message`.
 *   How chat interactions automatically persist data.
+*   How to work with structured output in your Rails models.
 *   A basic approach for integrating streaming responses with Hotwire/Turbo Streams.
 
 ## Setup
@@ -174,6 +175,89 @@ system_message = chat_record.messages.find_by(role: :system)
 puts system_message.content # => "You are a concise Ruby expert."
 ```
 
+## Working with Structured Output
+
+RubyLLM 1.3.0+ supports structured output with JSON schema validation. This works seamlessly with Rails integration, allowing you to get and persist structured data from AI models.
+
+### Database Considerations
+
+For best results with structured output, use a database that supports JSON data natively:
+
+```ruby
+# For PostgreSQL, use jsonb for the content column
+class CreateMessages < ActiveRecord::Migration[7.1]
+  def change
+    create_table :messages do |t|
+      t.references :chat, null: false, foreign_key: true
+      t.string :role
+      t.jsonb :content # Use jsonb instead of text for PostgreSQL
+      # ...other fields...
+    end
+  end
+end
+```
+
+For databases without native JSON support, you can use text columns with serialization:
+
+```ruby
+# app/models/message.rb
+class Message < ApplicationRecord
+  acts_as_message
+  serialize :content, JSON # Add this for text columns
+end
+```
+
+### Using Structured Output
+
+The `with_output_schema` method is available on your `Chat` model thanks to `acts_as_chat`:
+
+```ruby
+# Make sure to use a model that supports structured output
+chat_record = Chat.create!(model_id: 'gpt-4.1-nano')
+
+# Define your JSON schema
+schema = {
+  type: "object",
+  properties: {
+    name: { type: "string" },
+    version: { type: "string" },
+    features: { 
+      type: "array", 
+      items: { type: "string" }
+    }
+  },
+  required: ["name", "version"]
+}
+
+begin
+  # Get structured data instead of plain text
+  response = chat_record.with_output_schema(schema).ask("Tell me about Ruby")
+
+  # The response content is a Hash (or serialized JSON in text columns)
+  response.content # => {"name"=>"Ruby", "version"=>"3.2.0", "features"=>["Blocks", "Procs"]}
+
+  # You can access the persisted message as usual
+  message = chat_record.messages.where(role: 'assistant').last
+  message.content['name'] # => "Ruby"
+
+  # In your views, you can easily display structured data:
+  # <%= message.content['name'] %> <%= message.content['version'] %>
+  # <ul>
+  #   <% message.content['features'].each do |feature| %>
+  #     <li><%= feature %></li>
+  #   <% end %>
+  # </ul>
+rescue RubyLLM::UnsupportedStructuredOutputError => e
+  # Handle case where the model doesn't support structured output
+  puts "This model doesn't support structured output: #{e.message}"
+rescue RubyLLM::InvalidStructuredOutput => e
+  # Handle case where the model returns invalid JSON
+  puts "The model returned invalid JSON: #{e.message}"
+end
+```
+
+With this approach, you can build robust data-driven applications that leverage the structured output capabilities of AI models while properly handling errors.
+
 ## Streaming Responses with Hotwire/Turbo
 
 You can combine `acts_as_chat` with streaming and Turbo Streams for real-time UI updates. The persistence logic works seamlessly alongside the streaming block.

docs/guides/structured-output.md

@@ -0,0 +1,204 @@
+---
+layout: default
+title: Structured Output
+parent: Guides
+nav_order: 7
+---
+
+# Structured Output
+
+RubyLLM allows you to request structured data from language models by providing a JSON schema. When you use the `with_output_schema` method, RubyLLM will ensure the model returns data matching your schema instead of free-form text.
+
+## Basic Usage
+
+```ruby
+# Define a JSON schema
+schema = {
+  type: "object",
+  properties: {
+    name: { type: "string" },
+    age: { type: "integer" },
+    interests: { 
+      type: "array", 
+      items: { type: "string" }
+    }
+  },
+  required: ["name", "age", "interests"]
+}
+
+# Get structured output as a Hash
+response = RubyLLM.chat
+  .with_output_schema(schema)
+  .ask("Create a profile for a Ruby developer")
+
+# Access the structured data
+puts "Name: #{response.content['name']}"
+puts "Age: #{response.content['age']}"
+puts "Interests: #{response.content['interests'].join(', ')}"
+```
+
+## Provider Support
+
+### Strict Mode (Default)
+
+By default, RubyLLM uses "strict mode" which only allows providers that officially support structured JSON output:
+
+- **OpenAI**: For models that support JSON mode (like GPT-4.1, GPT-4o), RubyLLM uses the native `response_format: {type: "json_object"}` parameter.
+
+If you try to use an unsupported model in strict mode, RubyLLM will raise an `UnsupportedStructuredOutputError` (see [Error Handling](#error-handling)).
+
+### Non-Strict Mode
+
+You can disable strict mode by setting `strict: false` when calling `with_output_schema`:
+
+```ruby
+# Allow structured output with non-OpenAI models
+chat = RubyLLM.chat(model: "gemini-2.0-flash")
+response = chat.with_output_schema(schema, strict: false)
+                .ask("Create a profile for a Ruby developer")
+
+# The response.content will be a Hash if JSON parsing succeeds
+if response.content.is_a?(Hash)
+  puts "Name: #{response.content['name']}"
+  puts "Age: #{response.content['age']}"
+else
+  # Fall back to treating as string if parsing failed
+  puts "Got text response: #{response.content}"
+end
+```
+
+In non-strict mode:
+- The system will not validate if the model officially supports structured output
+- The schema is still included in the system prompt to guide the model
+- RubyLLM automatically attempts to handle markdown code blocks (like ````json\n{...}````)
+- JSON is parsed when possible, but might fall back to raw text in some cases
+- Works with Anthropic Claude and Google Gemini models, but results can vary
+
+This is useful for experimentation with models like Anthropic's Claude or Gemini, but should be used with caution in production environments.
+
+## Error Handling
+
+RubyLLM has two error types related to structured output:
+
+1. **UnsupportedStructuredOutputError**: Raised when you try to use structured output with a model that doesn't support it in strict mode:
+
+```ruby
+begin
+  chat = RubyLLM.chat(model: 'claude-3-5-haiku')
+  chat.with_output_schema(schema) # This will raise an error
+rescue RubyLLM::UnsupportedStructuredOutputError => e
+  puts "This model doesn't support structured output: #{e.message}"
+
+  # You can try with strict mode disabled
+  chat.with_output_schema(schema, strict: false)
+end
+```
+
+2. **InvalidStructuredOutput**: Raised if the model returns invalid JSON:
+
+```ruby
+begin
+  response = chat.with_output_schema(schema).ask("Create a profile")
+rescue RubyLLM::InvalidStructuredOutput => e
+  puts "The model returned invalid JSON: #{e.message}"
+end
+```
+
+Note that the current implementation only checks that the response is valid JSON that can be parsed. It does not verify that the parsed content conforms to the schema structure (e.g., having all required fields or correct data types). If you need full schema validation, you'll need to implement it using a library like `json-schema`.
+
+## With ActiveRecord and Rails
+
+The structured output feature works seamlessly with RubyLLM's Rails integration. Message content can now be either a String or a Hash.
+
+If you're storing message content in your database and want to use structured output, ensure your messages table can store JSON. PostgreSQL's `jsonb` column type is ideal:
+
+```ruby
+# In a migration
+create_table :messages do |t|
+  t.references :chat
+  t.string :role
+  t.jsonb :content # Use jsonb for efficient JSON storage
+  # other fields...
+end
+```
+
+If you have an existing application with a text-based content column, you can add serialization:
+
+```ruby
+# In your Message model
+class Message < ApplicationRecord
+  serialize :content, JSON
+  acts_as_message
+end
+```
+
+## Tips for Effective Schemas
+
+1. **Be specific**: Provide clear property descriptions to guide the model's output.
+2. **Start simple**: Begin with basic schemas and add complexity gradually.
+3. **Include required fields**: Specify which properties are required.
+4. **Use appropriate types**: Match JSON Schema types to your expected data.
+5. **Validate locally**: Consider using a gem like `json-schema` for additional validation.
+
+## Example: Complex Schema
+
+```ruby
+schema = {
+  type: "object",
+  properties: {
+    products: {
+      type: "array",
+      items: {
+        type: "object",
+        properties: {
+          name: { type: "string" },
+          price: { type: "number" },
+          in_stock: { type: "boolean" },
+          categories: {
+            type: "array",
+            items: { type: "string" }
+          }
+        },
+        required: ["name", "price", "in_stock"]
+      }
+    },
+    total_products: { type: "integer" },
+    store_info: {
+      type: "object",
+      properties: {
+        name: { type: "string" },
+        location: { type: "string" }
+      }
+    }
+  },
+  required: ["products", "total_products"]
+}
+
+inventory = chat.with_output_schema(schema).ask("Create an inventory for a Ruby gem store")
+```
+
+## Implementation Details
+
+The current implementation of structured output in RubyLLM:
+
+1. **For OpenAI**: 
+   - Uses OpenAI's native JSON mode via `response_format: {type: "json_object"}`
+   - Returns parsed Hash objects directly
+   - Works reliably in production settings
+
+2. **For other providers (with strict: false)**:
+   - Includes schema guidance in the system prompt
+   - Does not use provider-specific JSON modes
+   - Automatically handles markdown code blocks (like ````json\n{...}````)
+   - Attempts to parse JSON responses when possible
+   - Returns varying results depending on the model's capabilities
+   - Better suited for experimentation than production use
+
+### Limitations
+
+- No schema validation beyond JSON parsing
+- No enforcement of required fields or data types
+- Not all providers support structured output reliably
+- Response format consistency varies between providers
+
+This feature is currently in alpha and we welcome feedback on how it can be improved. Future versions will likely include more robust schema validation and better support for additional providers.