Skip to content

Add support for attachment to rails integration #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,47 @@ end
# That's it - chat history is automatically saved
```

### Configurable Attachment Storage

By default, RubyLLM stores attachments (images, PDFs, audio) as base64-encoded data directly in your database. For applications with different storage needs, you can configure custom storage adapters:

```ruby
# Custom adapter that transforms base64 to URLs (e.g., using ActiveStorage)
url_adapter = lambda do |operation, part, record: nil|
case operation
when :store
if part[:type] == 'image' && part[:source].is_a?(Hash) && part[:source][:type] == 'base64'
# Store the image using your preferred storage system
# For example, with ActiveStorage:
blob = ActiveStorage::Blob.create_and_upload!(
io: StringIO.new(Base64.decode64(part[:source][:data])),
filename: "image_#{SecureRandom.hex(4)}.png",
content_type: part[:source][:media_type]
)

# Return a transformed part with URL instead of base64 data
{
type: part[:type],
source: {
url: Rails.application.routes.url_helpers.url_for(blob)
}
}
else
part # Return other parts unchanged
end
when :retrieve
# Handle retrieval logic if needed
part
end
end

# Configure your models to use the custom adapter
class Message < ApplicationRecord
acts_as_message attachment_storage: url_adapter
end
```


## Creating tools is a breeze

```ruby
Expand Down
130 changes: 130 additions & 0 deletions docs/guides/rails.md
Original file line number Diff line number Diff line change
Expand Up @@ -459,6 +459,136 @@ end
AskAiJob.perform_later(chat.id, "Tell me about Ruby")
```

## Attachment Storage

RubyLLM supports rich content like images, PDFs, and audio files in your conversations. By default, these attachments are stored as base64-encoded data directly in your database. However, for production applications, you might want to use a different storage strategy.

### Default Storage: Base64

With the default configuration, attachments are stored directly in the database as base64-encoded strings:

```ruby
class Chat < ApplicationRecord
acts_as_chat
# Uses default base64 storage
end

class Message < ApplicationRecord
acts_as_message
# Uses default base64 storage
end
```

This approach is simple and requires no additional setup, but it can lead to database bloat with large attachments.

### Custom Storage Adapters

RubyLLM provides a flexible adapter system that allows you to customize how attachments are stored and retrieved:

```ruby
class Chat < ApplicationRecord
acts_as_chat attachment_storage: :custom_adapter
end

class Message < ApplicationRecord
acts_as_message attachment_storage: :custom_adapter
end
```

### Using ActiveStorage

A common use case is storing attachments with ActiveStorage instead of in the database. Here's how to implement a custom adapter for ActiveStorage:

```ruby
# app/models/concerns/active_storage_adapter.rb
module ActiveStorageAdapter
def self.call(operation, part, record: nil)
case operation
when :store
store_attachment(part, record)
when :retrieve
retrieve_attachment(part)
end
end

def self.store_attachment(part, record)
# Skip non-attachment parts or already processed parts
return part unless attachment?(part)
return part if part[:source].key?(:url)
return part unless record

attachment_type = part[:type].to_sym # :image, :pdf, or :audio

# Create a blob from the base64 data
if part[:source][:type] == 'base64' && part[:source][:data].present?
blob = create_blob_from_base64(
data: part[:source][:data],
filename: "#{attachment_type}_#{SecureRandom.hex(8)}",
content_type: part[:source][:media_type]
)

# Attach the blob to the record
record.attachments.attach(blob)

# Return a transformed part with a URL instead of base64 data
{
type: part[:type],
source: {
url: Rails.application.routes.url_helpers.url_for(blob),
media_type: part[:source][:media_type]
}
}
else
part
end
end

def self.retrieve_attachment(part)
# Just return the part as is for now
# You could implement additional logic here if needed
part
end

private

def self.attachment?(part)
%w[image pdf audio].include?(part[:type]) &&
part[:source].is_a?(Hash)
end

def self.create_blob_from_base64(data:, filename:, content_type:)
decoded_data = Base64.decode64(data)
ActiveStorage::Blob.create_and_upload!(
io: StringIO.new(decoded_data),
filename: filename,
content_type: content_type
)
end
end

# app/models/message.rb
class Message < ApplicationRecord
acts_as_message attachment_storage: ActiveStorageAdapter

has_many_attached :attachments
end
```

The adapter system passes the record directly to your adapter, giving you access to the ActiveRecord model when processing attachments. This allows you to:

1. Access model attributes and associations
2. Attach files directly to the model
3. Use model-specific logic for attachment handling

### Adapter Operations

Custom adapters must handle two operations:

1. `:store` - Called when saving attachments to the database
2. `:retrieve` - Called when retrieving attachments from the database

Each operation receives a content part (a hash with type and source information) and the record being processed, and should return a transformed part.

## Next Steps

Now that you've integrated RubyLLM with Rails, you might want to explore:
Expand Down
Loading