Skip to content

JSON schema tool parameters #76

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
K4sku opened this issue Mar 28, 2025 · 3 comments
Open

JSON schema tool parameters #76

K4sku opened this issue Mar 28, 2025 · 3 comments
Labels
enhancement New feature or request

Comments

@K4sku
Copy link

K4sku commented Mar 28, 2025

Hello! Thank you for the gem!

I'm exploring this gem by rewriting a small llm agent in a Rails app. And I got to a point where I'm rewriting tools.
In original implementation I'm using an enum parameter. And I do not know how to implement it here.

So far I have squished it into a description:
"The current unit of the temperature value. Must be one of: 'celsius', 'fahrenheit'."

While looking for an solution I found this example: https://rubyllm.com/guides/tools#advanced-tool-parameters

class DataAnalysis < RubyLLM::Tool
  description "Analyzes numerical data"

  param :data,
    type: :array,
    desc: "Array of numbers to analyze"

  param :operations,
    type: :object,
    desc: "Analysis operations to perform",
    required: false

  def execute(data:, operations: {mean: true, median: false})
    result = {}

    result[:mean] = data.sum.to_f / data.size if operations[:mean]
    result[:median] = calculate_median(data) if operations[:median]

    result
  end

(...)
end

I do not understand how llm would know that operations cloud be mean and median, and that it should be a hash at all. I'm discussing it with an LLM (ironic, isn't it?) , and it says the example would not work.
To make it work the description should be much more specific:
"Specifies which analysis to perform. Should be an object with optional boolean keys: 'mean' and 'median'. For example: {'mean': true, 'median': false}. If omitted, both mean and median might be calculated.".

I saw the issue #11 about Structured output and I think structured input would be nice to have too.
What do you think about including this?

I have no ideas on implementation or API for these yet, but I can try working on it.

@crmne
Copy link
Owner

crmne commented Apr 1, 2025

Thanks for catching this! You're right, we don't actually support complex parameter types like objects and arrays yet because support is inconsistent across providers. I've updated the docs to remove the misleading example in 0a73d74

I think that just asking users to flattening params might be simpler to implement than full complex tool parameters. However, if you're interested in contributing to this feature, I'd be happy to discuss approaches.

@crmne crmne added the enhancement New feature or request label Apr 1, 2025
@K4sku
Copy link
Author

K4sku commented Apr 9, 2025

I have been thinking about how to handle this.

I think the solution to defining tools with JSON schema params should be parallel to eventual solution used in Structured output parser. At least on the public API level.

The other issue is support for providers that do not have this feature that you raised.

Easiest solution is to just parse everything into description string.

Other possible solution is to do some parsing.
For example enum could be parsed as Must be one of: array.join(", "). That might give better results, but then some option to override it would be nice to have.

Another idea, using provider that does not support "advanced tool parameter" with such tool could at least leave some warning in log (with an option to suppress these logs).

@jayelkaake
Copy link

Just posting here for visibility but this is solved by my PR @ #124

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants
@crmne @jayelkaake @K4sku and others