Skip to content

ToolAnnotations #185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Mar 26, 2025
+103 −6
Merged

ToolAnnotations #185

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
25c8e7c
Remove `category` and `extraInfo`, add `openWorld` and `caption`, add…
bhosmer-ant Mar 16, 2025
bf513ca
update schema.json
bhosmer-ant Mar 16, 2025
c17ee30
rebase on #208, split out DisplayAnnotations
bhosmer-ant Mar 18, 2025
3a96a59
Refactor tool annotations to add effectHints property
bhosmer-ant Mar 21, 2025
f025054
Refactor ToolAnnotations:
bhosmer-ant Mar 24, 2025
2295e7d
prettier
bhosmer-ant Mar 24, 2025
8cf6f27
add closing callout tag
bhosmer-ant Mar 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/specification/draft/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,9 @@ considerations that all implementors must carefully address.
3. **Tool Safety**

- Tools represent arbitrary code execution and must be treated with appropriate
caution
caution.
- In particular, descriptions of tool behavior such as annotations should be
considered untrusted, unless obtained from a trusted server.
- Hosts must obtain explicit user consent before invoking any tool
- Users should understand what each tool does before authorizing its use

Expand Down
4 changes: 4 additions & 0 deletions docs/specification/draft/server/tools.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,10 @@ A tool definition includes:
- `name`: Unique identifier for the tool
- `description`: Human-readable description of functionality
- `inputSchema`: JSON Schema defining expected parameters
- `annotations`: optional properties describing tool behavior

{{< callout type="warning" >}} For trust & safety and security, clients **MUST** consider
tool annotations to be untrusted unless they come from trusted servers. {{< /callout >}}

### Tool Result

Expand Down
32 changes: 31 additions & 1 deletion schema/draft/schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1938,8 +1938,12 @@
"Tool": {
"description": "Definition for a tool the client can call.",
"properties": {
"annotations": {
"$ref": "#/definitions/ToolAnnotations",
"description": "Optional additional tool information."
},
"description": {
"description": "A human-readable description of the tool.",
"description": "A human-readable description of the tool.\n\nThis can be used by clients to improve the LLM's understanding of available tools. It can be thought of like a \"hint\" to the model.",
"type": "string"
},
"inputSchema": {
Expand Down Expand Up @@ -1980,6 +1984,32 @@
],
"type": "object"
},
"ToolAnnotations": {
Copy link
Contributor

@PederHP PederHP Mar 21, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having some kind of additional properties mechanism to have free-form annotations would be very useful. The use case is having internal MCP servers that we use with internal MCP clients (that are back-end services). The annotations we'd use (which could include rate-limiting information, information sensitivity classifiers, etc.) would likely not be general enough to ever find their way into the spec, but they'd of great value to us.

The alternative is that we need maintain this information elsewhere - or do a branch of the SDK(s) and add the fields as what we do is all internal, but that's not optimal either. Free-form annotations are very useful for MCP-based infrastructures that aren't directly user-facing. We don't need to have our annotations be known to clients at large, for servers we'll never distribute externally.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All objects in the MCP spec are intentionally open-ended, so it's always possible to add custom fields that only your server and client implementations understand. (No forking required!)

Copy link

@hackjammer hackjammer Mar 24, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another drive by comment, I just stumbled on this PR whilst considering a parallel internal implementation. Some thoughts:

  • Information sensitivity classifiers is one of the examples we had. So this is always going to need to be extensible for different businesses with different definitions / risks / policies
  • Looks like effectHints is not extensible as currently implemented? We had other property ideas e.g. "appendOnly" or "irrevocable". If we want this to be extensible for now then maybe effectHints should be too
  • In general it might be good to give examples of how this should be extended for custom metadata / hints and guidance on when to share back standards

Copy link
Contributor

@PederHP PederHP Mar 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All objects in the MCP spec are intentionally open-ended, so it's always possible to add custom fields that only your server and client implementations understand. (No forking required!)

I hope you don't mind a comment on this on a merged PR, but it seemed like the obvious place. Very glad to hear it - I think this makes the protocol stronger and helps facilitate bottom-up growth.

This is true for weakly-typed languages, like Python or TypeScript, but for C# it makes a difference if the schemas explicitly has metadata and/or kwargs-style additional properties (as some types do, but not all).

So I think either adding a basetype that makes this explicit, or adding it explicitly to types where it is likely to be used (like Tools) would be very helpful. Especially for cases where a server is Python/TypeScript and the Client is C#.

I assume similar concerns are valid for Rust and Java. Perhaps I should open an issue about consistency in terms of how open-ended types are handled in the scehma?

"description": "Additional properties describing a Tool to clients.\n\nNOTE: all properties in ToolAnnotations are **hints**. \nThey are not guaranteed to provide a faithful description of \ntool behavior (including descriptive properties like `title`).\n\nClients should never make tool use decisions based on ToolAnnotations\nreceived from untrusted servers.",
"properties": {
"destructiveHint": {
"description": "If true, the tool may perform destructive updates to its environment.\nIf false, the tool performs only additive updates.\n\n(This property is meaningful only when `readOnlyHint == false`)\n\nDefault: true",
"type": "boolean"
},
"idempotentHint": {
"description": "If true, calling the tool repeatedly with the same arguments \nwill have no additional effect on the its environment.\n\n(This property is meaningful only when `readOnlyHint == false`)\n\nDefault: false",
"type": "boolean"
},
"openWorldHint": {
"description": "If true, this tool may interact with an \"open world\" of external\nentities. If false, the tool's domain of interaction is closed.\nFor example, the world of a web search tool is open, whereas that\nof a memory tool is not.\n\nDefault: true",
"type": "boolean"
},
"readOnlyHint": {
"description": "If true, the tool does not modify its environment.\n\nDefault: false",
"type": "boolean"
},
"title": {
"description": "A human-readable title for the tool.",
"type": "string"
}
},
"type": "object"
},
"ToolListChangedNotification": {
"description": "An optional notification from the server to the client, informing it that the list of tools it offers has changed. This may be issued by servers without any previous subscription from the client.",
"properties": {
Expand Down
69 changes: 65 additions & 4 deletions schema/draft/schema.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,60 @@ export interface ToolListChangedNotification extends Notification {
method: "notifications/tools/list_changed";
}

/**
* Additional properties describing a Tool to clients.
*
* NOTE: all properties in ToolAnnotations are **hints**.
* They are not guaranteed to provide a faithful description of
* tool behavior (including descriptive properties like `title`).
*
* Clients should never make tool use decisions based on ToolAnnotations
* received from untrusted servers.
*/
export interface ToolAnnotations {
/**
* A human-readable title for the tool.
*/
title?: string;

/**
* If true, the tool does not modify its environment.
*
* Default: false
*/
readOnlyHint?: boolean;

/**
* If true, the tool may perform destructive updates to its environment.
* If false, the tool performs only additive updates.
*
* (This property is meaningful only when `readOnlyHint == false`)
*
* Default: true
*/
destructiveHint?: boolean;

/**
* If true, calling the tool repeatedly with the same arguments
* will have no additional effect on the its environment.
*
* (This property is meaningful only when `readOnlyHint == false`)
*
* Default: false
*/
idempotentHint?: boolean;

/**
* If true, this tool may interact with an "open world" of external
* entities. If false, the tool's domain of interaction is closed.
* For example, the world of a web search tool is open, whereas that
* of a memory tool is not.
*
* Default: true
*/
openWorldHint?: boolean;
}

/**
* Definition for a tool the client can call.
*/
Expand All @@ -704,10 +758,14 @@ export interface Tool {
* The name of the tool.
*/
name: string;

/**
* A human-readable description of the tool.
*
* This can be used by clients to improve the LLM's understanding of available tools. It can be thought of like a "hint" to the model.
*/
description?: string;

/**
* A JSON Schema object defining the expected parameters for the tool.
*/
Expand All @@ -716,6 +774,11 @@ export interface Tool {
properties?: { [key: string]: object };
required?: string[];
};

/**
* Optional additional tool information.
*/
annotations?: ToolAnnotations;
}

/* Logging */
Expand Down Expand Up @@ -833,14 +896,14 @@ export interface SamplingMessage {
export interface Annotations {
/**
* Describes who the intended customer of this object or data is.
*
*
* It can include multiple entries to indicate content useful for multiple audiences (e.g., `["user", "assistant"]`).
*/
audience?: Role[];

/**
* Describes how important this data is for operating the server.
*
*
* A value of 1 means "most important," and indicates that the data is
* effectively required, while 0 means "least important," and indicates that
* the data is entirely optional.
Expand Down Expand Up @@ -893,7 +956,6 @@ export interface ImageContent {
annotations?: Annotations;
}


/**
* Audio provided to or from an LLM.
*/
Expand All @@ -918,7 +980,6 @@ export interface AudioContent {
annotations?: Annotations;
}


/**
* The server's preferences for model selection, requested of the client during sampling.
*
Expand Down