from __future__ import annotations causes ai functions to return strings

[jimkring]

First check

• I added a descriptive title to this issue.
• I used the GitHub search to try to find a similar issue and didn't find one.
• I searched the Marvin documentation for this issue.

Bug summary

I was having a strange issue where ai functions were returning string data instead of the type specified in my function signature. It turns out that having from __future__ import annotations was causing the issue.

Reproduction

from __future__ import annotations  # this causes the problem to appear
import inspect  # we'll use this to look at why it's happening 

import marvin
from pydantic import BaseModel

@marvin.fn
def recipe(
        ingredients: list[str],
        max_cook_time: int = 15,
        cuisine: str = "North Italy",
        experience_level: str = "beginner",
) -> Recipe:
    """
    Returns a complete recipe that uses all the `ingredients` and
    takes less than `max_cook_time`  minutes to prepare. Takes 
    `cuisine` style and the chef's `experience_level` into account 
    as well.
    """

# get a recipe
my_recipe = recipe(["milk", "sugar", "flour"])

# verify the output is a Recipe
assert isinstance(my_recipe, Recipe)

# let's check the function signature
sig = inspect.signature(recipe)

# make sure it's output is a Recipe
assert type(sig.return_annotation) is Recipe

Error

No response

Versions

> marvin version
Version:                2.3.6
Python version:         3.11.4
OS/Arch:                windows/amd64

Additional context

No response

[zzstoatzz]

hi @jimkring - thanks for the issue!

can you show the definition of the Recipe type?

[jimkring]

This is from the Marvin examples. https://www.askmarvin.ai/docs/text/functions/#parameters

class Recipe(BaseModel):
    name: str
    cook_time_minutes: int
    ingredients: list[str]
    steps: list[str]

[jimkring]

@zzstoatzz lmk if this is enough info for you to reproduce. Thx.

[zzstoatzz]

hi @jimkring

thanks for the issue! i have reproduced, it looks like the import itself indeed causes this, which is interesting. will take a look as soon as I can

[jimkring]

@zzstoatzz Thank you. I was able to work around this issue by keeping my model definitions separate (in a different module) from where I call marvin. So, if a fix is not easy, a suitable workaround might be to just generate an error if this condition is detected along with a message that helps point to a solution.

[zzstoatzz]

hey @jimkring,

finally had some time to look at this! sorry yeah, that from __future__ import annotations definitely causes weirdness even now in 3.x. As you alluded to, it turns type hints into strings per PEP 563 and so with the same example I'm seeing:

» uv run repros/950.py

Traceback (most recent call last):
  File "/Users/nate/github.com/prefecthq/marvin/repros/950.py", line 32, in <module>
    my_recipe = recipe(["milk", "sugar", "flour"])
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/src/marvin/fns/fn.py", line 151, in wrapper
    return run_sync(coro)
           ^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/src/marvin/utilities/asyncio.py", line 42, in run_sync
    return ctx.run(loop.run_until_complete, coro)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.12.7_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/src/marvin/fns/fn.py", line 199, in _fn
    result = await task.run_async(thread=thread)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/src/marvin/tasks/task.py", line 476, in run_async
    await marvin.fns.run.run_tasks_async(
  File "/Users/nate/github.com/prefecthq/marvin/src/marvin/fns/run.py", line 29, in run_tasks_async
    await orchestrator.run(raise_on_failure=raise_on_failure)
  File "/Users/nate/github.com/prefecthq/marvin/src/marvin/engine/orchestrator.py", line 246, in run
    result = await self.run_once(actor=actor)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/src/marvin/engine/orchestrator.py", line 169, in run_once
    agentlet = await actor.get_agentlet(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/src/marvin/agents/agent.py", line 147, in get_agentlet
    agentlet = pydantic_ai.Agent[Any, result_type](  # type: ignore
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.12.7_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/typing.py", line 1184, in __call__
    result = self.__origin__(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/.venv/lib/python3.12/site-packages/pydantic_ai/agent.py", line 215, in __init__
    self._result_schema: _result.ResultSchema[ResultDataT] | None = _result.ResultSchema[result_type].build(
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/.venv/lib/python3.12/site-packages/pydantic_ai/_result.py", line 114, in build
    tools[name] = _build_tool(response_type, name, False)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/.venv/lib/python3.12/site-packages/pydantic_ai/_result.py", line 104, in _build_tool
    return cast(ResultTool[T], ResultTool(a, tool_name_, description, multiple))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/.venv/lib/python3.12/site-packages/pydantic_ai/_result.py", line 162, in __init__
    parameters_json_schema = _utils.check_object_json_schema(self.type_adapter.json_schema())
                                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nate/github.com/prefecthq/marvin/.venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 617, in json_schema
    self.core_schema.rebuild()
  File "/Users/nate/github.com/prefecthq/marvin/.venv/lib/python3.12/site-packages/pydantic/_internal/_mock_val_ser.py", line 67, in rebuild
    raise PydanticUserError(self._error_message, code=self._code)
pydantic.errors.PydanticUserError: `TypeAdapter[<class 'marvin.engine.end_turn.create_mark_task_successful.<locals>._MarkTaskSuccessful'>]` is not fully defined; you should define `<class 'marvin.engine.end_turn.create_mark_task_successful.<locals>._MarkTaskSuccessful'>` and all referenced types, then call `.rebuild()` on the instance.

For further information visit https://errors.pydantic.dev/2.10/u/class-not-fully-defined

this might end up being something we can fix downstream, but for now I'd just remove the __future__ import if possible or organize around it as you mention