Pydantic-based response schema not followed consistently, but dictionary-based schema is.

[suprateembanerjee]

Description of the bug:

I'm trying to generate slide deck content following a certain schema.

Pydantic Schema:

class SlideSchema(BaseModel):
    header: str
    subheader: str | None
    body: str | None
    image: str | None
    teachers_notes: str | None

Actual vs expected behavior:

The following code is highly inconsistent (2/10 success):

model = genai.GenerativeModel('gemini-1.5-flash')
response = model.generate_content(
                slides_prompt,
                generation_config={
                        'response_mime_type': 'application/json',
                        'response_schema': list[SlideSchema]})

However, the following code is highly consistent (10/10 success), although less readable and less structured:

model = genai.GenerativeModel('gemini-1.5-flash')
response = model.generate_content(
                slides_prompt,
                generation_config={
                        'response_mime_type': 'application/json',
                        'response_schema': {
                                'type': 'ARRAY',
                                'items': {
                                        'type': 'OBJECT',
                                        'properties': {
                                                'header': {'type': 'STRING', 'nullable': False},
                                                'subheader': {'type': 'STRING', 'nullable': True},
                                                'body': {'type': 'STRING', 'nullable': True},
                                                'image': {'type': 'STRING', 'nullable': True},
                                                'teachers_notes': {'type': 'STRING', 'nullable': True} },
                                        'required': ['header', 'subheader', 'body', 'image', 'teachers_notes']
                                        }
                                }
                        }

Any other information you'd like to share?

How do we incorporate the expected consistent behavior using Pydantic schema?

[manojssmk]

Hi @suprateembanerjee

What you mentioned is correct. The output is inconsistent when using the Pydantic model. Here’s the gist with the code.

Thanks

[MarkDaoust]

This could be a difference in how it handles "nullable" and "required". You can see what dictionary the pydantic schema gets converted to using this function.

https://github.com/google-gemini/generative-ai-python/blob/a7e19280352017892ee39c2b4b9d1be0916bcb7b/google/generativeai/types/content_types.py#L337

from google.generativeai.types import content_types 

class SlideSchema(BaseModel):
        header: str
        subheader: str | None
        body: str | None
        image: str | None
        teachers_notes: str | None

content_types._scehma_from_class(SlideSchema)

{'items': {'properties': {'header': {'type': 'string'},
   'subheader': {'type': 'string', 'nullable': True},
   'body': {'type': 'string', 'nullable': True},
   'image': {'type': 'string', 'nullable': True},
   'teachers_notes': {'type': 'string', 'nullable': True}},
  'type': 'object'},
 'type': 'array'}

It looks like this SDK is not including the required.

The SDK for Gemini-2+, is currently working on better support, I think this issue is fixed there.

[MarkDaoust]

So this is a duplicate of: #642