Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with Integrated vectorization enabled #2383

Open
TB-Shyam opened this issue Feb 26, 2025 · 7 comments · May be fixed by #2470
Open

Issue with Integrated vectorization enabled #2383

TB-Shyam opened this issue Feb 26, 2025 · 7 comments · May be fixed by #2470
Assignees
Labels
bug Something isn't working

Comments

@TB-Shyam
Copy link

I enabled integrated vectorization in the environment variables. Deleted the old index and run azd provison. I get the below error.

Image

It states that Field mapping specifies target field 'title' that is not present in the index.

I checked the searchmanager.py file and that index schema doesnt have the title field in it.

I have two options:

We have two options:
a) Add a "title" field to the index schema in searchmanager.py
b) Change the field mapping in integratedvectorizerstrategy.py to map to an existing field like "sourcepage" or "sourcefile"

Which way would you recommend me to proceed with?

@Bennzn
Copy link

Bennzn commented Feb 27, 2025

I receive the same error message (seems the last update of the repo?) even when I try to create a fresh new index using prepdocs.py or using azd up. Also with a complete fresh initialized project and start it up from scratch.
Tried on 2 different clients, changed python version ecc.

Or there was some change on the backend of the AI Search service? As there is a HttpResponseError

I compared the relevant py files with an older project that I spun up 2 months ago without a problem, but didn't find some difference

Cheers,
Ben

@Bennzn
Copy link

Bennzn commented Mar 5, 2025

Side fact:
On an existing deployment which was working very well, now the existing Indexer throws the same error.
We did not redeploy or other. It was a "productive" demo.
New the indexer fails always with "Field mapping specifies target field 'title' that is not present in the target index" like during a new / fresh deployment.

So for me it seems there was a change in the backend of AI Search.

EDIT:

It is / was a change on the backend:
https://learn.microsoft.com/en-us/answers/questions/2182611/ai-search-indexer-started-failing-with-field-mappi

When you change in integratedvectorizerstrategy.py line 175:

from:
field_mappings=[FieldMapping(source_field_name="metadata_storage_name", target_field_name="title")],

to:
field_mappings=[FieldMapping(source_field_name="metadata_storage_name", target_field_name="sourcefile")],

The deployment succeed.

Need to check now the impact on existing indexes and change the field mapping

@TB-Shyam
Copy link
Author

TB-Shyam commented Mar 7, 2025

Thank you so much. In my case I have added the title feild in the index. It works for me in that way

@egor-yudkin
Copy link
Contributor

I believe changing the line 175 in integratedvectorizerstrategy.py as @Bennzn suggested is the right approach.
The rest of the application doesn't use field "title", but uses "sourcefile" in some operations such as updating content in index.

@TB-Shyam you may want to test if the index is being updated properly after your change.

@pamelafox pamelafox added the bug Something isn't working label Apr 2, 2025
@pamelafox pamelafox self-assigned this Apr 2, 2025
@pamelafox
Copy link
Collaborator

I am fixing this in upcoming PR to port to embedding-3 models:
https://github.com/Azure-Samples/azure-search-openai-demo/pull/2470/files#diff-d1555d6523fdd0048963ceb0b11ef106a227cb9838b956eb57e80b1ea5838349R111

But you can also bring in those changes to the mapping. We also added storageUrl as it was easy to do.

@TB-Shyam
Copy link
Author

TB-Shyam commented Apr 8, 2025

@pamelafox Could you please guide me how I can bring this changes so its fixes the integrated vectorization issue that I am currently facing. Thank you

@cforce
Copy link
Contributor

cforce commented Apr 8, 2025

Please merge the PR with the fix included or at least merge the below quick fix because without app is not runnable (prepdocs fails which is mandatory for app deployment)

from:

field_mappings=[FieldMapping(source_field_name="metadata_storage_name", target_field_name="title")],

to:

field_mappings=[FieldMapping(source_field_name="metadata_storage_name", target_field_name="sourcefile")],

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
5 participants