Skip to content

Fix issue with transformers library huggingface #11027

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

alexcbb
Copy link

@alexcbb alexcbb commented Mar 12, 2025

Description

I was trying to modifiy a RLDS dataset built on top of TFDS following this repo : https://github.com/kpertsch/rlds_dataset_builder
I needed to extract some features from images with models from the transformers library of HuggingFace but was facing an issue during the import : raise ValueError('{}.__spec__ is None'.format(name)) ValueError: datasets.__spec__ is None

And more specifically this one : transformers/utils/import_utils.py", line 120, in <module> _datasets_available = _is_package_available("datasets")

It verifies if datasets (the HF library) is available by looking at the __spec__ attribute. As tfds is overwritting datasets by a mock, it does not create the attribute, which causes the issue.

In the PR I fixed the issue by simply creating the needed attribute in the Mock in order to solve the problem.

Here are the versions of the libraries involved in the problem :
tensorflow-datasets 4.9.3
transformers 4.50.0.dev0

Transformers library in the last versions verify if datasets contains a _spec_ to check if it is available
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant