Requirements update

[justusschock]

What does this PR do?

This PR adds a resumable iterable dataset (LightningIterableDataset)

Before submitting

• Was this discussed/agreed via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @Borda @carmocca @justusschock @awaelchli

[github-actions]

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs)	success	✅

These checks are required after the changes to src/lightning/__init__.py.

🟢 lightning_data: CPU workflow

Check ID	Status
data-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
data-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
data-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅

These checks are required after the changes to src/lightning/data/__init__.py, src/lightning/data/datasets/__init__.py, src/lightning/data/datasets/base.py, src/lightning/data/datasets/env.py, src/lightning/data/datasets/index.py, src/lightning/data/datasets/iterable.py, src/lightning/data/datasets/mapping.py, tests/tests_data/conftest.py, tests/tests_data/datasets/__init__.py, tests/tests_data/datasets/test_data/test_index.txt, tests/tests_data/datasets/test_data/test_index_s3.txt, tests/tests_data/datasets/test_env.py, tests/tests_data/datasets/test_get_index.py, tests/tests_data/datasets/test_iterable.py, tests/tests_data/datasets/test_mapping.py, .github/workflows/ci-tests-data.yml.

🟢 lightning_fabric: Azure GPU

Check ID	Status
lightning-fabric (GPUs)	success	✅

These checks are required after the changes to src/lightning/__init__.py.

🟢 lightning: Store

Check ID	Status
Store.cloud (test_store Linux)	success	✅
Store.cloud (test_store Mac)	success	✅
Store.cloud (test_store Windows)	success	✅

These checks are required after the changes to src/lightning/__init__.py.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to src/lightning/__init__.py, src/lightning/data/__init__.py, src/lightning/data/datasets/__init__.py, src/lightning/data/datasets/base.py, src/lightning/data/datasets/env.py, src/lightning/data/datasets/index.py, src/lightning/data/datasets/iterable.py, src/lightning/data/datasets/mapping.py.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.10)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.10)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.10)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.10)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.10)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.10)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.10)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.10)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.10)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.10)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.10)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.10)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.10)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.10)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.10)	success	✅

These checks are required after the changes to src/lightning/__init__.py, src/lightning/data/__init__.py, src/lightning/data/datasets/__init__.py, src/lightning/data/datasets/base.py, src/lightning/data/datasets/env.py, src/lightning/data/datasets/index.py, src/lightning/data/datasets/iterable.py, src/lightning/data/datasets/mapping.py.

🟢 link-check

Check ID	Status
check-md-links / markdown-link-check	success	✅

These checks are required after the changes to src/lightning/data/CHANGELOG.md.

---

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

[codecov]

Codecov Report

Merging #17998 (3273ae5) into master (b14ddd9) will decrease coverage by 22%.
The diff coverage is 79%.

❗ Current head 3273ae5 differs from pull request most recent head 5429c00. Consider uploading reports for the commit 5429c00 to get more accurate results

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #17998     +/-   ##
=========================================
- Coverage      84%      61%    -22%     
=========================================
  Files         425      426      +1     
  Lines       32076    32369    +293     
=========================================
- Hits        26834    19890   -6944     
- Misses       5242    12479   +7237     

[nohalon]

lgtm

[awaelchli]

Minor nits and questions. Great work!