add `accelerator.is_available()` check #12104

jjenniferdai · 2022-02-24T22:15:33Z

What does this PR do?

part of #11449: Enable accelerator.is_available() check
reference: implemented in PR #11797

Does your PR introduce any breaking changes? If yes, please list them.

adds hardware check in Trainer (AcceleratorConnector) init

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

for more information, see https://pre-commit.ci

pytorch_lightning/trainer/connectors/accelerator_connector.py

four4fish

LGTM! Thank you for adding this

for more information, see https://pre-commit.ci

pytorch_lightning/trainer/connectors/accelerator_connector.py

tchaton

LGTM !

pytorch_lightning/trainer/connectors/accelerator_connector.py

rohitgr7

LGTM. Once others' suggestions are addressed.

for more information, see https://pre-commit.ci

pytorch_lightning/accelerators/cpu.py

pytorch_lightning/trainer/connectors/accelerator_connector.py

tests/trainer/properties/test_estimated_stepping_batches.py

tests/plugins/test_cluster_integration.py

tests/accelerators/test_ipu.py

tests/accelerators/test_accelerator_connector.py

…s.py Co-authored-by: ananthsub <[email protected]>

Co-authored-by: ananthsub <[email protected]>

pytorch_lightning/trainer/connectors/accelerator_connector.py

awaelchli · 2022-03-06T11:23:33Z

pytorch_lightning/trainer/connectors/accelerator_connector.py

@@ -475,6 +474,15 @@ def _set_parallel_devices_and_init_accelerator(self) -> None:
            accelerator_class = ACCELERATORS[self._accelerator_flag]
            self.accelerator = accelerator_class()  # type: ignore[abstract]

+        if not self.accelerator.is_available():


depending on the accelerator, we might not be able to instantiate it without it being available. The init should have the freedom to assume it is available or raise additional errors. IMO the check should have been before the instantiation.

jjenniferdai and others added 5 commits February 24, 2022 14:14

first commit

3ffe5bf

[pre-commit.ci] auto fixes from pre-commit.com hooks

4c0f4a7

for more information, see https://pre-commit.ci

update

9c642f9

doctest?

55b4995

codeblock instead

729a8aa

jjenniferdai changed the title ~~[wip] add accelerator.is_available() check~~ add accelerator.is_available() check Feb 25, 2022

four4fish reviewed Feb 25, 2022

View reviewed changes

pytorch_lightning/trainer/connectors/accelerator_connector.py Outdated Show resolved Hide resolved

jjenniferdai marked this pull request as ready for review February 25, 2022 18:41

jjenniferdai requested review from tchaton, SeanNaren, carmocca, Borda, edenlightning, awaelchli, williamFalcon, justusschock, kaushikb11 and rohitgr7 as code owners February 25, 2022 18:41

four4fish approved these changes Feb 25, 2022

View reviewed changes

jjenniferdai and others added 2 commits February 25, 2022 15:06

update msg

15d386c

[pre-commit.ci] auto fixes from pre-commit.com hooks

d36d852

for more information, see https://pre-commit.ci

ananthsub reviewed Feb 25, 2022

View reviewed changes

pytorch_lightning/trainer/connectors/accelerator_connector.py Outdated Show resolved Hide resolved

tchaton approved these changes Feb 28, 2022

View reviewed changes

pytorch_lightning/trainer/connectors/accelerator_connector.py Outdated Show resolved Hide resolved

carmocca approved these changes Feb 28, 2022

View reviewed changes

pytorch_lightning/trainer/connectors/accelerator_connector.py Outdated Show resolved Hide resolved

pytorch_lightning/trainer/connectors/accelerator_connector.py Outdated Show resolved Hide resolved

pytorch_lightning/trainer/connectors/accelerator_connector.py Outdated Show resolved Hide resolved

mergify bot added the ready PRs ready to be merged label Feb 28, 2022

rohitgr7 added this to the 1.6 milestone Feb 28, 2022

rohitgr7 added feature Is an improvement or enhancement refactor and removed feature Is an improvement or enhancement labels Feb 28, 2022

rohitgr7 approved these changes Feb 28, 2022

View reviewed changes

mergify bot added the has conflicts label Feb 28, 2022

Merge branch 'master' into acc-available-check

99b4ecf

mergify bot removed the has conflicts label Feb 28, 2022

pre-commit-ci bot and others added 4 commits February 28, 2022 22:16

[pre-commit.ci] auto fixes from pre-commit.com hooks

c69bcf7

for more information, see https://pre-commit.ci

update msg, hardware_name

2385855

[pre-commit.ci] auto fixes from pre-commit.com hooks

250dc2c

for more information, see https://pre-commit.ci

format

d76b7e0

jjenniferdai requested review from ananthsub, tchaton and carmocca March 1, 2022 00:46

kaushikb11 reviewed Mar 1, 2022

View reviewed changes

pytorch_lightning/accelerators/cpu.py Outdated Show resolved Hide resolved

ananthsub reviewed Mar 1, 2022

View reviewed changes

jjenniferdai and others added 8 commits March 1, 2022 17:46

format Update tests/trainer/properties/test_estimated_stepping_batche…

cd920ac

…s.py Co-authored-by: ananthsub <[email protected]>

format Update tests/plugins/test_cluster_integration.py

1e031f4

Co-authored-by: ananthsub <[email protected]>

format Update tests/accelerators/test_ipu.py

4c57f8a

Co-authored-by: ananthsub <[email protected]>

format Update tests/accelerators/test_accelerator_connector.py

8b632b4

Co-authored-by: ananthsub <[email protected]>

format Update tests/accelerators/test_accelerator_connector.py

2193484

Co-authored-by: ananthsub <[email protected]>

format Update tests/accelerators/test_accelerator_connector.py

fadb69b

Co-authored-by: ananthsub <[email protected]>

req base, update name, lowercase

31a54eb

update

67301d6

kaushikb11 approved these changes Mar 2, 2022

View reviewed changes

kaushikb11 added 2 commits March 2, 2022 14:43

Address reviews

70e9f71

Fix test

65b23b6

kaushikb11 enabled auto-merge (squash) March 2, 2022 09:45

carmocca approved these changes Mar 2, 2022

View reviewed changes

kaushikb11 merged commit 89d3756 into Lightning-AI:master Mar 2, 2022

rohitgr7 mentioned this pull request Mar 2, 2022

Trainer(accelerator="tpu") should raise exception if TPU not found #12047

Closed

ananthsub reviewed Mar 2, 2022

View reviewed changes

pytorch_lightning/trainer/connectors/accelerator_connector.py Show resolved Hide resolved

awaelchli reviewed Mar 6, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add `accelerator.is_available()` check #12104

add `accelerator.is_available()` check #12104

Uh oh!

jjenniferdai commented Feb 24, 2022 •

edited

Loading

Uh oh!

Uh oh!

four4fish left a comment

Uh oh!

Uh oh!

tchaton left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rohitgr7 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

awaelchli Mar 6, 2022

Uh oh!

Uh oh!

add accelerator.is_available() check #12104

add accelerator.is_available() check #12104

Uh oh!

Conversation

jjenniferdai commented Feb 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

PR review

Did you have fun?

Uh oh!

Uh oh!

four4fish left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tchaton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rohitgr7 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

awaelchli Mar 6, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

add `accelerator.is_available()` check #12104

add `accelerator.is_available()` check #12104

jjenniferdai commented Feb 24, 2022 •

edited

Loading