[WIP] Add typing for `pytorch_lightning.utilities` #7218

stancld · 2021-04-26T20:23:55Z

What does this PR do?

Part of #7037

Before submitting

Was this discussed/approved via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is free to review the PR once the tests have passed.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃

…lities

* [WIP] Need to verify if logged metrics are of a type torch.Tensor or whether Any can be passed in

* Files: seed, upgrade_checkpoint, warnings * Fix typo in parsing

…lities

pep8speaks · 2021-04-26T20:24:02Z

Hello @stancld! Thanks for updating this PR.

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-05-27 21:20:42 UTC

codecov · 2021-04-26T20:38:25Z

Codecov Report

Merging #7218 (eb902c7) into master (9097347) will decrease coverage by 48%.
The diff coverage is 80%.

@@           Coverage Diff            @@
##           master   #7218     +/-   ##
========================================
- Coverage      92%     44%    -48%     
========================================
  Files         199     199             
  Lines       12996   13014     +18     
========================================
- Hits        12003    5726   -6277     
- Misses        993    7288   +6295

pytorch_lightning/utilities/device_parser.py

Borda · 2021-05-26T08:55:26Z

pytorch_lightning/utilities/distributed.py

-        hook=ddp_comm_hook,
-    )
+    # If condition required for mypy compatibility
+    if ddp_comm_hook is not None:


rather allow passing None, or what happen if none is passed?

If this condition is dropped, the following mypy issue is raised:

utilities/distributed.py:320: error: Item "None" of "Optional[Callable[..., Any]]" has no attribute "__qualname__"

pytorch_lightning/utilities/memory.py

Borda · 2021-05-26T09:00:15Z

pytorch_lightning/utilities/parsing.py

-        return collect_init_args(frame.f_back, path_args, inside=True)
-    elif not inside:
-        return collect_init_args(frame.f_back, path_args, inside)
+    if isinstance(frame.f_back, FrameType):


mind comment on why this?

Dropping this condition leads to the following two issues:

utilities/parsing.py:174: error: Argument 1 to "collect_init_args" has incompatible type "Optional[FrameType]"; expected "FrameType" utilities/parsing.py:176: error: Argument 1 to "collect_init_args" has incompatible type "Optional[FrameType]"; expected "FrameType"

Maybe, there's another nice workaround.

pytorch_lightning/utilities/device_parser.py

for more information, see https://pre-commit.ci

stancld · 2021-06-24T21:23:52Z

@Borda Hi, sorry for not responding. This PR looks now obsolete and pretty complicated to merge due to its large extent. Would you mind if I close this PR and maybe open some little PRs for individual files to finish this mypy task?

awaelchli · 2021-06-24T21:29:38Z

If it's possible to break down in smaller pieces that would be awesome! very welcomed!
a smaller, incremental PRs would make it easier to review, lower the changes of conflicts and increase the changes to get merged <3

stancld added 18 commits April 26, 2021 13:06

Add typing for apply_func

ca39eb5

Add typing for argparse'

4146be6

Add typing for cli

5582fad

Merge remote-tracking branch 'upstream/master' into typing_for_pl-uti…

1282cd2

…lities

Add typing for cloud_io

5a432dd

Add typing for data and debugging

71da685

* [WIP] Need to verify if logged metrics are of a type torch.Tensor or whether Any can be passed in

Add typing for device_dtype_mixin.py

31a8315

Add typing for device_parser

cf1240c

Add typing for distributed

65d3bd5

Add typing for imports

e67d4c7

Add typing for memory

a854722

Add typing for metrics

ff7c24a

Add typing for model_helpers

bbb01c4

Add typing for parsing

b2cdfc1

Add typing for parsing

900d994

Add typing to multiple files and fix a typo

24b0125

* Files: seed, upgrade_checkpoint, warnings * Fix typo in parsing

Add typing for xla_device

7ca6024

Merge remote-tracking branch 'upstream/master' into typing_for_pl-uti…

eae8620

…lities

stancld requested review from Borda, carmocca, SeanNaren and tchaton as code owners April 26, 2021 20:23

Add missing whitespace afer ':' in parsing.py

7576c45

mergify bot added the has conflicts label Apr 27, 2021

Merge branch 'master' into typing_for_pl-utilities

ab50848

mergify bot added has conflicts and removed has conflicts labels Apr 28, 2021

Add some missing typing

2cee78f

Merge 'upstream/master' into typing_for_pl-utilities

4ee6a8f

Borda removed the ready PRs ready to be merged label May 19, 2021

Add back deleted MisconfigurationException (device)

9a71d85

mergify bot added the has conflicts label May 24, 2021

stancld added 2 commits May 25, 2021 08:10

Merge upstream/master into typing_for_pl-utilities

38683f3

[WIP] Tackle some other mypy issues

b7f9ca7

stancld changed the title ~~Add typing for pytorch_lightning.utilities~~ [WIP] Add typing for pytorch_lightning.utilities May 25, 2021

mergify bot removed the has conflicts label May 25, 2021

Borda reviewed May 26, 2021

View reviewed changes

stancld and others added 3 commits May 27, 2021 23:19

Fix some issues after a review

d2e06b3

Merge 'upstream/master' into typing_for_pl-utilities

e8db47e

[pre-commit.ci] auto fixes from pre-commit.com hooks

eb902c7

for more information, see https://pre-commit.ci

mergify bot added the has conflicts label May 30, 2021

This was referenced Jun 25, 2021

Fix mypy in utilities.argparse #8124

Merged

Fix mypy in utilities.device_dtype_mixin #8127

Merged

Fix mypy in utilities.parsing #8132

Merged

stancld closed this Jun 25, 2021

This was referenced Jun 25, 2021

Fix mypy in utilities.device_parser #8136

Merged

Fix mypy in utilities.distributed #8201

Merged

stancld mentioned this pull request Sep 11, 2021

[WIP] Fix mypy typing for utilities.auto_restart #9454

Closed

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] Add typing for `pytorch_lightning.utilities` #7218

[WIP] Add typing for `pytorch_lightning.utilities` #7218

Uh oh!

stancld commented Apr 26, 2021 •

edited by awaelchli

Loading

Uh oh!

pep8speaks commented Apr 26, 2021 •

edited

Loading

Uh oh!

codecov bot commented Apr 26, 2021 •

edited

Loading

Uh oh!

Uh oh!

Borda May 26, 2021

Uh oh!

stancld May 27, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Borda May 26, 2021

Uh oh!

stancld May 27, 2021

Uh oh!

Uh oh!

stancld commented Jun 24, 2021

Uh oh!

awaelchli commented Jun 24, 2021

Uh oh!

Uh oh!

[WIP] Add typing for pytorch_lightning.utilities #7218

[WIP] Add typing for pytorch_lightning.utilities #7218

Uh oh!

Conversation

stancld commented Apr 26, 2021 • edited by awaelchli Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

PR review

Did you have fun?

Uh oh!

pep8speaks commented Apr 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2021-05-27 21:20:42 UTC

Uh oh!

codecov bot commented Apr 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Borda May 26, 2021

Choose a reason for hiding this comment

Uh oh!

stancld May 27, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Borda May 26, 2021

Choose a reason for hiding this comment

Uh oh!

stancld May 27, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stancld commented Jun 24, 2021

Uh oh!

awaelchli commented Jun 24, 2021

Uh oh!

Uh oh!

[WIP] Add typing for `pytorch_lightning.utilities` #7218

[WIP] Add typing for `pytorch_lightning.utilities` #7218

stancld commented Apr 26, 2021 •

edited by awaelchli

Loading

pep8speaks commented Apr 26, 2021 •

edited

Loading

codecov bot commented Apr 26, 2021 •

edited

Loading