Skip to content

Mark Trainer.run_stage as protected #11000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 121 commits into from
Dec 17, 2021
Merged
Show file tree
Hide file tree
Changes from 119 commits
Commits
Show all changes
121 commits
Select commit Hold shift + click to select a range
d05363f
improve spawn queue
awaelchli Oct 20, 2021
d650e26
clean up
awaelchli Oct 20, 2021
5fda23a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 20, 2021
d6b4a34
Merge branch 'master' into feature/simple-spawn
awaelchli Nov 30, 2021
bcfb853
fix
awaelchli Nov 30, 2021
97b4bf6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 30, 2021
38b3a54
rename
awaelchli Nov 30, 2021
955b6c8
delete dead code
awaelchli Nov 30, 2021
13393e8
Merge remote-tracking branch 'origin/feature/simple-spawn' into featu…
awaelchli Nov 30, 2021
f3216b2
clean up
awaelchli Nov 30, 2021
2d00231
update lite
awaelchli Nov 30, 2021
7aa3646
retain the queue interface in hooks
awaelchli Nov 30, 2021
fb0c0d8
update tests
awaelchli Nov 30, 2021
1bc59ae
Merge branch 'master' into feature/simple-spawn
awaelchli Nov 30, 2021
7e6c75e
_notebooks
awaelchli Nov 30, 2021
b7efc50
reset notebooks
awaelchli Nov 30, 2021
84ca8b4
avoid circular import
awaelchli Nov 30, 2021
965c724
fix unused imports
awaelchli Nov 30, 2021
1aae8dd
reset debugging script
awaelchli Nov 30, 2021
4b998db
typing _ExtraQueue
awaelchli Nov 30, 2021
5871a4b
bring changes to tpu_spawn plugin
awaelchli Nov 30, 2021
aa76840
unify
awaelchli Nov 30, 2021
37f9db9
remove dead code
awaelchli Nov 30, 2021
d68cb35
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 30, 2021
dd80be9
remove queue from tpu spawn
awaelchli Nov 30, 2021
f97eee8
type annotation for new_process
awaelchli Nov 30, 2021
ad61d27
Merge remote-tracking branch 'origin/feature/simple-spawn' into refac…
awaelchli Nov 30, 2021
459121e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 30, 2021
72535ff
unused imports
awaelchli Nov 30, 2021
3095da9
Merge remote-tracking branch 'origin/feature/simple-spawn' into refac…
awaelchli Nov 30, 2021
61192df
move check
awaelchli Nov 30, 2021
801f529
revert
awaelchli Nov 30, 2021
1cd258b
collect results on tpu
awaelchli Nov 30, 2021
ae6019e
Merge branch 'master' into refactor/spawn/simple-spawn
awaelchli Nov 30, 2021
10ecbfd
rename
awaelchli Nov 30, 2021
ebba63f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 30, 2021
d7df4d9
fix merge errors
awaelchli Nov 30, 2021
4c547aa
fix merge errors
awaelchli Nov 30, 2021
e4e2a77
re-add clean_logger
awaelchli Dec 1, 2021
86e43b2
Merge branch 'master' into refactor/spawn/simple-spawn
awaelchli Dec 1, 2021
acac29d
fix typing
awaelchli Dec 1, 2021
0ae457a
Merge branch 'master' into refactor/spawn/simple-spawn
awaelchli Dec 1, 2021
880c8fc
changelog entries
awaelchli Dec 1, 2021
5eeb02a
Merge branch 'master' into refactor/spawn/simple-spawn
awaelchli Dec 1, 2021
7520adc
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 1, 2021
96f2749
rename _ExtraQueue -> _FakeQueue
awaelchli Dec 1, 2021
65d183c
missing typing updates
awaelchli Dec 1, 2021
8c4e2e4
Introducing NamedTuple for spawn output typing
awaelchli Dec 1, 2021
213b447
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 1, 2021
de4617f
remove post_dispatch
awaelchli Dec 2, 2021
815172e
step 1
awaelchli Dec 2, 2021
be735bd
update flow
awaelchli Dec 2, 2021
2879ccb
fix it
awaelchli Dec 2, 2021
ace196e
jackpot!
awaelchli Dec 2, 2021
4ff41a9
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 2, 2021
34a889a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 2, 2021
ad3f39d
update sharded and tests
awaelchli Dec 2, 2021
c897a20
pull down spawn call
awaelchli Dec 2, 2021
90054cf
simplify test
awaelchli Dec 2, 2021
009abfa
attach model as early as possible
awaelchli Dec 2, 2021
376e4fe
demonstrate which tests fails
awaelchli Dec 2, 2021
de1811e
set module
awaelchli Dec 3, 2021
ef61a0b
update exception
awaelchli Dec 3, 2021
809014a
imports
awaelchli Dec 3, 2021
440b639
transfer trainer state
awaelchli Dec 3, 2021
ab5559e
fix problem with getqueue
awaelchli Dec 3, 2021
f4f1269
deprecation calls don't come through ddp_spawn
awaelchli Dec 3, 2021
b30c352
prepare data only gets called on rank 0
awaelchli Dec 3, 2021
5434ae5
import
awaelchli Dec 3, 2021
24f05f1
update test
awaelchli Dec 3, 2021
3959955
update exception
awaelchli Dec 3, 2021
f491abe
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 3, 2021
0c808ce
adapt tpu spawn
awaelchli Dec 3, 2021
d6dd343
imports
awaelchli Dec 3, 2021
63e6e21
Merge remote-tracking branch 'origin/refactor/spawn/dispatch' into re…
awaelchli Dec 3, 2021
15dabb8
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 3, 2021
b047687
update
awaelchli Dec 3, 2021
c524e52
add missing arg
awaelchli Dec 3, 2021
223e7aa
fix exception import on torch < 1.8
awaelchli Dec 3, 2021
ed309d6
debug
awaelchli Dec 3, 2021
12eed61
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 3, 2021
9401e66
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 3, 2021
be73261
debug tpu
awaelchli Dec 3, 2021
c71fc57
fix docs
awaelchli Dec 3, 2021
2ed6333
fix teardown being called twice
awaelchli Dec 3, 2021
2a8b9b4
revert a sate check
awaelchli Dec 3, 2021
5335664
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 3, 2021
5c7e159
Merge remote-tracking branch 'origin/refactor/spawn/dispatch' into re…
awaelchli Dec 3, 2021
93cfaf8
fix
awaelchli Dec 3, 2021
26408b8
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 4, 2021
70b332d
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 6, 2021
d9669c7
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 6, 2021
3663bd7
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 6, 2021
3d81c11
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 6, 2021
fb47802
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 6, 2021
dde5a3a
reset bug report model
awaelchli Dec 6, 2021
77329b2
fix merge error
awaelchli Dec 6, 2021
eb05fc9
barrier clean ups
awaelchli Dec 7, 2021
dbcb76c
update comments in trainer
awaelchli Dec 7, 2021
ed0defa
unused import
awaelchli Dec 7, 2021
79975f2
debug
awaelchli Dec 7, 2021
d5ec0b7
changelog
awaelchli Dec 7, 2021
b2f8347
update changelog
awaelchli Dec 7, 2021
d8e6218
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 7, 2021
436572b
update changelog
awaelchli Dec 7, 2021
a3bc1b1
Update tests/trainer/test_trainer.py
awaelchli Dec 7, 2021
b2ce8eb
Merge remote-tracking branch 'origin/refactor/spawn/dispatch' into re…
awaelchli Dec 7, 2021
bafd95c
add clarification comment
awaelchli Dec 8, 2021
338605a
update changelog
awaelchli Dec 8, 2021
c992a55
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 8, 2021
ac1428d
skip test that can't run on too old torch version on windows
awaelchli Dec 8, 2021
77ee0ef
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 8, 2021
c7dd23d
remove todo
awaelchli Dec 8, 2021
18f32bc
Merge branch 'master' into refactor/spawn/dispatch
awaelchli Dec 9, 2021
8af6d48
remove obsolete start_* methods from TTP
awaelchli Dec 9, 2021
f181c59
update changelog
awaelchli Dec 9, 2021
ec2e852
Deprecate Trainer.run_stage
awaelchli Dec 9, 2021
95b6854
add changelog
awaelchli Dec 9, 2021
0b37730
Merge branch 'master' into refactor/protected-run-stage
awaelchli Dec 15, 2021
df02fbb
Update pytorch_lightning/trainer/trainer.py
awaelchli Dec 15, 2021
fa0a532
Merge branch 'master' into refactor/protected-run-stage
carmocca Dec 17, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Deprecated `ModelIO.on_hpc_{save/load}` in favor of `CheckpointHooks.on_{save/load}_checkpoint` ([#10911](https://github.com/PyTorchLightning/pytorch-lightning/pull/10911))


- Deprecated `Trainer.run_stage` in favor of `Trainer.{fit,validate,test,predict}` ([#11000](https://github.com/PyTorchLightning/pytorch-lightning/pull/11000))


### Removed

- Removed deprecated parameter `method` in `pytorch_lightning.utilities.model_helpers.is_overridden` ([#10507](https://github.com/PyTorchLightning/pytorch-lightning/pull/10507))
Expand Down
13 changes: 10 additions & 3 deletions pytorch_lightning/trainer/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -1129,7 +1129,7 @@ def _run(
setup accelerator ||
and strategy || LIGHTNING
| ||
{self.run_stage} || FLOW
{self._run_stage} || FLOW
| ||
{self._run_train} || DIRECTION
or {self._run_evaluate} ||
Expand Down Expand Up @@ -1164,7 +1164,7 @@ def _run(

self.checkpoint_connector.resume_end()

results = self.run_stage()
results = self._run_stage()
self._teardown()

# ----------------------------
Expand Down Expand Up @@ -1237,7 +1237,14 @@ def _teardown(self):
self.logger_connector.teardown()
self.signal_connector.teardown()

def run_stage(self):
def run_stage(self) -> None:
rank_zero_deprecation(
"`Trainer.run_stage` is deprecated in v1.6 and will be removed in v1.8. Use"
" `Trainer.{fit,validate,test,predict}` instead."
)
return self._run_stage()

def _run_stage(self):
self.training_type_plugin.barrier("run-stage")
self.training_type_plugin.dispatch(self)
self.__setup_profiler()
Expand Down
8 changes: 8 additions & 0 deletions tests/deprecated_api/test_remove_1-8.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
"""Test deprecated functionality which will be removed in v1.8.0."""
from unittest.mock import Mock

import pytest
import torch
Expand Down Expand Up @@ -106,3 +107,10 @@ def on_hpc_load(self):
match=r"Method `LightningModule.on_hpc_load` is deprecated in v1.6 and will be removed in v1.8."
):
trainer.fit(load_model)


def test_v1_8_0_deprecated_run_stage():
trainer = Trainer()
trainer._run_stage = Mock()
with pytest.deprecated_call(match="`Trainer.run_stage` is deprecated in v1.6 and will be removed in v1.8."):
trainer.run_stage()