Skip to content

Commit 46f6b80

Browse files
authored
[BugFix] hyper parallel failed with > 128 workers (#5765)
1 parent 2b92fd2 commit 46f6b80

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

paddlenlp/trainer/trainer_utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -890,7 +890,7 @@ def set_hyrbid_parallel_seed(basic_seed, dataset_rank, tp_rank, pp_rank=0):
890890
paddle.seed(basic_seed + dataset_rank)
891891

892892
# local_seed/ global_seed is used to control dropout in ModelParallel
893-
local_seed = basic_seed + 123 + tp_rank * 10 + pp_rank * 1000
893+
local_seed = basic_seed + 59999 + tp_rank * 10 + pp_rank * 1000
894894
global_seed = basic_seed + dataset_rank
895895
tracker = get_rng_state_tracker()
896896
tracker.add("global_seed", global_seed)

0 commit comments

Comments
 (0)