Skip to content

Commit c1ce74d

Browse files
ivbergrachguo
authored and
rachguo
committedFeb 8, 2024
Windows - Only set thread affinity on Server with auto affinity (#19318)
### Description Only set thread affinity on Server with auto affinity. Auto affinity = when API user does specify thread settings or affinity themselves. ### Motivation and Context On client best to let OS scheduler handle. On big (P-Core) / little (E-Core) CPU designs affinity overrides win32 Quality of Service (QoS) and has high power usage. Specifically on background workloads whose process is tagged QoS Utility (Background), this affinity setting overrides the OS scheduler that only wants to schedule on the E-Cores. Thus P-Cores waking up uses more energy than intended on client and users gets less battery life. Foreground AI workloads would be tagged QoS High and would run the ORT threads on all cores.
1 parent 5269e93 commit c1ce74d

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed
 

‎onnxruntime/core/util/thread_utils.cc

+10
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77

88
#ifdef _WIN32
99
#include <Windows.h>
10+
#include <versionhelpers.h>
1011
#endif
1112
#include <thread>
1213
#include "core/session/ort_apis.h"
@@ -98,7 +99,16 @@ CreateThreadPoolHelper(Env* env, OrtThreadPoolParams options) {
9899
}
99100
options.thread_pool_size = static_cast<int>(default_affinities.size());
100101
if (options.auto_set_affinity) {
102+
#ifdef _WIN32
103+
// Only set thread affinity on Server with auto affinity.
104+
// On client best to let OS scheduler handle.
105+
// On big (P-Core) / little (E-Core) CPU designs affinity overrides QoS and has high power usage
106+
if (IsWindowsServer()) {
107+
to.affinities = std::move(default_affinities);
108+
}
109+
#else
101110
to.affinities = std::move(default_affinities);
111+
#endif
102112
}
103113
}
104114
if (options.thread_pool_size <= 1) {

0 commit comments

Comments
 (0)
Please sign in to comment.