Add work item dumping support to SOS' ThreadPool command

[stephentoub]

Adds a -wi switch to the ThreadPool command that will enumerate all queues dumping out all found work items.

For example, given this program:

using System;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        for (int i = 0; i < Environment.ProcessorCount; i++) Task.Run(QueueWork);
        QueueWork();
        Console.ReadLine();
    }

    static void QueueWork()
    {
        Task.Run(A);
        ThreadPool.QueueUserWorkItem(B, null);
        ThreadPool.QueueUserWorkItem(C, 42, true);
        Task.Run(D);
    }

    static void A() => Thread.CurrentThread.Join();
    static void B(object s) => Thread.CurrentThread.Join();
    static void C(int s) => Thread.CurrentThread.Join();
    static async Task D() { await Task.Yield(); Thread.CurrentThread.Join(); }
}

Each of the A, B, C, D methods block forever, and QueueWork creates independent work for each, so ultimately we should end up with (NumProcs + 1)*4 blocked threads. However the threadpool starts with only NumProc threads, so after that the pool only adds a thread slowly (2 every second), so if we stop reasonably quickly we should see things in the threadpool queue.

Running it and quickly attaching WinDBG/sos yielded the following output:

0:022> !threadpool -wi
CPU utilization: 19%
Worker Thread: Total: 11 Running: 11 Idle: 0 MaxLimit: 32767 MinLimit: 8
Work Request in Queue: 1
    AsyncTimerCallbackCompletion TimerInfo@000001d831d959b0

Queued work items:
           Queue          Address Work Item
        [Global] 000001d819a337d0 System.Threading.QueueUserWorkItemCallbackDefaultContext => Program.B(System.Object)
        [Global] 000001d819a38300 System.Threading.QueueUserWorkItemCallbackDefaultContext => Program.B(System.Object)
        [Global] 000001d819a3a308 System.Threading.QueueUserWorkItemCallbackDefaultContext => Program.B(System.Object)
        [Global] 000001d819a3c310 System.Threading.QueueUserWorkItemCallbackDefaultContext => Program.B(System.Object)
        [Global] 000001d819a3e318 System.Threading.QueueUserWorkItemCallbackDefaultContext => Program.B(System.Object)
        [Global] 000001d819a40320 System.Threading.QueueUserWorkItemCallbackDefaultContext => Program.B(System.Object)
        [Global] 000001d819a42328 System.Threading.QueueUserWorkItemCallbackDefaultContext => Program.B(System.Object)
        [Global] 000001d819a33830 System.Threading.QueueUserWorkItemCallbackDefaultContext`1[[System.Int32, System.Private.CoreLib]] => Program.C(Int32)
        [Global] 000001d819a33890 System.Threading.Tasks.Task`1[[System.Threading.Tasks.Task, System.Private.CoreLib]] => Program.D()
        [Global] 000001d819a40498 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+d__5, test]]
        [Global] 000001d819a36470 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+d__5, test]]
        [Global] 000001d819a3e490 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+d__5, test]]
        [Global] 000001d819a3c488 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+d__5, test]]
        [Global] 000001d819a3a480 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+d__5, test]]
        [Global] 000001d819a424a0 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+d__5, test]]
        [Global] 000001d819a38528 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+d__5, test]]
        [Global] 000001d819a34480 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+d__5, test]]
000001d819a34080 000001d819a34288 System.Threading.Tasks.Task => Program.A()
000001d819a36080 000001d819a36278 System.Threading.Tasks.Task => Program.A()
000001d819a38080 000001d819a38280 System.Threading.Tasks.Task => Program.A()
000001d819a3a080 000001d819a3a288 System.Threading.Tasks.Task => Program.A()
000001d819a3c080 000001d819a3c290 System.Threading.Tasks.Task => Program.A()
000001d819a3e080 000001d819a3e298 System.Threading.Tasks.Task => Program.A()
000001d819a40080 000001d819a402a0 System.Threading.Tasks.Task => Program.A()
000001d819a42080 000001d819a422a8 System.Threading.Tasks.Task => Program.A()
Statistics:
              MT    Count    TotalSize Class Name
00007ffee06b8e60        1           32 System.Threading.QueueUserWorkItemCallbackDefaultContext`1[[System.Int32, System.Private.CoreLib]]
00007fff34ecc928        1           72 System.Threading.Tasks.Task`1[[System.Threading.Tasks.Task, System.Private.CoreLib]]
00007fff34eddb70        7          224 System.Threading.QueueUserWorkItemCallbackDefaultContext
00007fff34eca440        8          512 System.Threading.Tasks.Task
00007ffee07814d8        8          896 System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[[System.Threading.Tasks.VoidTaskResult, System.Private.CoreLib],[Program+<D>d__5, test]]
Total 25 objects

--------------------------------------
Number of Timers: 1
--------------------------------------
Completion Port Thread:Total: 0 Free: 0 MaxFree: 16 CurrentLimit: 0 MaxLimit: 1000 MinLimit: 8

cc: @noahfalk, @mikem8361, @kouvel

[mikem8361]

Will this work or fail gracefully on older runtimes? When this is ported to the new diagnostic repo, it will need to either work or fail gracefully on 1.1, 2.0, 2.1, 2.2, etc. runtimes.

[davidfowl]

cc @pakrym @halter73 @anurse

[stephentoub]

Will this work or fail gracefully on older runtimes?

Everything is fully functional on 2.0+ (2.0, 2.1, 2.2, 3.0).

On 1.1, it'll show the contents of local queues only. The data structure for the global queue changed from 1.1 to 2.0 (it's fairly complicated in 1.1), and I don't think it's worth trying to allow SOS to enumerate its contents. But it won't crash or anything like that, it just won't find any objects to output for the global queue.

[mikem8361]

Sounds good. Thanks.

[stephentoub]

Thanks, Mike.