Process API Improvements in .NET 11 - .NET Blog

The System.Diagnostics.Process class is the primary way to create and interact with processes with .NET. We made the biggest update to it in years, with .NET 11. The changes add high-level APIs that make it easy to start a process and capture its output without deadlocks, give you full control over handle inheritance and standard handle redirection, introduce lifetime management features like KillOnParentExit, and include a lightweight SafeProcessHandle-based API surface that is more trimmer-friendly.
Here’s a recap of new Process APIs in .NET 11:
Other improvements include:
The rest of the blog post is a deep dive into each of these features.
When redirecting standard output and error of the process, it’s possible to run into deadlocks. Knowing why it’s possible is crucial to understanding the changes we have made. Let’s build a C# app that tries to read all output and error from a process.
First of all, we need to redirect standard output and error of the process to be able to read it. This is done by setting RedirectStandardOutput and RedirectStandardError properties of ProcessStartInfo to true. Then before the process is started, two dedicated pipes are created (one for standard output and one for standard error), and the process is created with the write ends of these pipes (each pipe comes with a read and write end) as standard output and error. The child process just writes to its standard output and error as usual, but instead of going to the console, the data is written to the pipes.
Pipes have limited buffer size (usually 4KB on Windows and 64KB on Unix). When the producer (in our case, the child process) writes to the pipe, the data is stored in a buffer until it’s read by the consumer (the parent process). If the producer writes more data than the size of the buffer and the consumer is not reading from the pipe at the same time, the producer will be blocked on write operation, waiting for the consumer to read from the pipe and free some space in the buffer.
If the consumer is waiting for the producer to exit (e.g. by calling WaitForExit) without reading from the pipe, it will be blocked as soon as the producer fills the buffer:
But does re-ordering the code help?
Not really. ReadToEnd is a blocking call – it reads until the stream reaches EOF, which only happens when the child process closes its end of the pipe (typically on exit). So in the code above, we first block on standard output until the child exits, and only then start reading standard error. While we’re waiting on standard output, nobody is draining standard error. If the child writes more to stderr than the pipe buffer can hold, the child blocks on its write – and we’re stuck waiting for each other.
The root cause is that we are reading the two streams sequentially, not simultaneously. To avoid this deadlock, we need to drain both standard output and error at the same time. So far, we had two options:
The existing APIs are not optimal in terms of simplicity and performance. I’ll show you a couple patterns.
The Process class exposes stream readers that you can read from, for example, with ReadToEndAsync.
These Process class events are raised when a line is written to standard output and error respectively.
We have added ReadAllText and ReadAllTextAsync (PR) methods to Process class. They drain standard output and error at the same time, helping us to avoid deadlocks. They decode the output using the encoding specified by the ProcessStartInfo.Standard[Output/Error]Encoding (or the default when not specified), and return the result as strings (per-line needs are handled by an API we’ll see shortly).
So the code to read all output and error from the process becomes much simpler:
We expect that capturing output and error and just waiting for the process to exit (process can close standard handles and keep running) is very common. That is why we have introduced RunAndCaptureText and RunAndCaptureTextAsync methods that combine starting the process, reading all output and error, and waiting for the process to exit in one method call.
Which makes the code to start a process and capture its output and error literally a one liner:
We also provided Process.Run and Process.RunAsync methods that don’t capture output and error, but just wait for the process to exit, for cases when you don’t care about the output.
If you need to capture output and error as lines, you can use ReadAllLines (PR) and ReadAllLinesAsync (PR) methods, which are implemented in the same way as their ReadAllText counterparts, but return an enumerable of ProcessOutputLine instead of a single string.
Let’s see how we can use it to read output and error from the process line by line as they are produced:
If you need to capture output and error as bytes, you can use ReadAllBytes method, which is internally used by ReadAllText (PR). It returns byte arrays instead of strings.
All the aforementioned methods that read from standard output and error support timeouts and cancellation. If the timeout is reached or the cancellation token is cancelled before the end of the stream is reached, the methods will throw TimeoutException or OperationCanceledException respectively. The high-level RunAndCaptureText[Async] and Run[Async] methods will also try to kill the process to avoid leaving it running.
The new methods are not just simpler to use, but also faster. Behind the scenes, the synchronous Process.RunAndCaptureText and Process.ReadAll[Bytes/Text] methods use multiplexing (poll on Unix and WaitForMultipleObjects on Windows) to read from both standard output and error using a single thread. They also implement a bunch of other optimizations such as using ArrayPool to reduce memory allocations. The asynchronous Process.RunAndCaptureTextAsync and Process.ReadAllTextAsync methods use asynchronous I/O operations without blocking any threads.
As you can see, on Windows the synchronous RunAndCaptureText method is about 2-3 ms faster than the old approaches and allocates abo