gh-47798: Docs: lead with run_pipeline for shell pipes; whatsnew entry

The "Replacing shell pipeline" recipe now recommends run_pipeline()
first and demotes the manual Popen chain to the streaming case.  Note
that PipelineError is a sibling of CalledProcessError, not a subclass.

Author: gpshead

Doc/library/subprocess.rst

@@ -448,6 +448,16 @@ underlying :class:`Popen` interface can be used directly.
    that returned a non-zero exit status. This is similar to the shell's
    ``pipefail`` behavior.
 
+   :exc:`PipelineError` is a *sibling* of :exc:`CalledProcessError`, not a
+   subclass: a pipeline carries N commands and N return codes, so there is
+   no single ``returncode`` or ``cmd`` value an existing
+   ``except CalledProcessError:`` handler could be shown without being
+   misleading.  To handle both single-command and pipeline failures with
+   one ``except`` clause, catch :exc:`SubprocessError` (which is also the
+   common base of :exc:`TimeoutExpired`).  Retry helpers and decorators
+   that match on :exc:`CalledProcessError` will not catch pipeline
+   failures by default.
+
    .. attribute:: commands
 
       List of commands that were used in the pipeline.
@@ -1606,24 +1616,32 @@ Replacing shell pipeline
 
 becomes::
 
-   p1 = Popen(["dmesg"], stdout=PIPE)
-   p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
-   p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
-   output = p2.communicate()[0]
+   result = run_pipeline(["dmesg"], ["grep", "hda"],
+                         capture_output=True, check=True)
+   output = result.stdout
 
-The ``p1.stdout.close()`` call after starting the p2 is important in order for
-p1 to receive a SIGPIPE if p2 exits before p1.
+:func:`run_pipeline` connects the stages, closes the parent's copies of the
+intermediate pipe ends, waits for every process, and (with ``check=True``)
+raises :exc:`PipelineError` if *any* stage fails -- equivalent to the
+shell's ``set -o pipefail`` without needing a shell.
 
-Alternatively, for trusted input, the shell's own pipeline support may still
-be used directly:
+If you need to read the final stage's output incrementally rather than
+waiting for the whole pipeline to finish (for example, streaming a large
+decompressed file), chain :class:`Popen` instances directly::
 
-.. code-block:: bash
-
-   output=$(dmesg | grep hda)
-
-becomes::
-
-   output = check_output("dmesg | grep hda", shell=True)
+   p1 = Popen(["dmesg"], stdout=PIPE)
+   p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
+   p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
+   for line in p2.stdout:
+       ...
+   p2.wait()
+   p1.wait()
+   if p1.returncode or p2.returncode:
+       ...  # handle failure in either stage
+
+The ``p1.stdout.close()`` call after starting p2 is important in order for
+p1 to receive a SIGPIPE if p2 exits before p1.  Each process must be
+waited on and its return code checked to detect failure in any stage.
 
 
 Replacing :func:`os.system`

Doc/whatsnew/3.15.rst

@@ -1121,6 +1121,17 @@ ssl
 subprocess
 ----------
 
+* Added :func:`subprocess.run_pipeline` for running a sequence of commands
+  connected stdout-to-stdin, similar to a shell ``a | b | c`` pipeline,
+  without invoking a shell.  It returns a
+  :class:`~subprocess.CompletedPipeline` carrying the return codes of every
+  stage; passing ``check=True`` raises :exc:`~subprocess.PipelineError` if
+  any stage fails (pipefail semantics).  This replaces the manual
+  :class:`~subprocess.Popen`-chaining recipe and the
+  ``bash -c "set -o pipefail; ..."`` workaround previously needed to detect
+  failures in non-final stages.
+  (Contributed by Gregory P. Smith in :gh:`47798`.)
+
 * :meth:`subprocess.Popen.wait`: when ``timeout`` is not ``None`` and the
   platform supports it, an efficient event-driven mechanism is used to wait for
   process termination: