Pipes and Redirection: The Composable Power of the CLI

Command-line tools become composable when they can receive input and produce output through standard interfaces. A pipe connects one command’s normal output to another command’s input. Redirection sends a stream to a file or another destination.

Learn the data flow before memorizing operators. Exact syntax differs among shells, and PowerShell pipelines can pass structured objects rather than only text.

A command can use three standard streams

A process commonly begins with three channels:

Normal output and error may look mixed because both appear in the same terminal. They remain distinct streams unless the shell or program combines them.

Consider a search command. Matching lines belong on standard output. A message that an input file cannot be opened belongs on standard error. Keeping them separate lets you save valid results without silently treating an error message as data.

Programs do not all follow these conventions perfectly. Check the selected tool’s documentation and test its behavior on harmless input.

A pipe connects processes

Conceptually:

command A --standard output--> command B --standard output--> terminal

Command B reads the data produced by command A. This permits small tools to perform separate jobs:

produce records → filter records → format result

The pipe normally carries data, not the entire visual terminal display. Prompts, colors, progress indicators, and errors may behave differently when output is not connected to an interactive terminal.

Do not assume every tool accepts piped input. Some require a filename, option, or explicit instruction to read standard input.

Redirection changes a destination

Output redirection sends a stream somewhere other than its usual terminal destination:

command --standard output--> result.txt

This can turn an observed result into an artifact that another process or person can inspect.

Redirection can also destroy information. Many shells use one operator to replace a file and a different operator to append. A typo in the target path can write in the wrong directory. Before redirecting:

1. confirm the current directory;
2. inspect whether the target exists;
3. use a disposable output name;
4. confirm whether the operation replaces or appends;
5. run the producing command without redirection first when safe; and
6. inspect the saved file afterward.

Use your shell’s official help for the exact operator. Do not infer behavior from an example written for another shell.

Read a pipeline from left to right

For a simple pipeline, annotate each segment:

[producer] | [filter] > [output file]

Ask:

• What does the producer emit?
• Does the filter read standard input?
• What does the filter remove or transform?
• Which stream reaches the output file?
• What happens to errors?
• Will the output file be replaced?

This decomposition matters for safety. The last segment is not the only command that runs. Each segment can read files, access a network, expose data, or change state.

Exit status needs deliberate handling

Each process ends with an exit status. Zero commonly means reported success and a nonzero value commonly signals a problem, but the meaning belongs to the program.

A pipeline includes several processes. Shells differ in which status becomes the pipeline’s status and what options can change that behavior. Therefore, a success status for the full line may not prove that every segment succeeded.

For important work:

• test each segment separately;
• inspect standard error;
• check the shell’s pipeline-status rules;
• verify the final artifact; and
• avoid discarding diagnostics until the workflow is understood.

Complete a safe exercise

Use a disposable folder. Create courses.txt:

CS 101
BIO 110
CS 210
HIS 105

Your goal is to:

1. display the file;
2. send its text through one filter that keeps lines containing CS;
3. save normal output as cs-courses.txt; and
4. inspect the result.

Use commands and operators documented for your current shell. The conceptual flow is:

courses.txt → display command → standard output
standard output → pipe → text filter
filtered standard output → redirection → cs-courses.txt

Expected content:

CS 101
CS 210

Verification:

• the source file remains unchanged;
• the output contains two lines;
• both lines contain CS;
• no error message was saved as data; and
• the exit status and visible diagnostics are understood.

If the output already exists, remove it only through a deliberate, safe practice step or choose another name. Do not learn overwrite behavior on important work.

Text and object pipelines differ

Traditional Unix-like shell pipelines generally connect byte or text streams. PowerShell can pass objects containing named properties between commands.

The mental model still applies—one process emits data that another receives—but the representation changes. A text filter expects lines and delimiters. An object-aware command can select properties without parsing displayed columns.

Platform companion posts should teach the exact syntax and data model. Avoid copying a Bash example into PowerShell or a PowerShell example into Bash without translation.

Common mistakes

• Treating output and error as one stream. Identify which data is normal output.
• Reading only the last command. Inspect every pipeline segment.
• Redirecting before checking the target. Confirm the path and overwrite behavior.
• Assuming a pipeline’s status covers every segment. Learn the shell rule.
• Parsing a visual table as stable data. Prefer documented machine-readable output when available.
• Mixing shell syntaxes. Use examples for the active shell.
• Saving output without verification. Inspect content and record count.

Do this now

Complete the courses.txt exercise using your shell’s official documentation. Draw the three-box data flow and label standard output, the pipe, and the output file.

Log what you learned

Record only:

• Result: What did the action produce?
• Evidence: What observation, test, or source supports that result?
• Next action or unresolved question: What should happen next?

Next, identify the small CLI toolkit worth learning when a real task requires it.