Turn text into answers with pipelines

By the end: Combine small tools to clean, group, filter, and summarize data.

The idea

A pipe connects one command's standard output to the next command's standard input. It does not normally include error output. Build a pipeline one stage at a time and inspect the intermediate result. These examples use tiny, deliberately simple records; CSV with quoted commas needs a CSV parser.

Your command toolkit

sort

What it solves: Put records into a useful order.

printf '%s\n' 12 3 8 | sort -n

-n sorts numerically. Plain sort is textual and its collation depends on locale; set LC_ALL=C for reproducible byte-oriented sorting.

uniq

What it solves: Count adjacent repeated records.

printf '%s\n' api web api | sort | uniq -c

uniq only merges adjacent duplicates, so sorting first matters. The counts tell you how often each value occurred.

cut

What it solves: Extract a simple delimited field.

printf 'api,200\nweb,500\n' | cut -d, -f2

Selects the second comma-delimited field. It does not understand quoted CSV fields or nested formats.

paste

What it solves: Join records side by side.

paste names.txt scores.txt

Joins corresponding lines, normally with a tab. Run the lab first to create these two files.

tr

What it solves: Translate or remove characters.

printf 'READY\n' | tr '[:upper:]' '[:lower:]'

tr works on a stream rather than accepting a filename argument. Character classes make intent clearer than hardcoded alphabets.

sed

What it solves: Apply a simple text transformation.

printf 'stage=draft\n' | sed 's/draft/review/'

Prints the transformed stream. No -i means the source is untouched, making this a useful preview before file edits.

awk

What it solves: Filter records and calculate fields.

printf 'api 12\nweb 8\n' | awk '{total += $2} END {print total}'

awk splits records into fields. Here it accumulates the second field and prints 20 at the end.

column

What it solves: Make a plain text table readable.

printf 'NAME SCORE\nAsha 9\nJuma 8\n' | column -t

Aligns whitespace-separated fields for a human reader. Keep the original data format for scripts.

xargs

What it solves: Turn a stream of filenames into arguments.

find . -maxdepth 1 -type f -print0 | xargs -0 -r wc -l

NUL delimiters preserve spaces and newlines in filenames. GNU -r avoids invoking the command when there is no input.

Before you move on

Try the guided lab below. Compare the result with the explanation, then record your reasoning in the workbook. A missing optional program or a restricted environment is something to explain, not a reason to change system settings blindly.