Globbing & Expansion

Bash performs seven expansions on every line, in a fixed order, before the command runs. Get the order wrong in your head and you'll write code that does almost-but-not-quite the right thing.

The expansion order

Bash processes each command line in this order:

1. Brace expansion — {a,b,c}, {1..10} (textual, before anything else)
2. Tilde expansion — ~, ~user
3. Parameter / variable expansion — $var, ${var:-default}
4. Command substitution — $(cmd), `cmd`
5. Arithmetic expansion — $(( expr ))
6. Word splitting — splits on IFS (only unquoted)
7. Pathname expansion (globbing) — *.log, ?.txt, [abc]*

Quote removal happens last, after all expansions.

The most important consequence: brace expansion happens before variable expansion. So {$x,$y} doesn't work the way you might hope:

x=1 y=2
echo {$x,$y}     # prints: {1,2}      (brace tried to expand first, saw {$x,$y} as one
                 #                     literal token; variable expansion came later)

# Use eval if you really need it (almost always wrong) or arrays.
arr=(1 2)
echo "${arr[@]}"  # 1 2

Globbing — pathname expansion

ls *.log              # all .log files in current dir
ls ?.txt              # one-character names: a.txt, b.txt, ...
ls [abc]*             # files starting with a, b, or c
ls [!abc]*            # files NOT starting with a, b, or c
ls [a-z]*.txt         # one lowercase letter, then .txt suffix

With shopt -s globstar (bash 4+, default in zsh):

shopt -s globstar
ls **/*.log           # all .log files in any subdirectory, recursive

What happens with no match

By default, bash leaves a non-matching glob literal:

ls *.nonexistent      # ls: *.nonexistent: No such file or directory

This means a script that does cp *.log dest/ and finds zero .log files actually runs cp '*.log' dest/, which errors out. Two fixes:

shopt -s nullglob     # no match → expand to nothing (zero words)
shopt -s failglob     # no match → fail with error, don't run command

nullglob is the safer default for scripts. Set it at the top.

Extended globs (extglob)

shopt -s extglob unlocks regex-like quantifiers in glob patterns:

shopt -s extglob

ls !(*.log)            # everything EXCEPT .log files
ls *(error|warn).log   # zero-or-more of (error|warn) followed by .log
ls @(error|warn).log   # exactly ONE of (error|warn) followed by .log
ls +(a|b).txt          # ONE OR MORE
ls ?(prefix-)*.csv     # zero-or-one of "prefix-"

Useful for surgical file selection without spawning find.

Brace expansion

Pure text-substitution. Bash, no fork, no glob check.

echo {a,b,c}                # a b c
echo {a,b,c}-{1,2}          # a-1 a-2 b-1 b-2 c-1 c-2 (cartesian)
echo file{1..5}.txt         # file1.txt file2.txt file3.txt file4.txt file5.txt
echo file{01..03}.txt       # file01.txt file02.txt file03.txt (zero-padded)
echo {1..10..2}             # 1 3 5 7 9 (with step)
echo {a..e}                 # a b c d e

Use it for:

• Backups: cp config.yaml{,.bak} — equivalent to cp config.yaml config.yaml.bak
• Bulk creation: mkdir -p ./project/{src,test,docs}/{old,new}
• Cleanup: rm -rf log.{0..7}

The brace pattern is purely textual — there's no filesystem check. cp file.{txt,bak} will run even if neither file exists; it's cp that errors out.

Tilde expansion

~                  # $HOME
~/                 # $HOME/
~user              # /home/user (other users; only at start of word)
~+                 # $PWD (current directory)
~-                 # $OLDPWD (previous directory)

Tilde only expands at the start of a word. --user=~/foo does NOT expand the tilde — quote-style protects it. Use --user="$HOME/foo" for safety.

Command substitution

$(cmd)              # modern, nests cleanly
`cmd`               # old, doesn't nest, awkward escapes

Always use $( ). The two-character cost pays back forever.

date=$(date +%Y-%m-%d)
files=$(ls *.log)
out=$(curl -fsS http://api/v1/users)

Don't use command substitution in tight loops — every $(...) is a fork. Bash builtin alternatives:

• $(<file) instead of $(cat file) — built-in file read, no fork.
• ${var:offset:len} instead of $(echo $var | cut ...) — built-in substring.
• ${var%pat} / ${var#pat} instead of $(echo $var | sed ...) — built-in trim.

Process substitution

<(cmd) and >(cmd) — file-like wrappers around command streams. Covered in lesson 05; mentioned here because it's an expansion form.

diff <(sort a) <(sort b)

Arithmetic expansion

$(( 2 + 2 ))         # 4
$(( a * b + c ))     # variables don't need $ inside (( ))
$(( 0x10 ))          # 16 (hex)
$(( 010 ))           # 8 (octal — leading zero!)
$(( 2#1010 ))        # 10 (binary, base#digits)

The (( )) form (without $) is a statement — runs and sets exit code. The $(( )) form is an expansion — substitutes the result as text.

Quoting and word splitting

After all the above, unquoted results undergo word splitting on IFS. This is where most bash bugs live.

IFS=$' \t\n'                  # default
files="a.log b.log c.log"
ls $files                     # ls a.log b.log c.log     (3 args — splits on space)
ls "$files"                   # ls "a.log b.log c.log"   (1 arg — no splitting)

x="a b"
echo $x                       # a b   (printed with default IFS — spaces preserved on output)
for f in $x; do echo "<$f>"; done  # <a><b>  (word-split happened in for)

Quote everything by default. Bare $var is a bug-in-waiting.

Quote removal

The final step. After all expansions, the shell removes any quotes you wrote that haven't already done their job:

echo "hello $USER"            # quotes removed; output: hello sam

This is rarely something you think about explicitly, but it explains why escapes like \$ survive into the command line.

Bash vs zsh

zsh has a richer expansion vocabulary:

• Globbing: *.log(om[1]) — most-recently-modified .log file as a single expansion. Bash needs ls -t *.log | head -n1 (a pipeline + fork).
• Glob qualifiers: (.) (regular files only), (/) (directories only), (L+1k) (files larger than 1K).
• Brace expansion: {1..10..2} works in both, but zsh has {(s/-/)var} for split-on-separator.
• Default behaviour: zsh has setopt EXTENDED_GLOB for extglob-equivalent (different syntax — ^, ~, #, ##).
• No accidental word-splitting. zsh leaves unquoted $var as one word. Convenient interactively, surprising for bash refugees.

For shareable scripts, target bash. For your interactive zsh, use the niceties.

Common bugs

Glob in literal form when no match. cp *.log dest/ errors when zero matches. Fix: shopt -s nullglob or test first with compgen -G '*.log' >/dev/null.

Brace expansion with vars. mkdir -p {$src,$dst} doesn't expand $src/$dst into braces — those happen earlier than brace expansion. Use mkdir -p "$src" "$dst".

* in rm argument. rm /tmp/foo/* runs rm with whatever /tmp/foo/* expands to. If the dir is empty AND nullglob is on, you get rm (no args, error). If nullglob is OFF, you get rm '/tmp/foo/*' (also error). Either way, defensive: check first or use find /tmp/foo -mindepth 1 -delete.

Path with ~ in a quoted string. cp "file" "~/dest" — the ~ does NOT expand inside quotes. Use "$HOME/dest" or ~/dest (unquoted, which also opens word-splitting questions if $HOME has spaces — Mac /Users/name rarely does, but Windows-mounted dirs do).

Hidden files (.dotfiles) skipped. Globs don't match leading-dot files by default. shopt -s dotglob to include them, or use explicit patterns: cp .* dest/ (which catches . and .. too — even worse).

Unquoted command substitution. for f in $(ls) word-splits the output, which fails on filenames with spaces. Use for f in * (real glob) or pipe to read.