More Unix shell:
wildcards, brace expansion, `find` & `xargs`

Optional self-study content

Author

Affiliations

Jelmer Poelstra

CFAES Bioinformatics Core, Ohio State University

Dept. of Plant Pathology, Ohio State University

Published on

September 21, 2026

Page last modified

October 4, 2026

1 Introduction

This page contains optional self-study material that builds on this week’s shell lectures (w4b and w4c):

Other shell wildcards
Brace expansion
The find command to find files
xargs to repeat commands

1.1 Setting up

At https://ondemand.osc.edu, start a VS Code session in /fs/ess/PAS3493/people/$USER.
In the terminal, navigate to your week04/sandbox dir, where you will create dummy files to practice on.

2 Other shell wildcards

In w4b, you learned about globbing (selecting files) with the * wildcard. While not as frequently used as *, a few other shell wildcards exist. Here is a complete overview of shell wildcards:

Wildcard	Matches
`*`	Any number of any characters, including none
`?`	Exactly one of any character
`[...]`	Exactly one of the characters listed in the brackets
`[^...]`	Exactly one character that is not listed in the brackets

To get set up, create some dummy FASTQ files to practice with:

# (You should be in /fs/ess/PAS3493/people/$USER/week04/sandbox)
# (If you did the w4b globbing section, these files already exist)
touch sample1_R1.fastq sample1_R2.fastq
touch sample2_R1.fastq sample2_R2.fastq
touch sample3_R1.fastq sample3_R2.fastq

Using the more restrictive wildcard ? can be useful to prevent accidental matches. Here is an example of using the ? wildcard to match both R1 and R2:

ls sample1_R?.fastq

sample1_R1.fastq  sample1_R2.fastq

The character class wildcard [] can be useful when you want to include or exclude multiple files among a set of files with rather similar names. For example, to match files for sample1 and sample2 using only a character class with []:

Method 1 – List all possible characters (1 and 2 in this case):

# This is saying: after 'sample', there should be a 1 OR a 2 ('[12]'),
# followed by any character (*):
ls sample[12]*

sample1_R1.fastq  sample1_R2.fastq  sample2_R1.fastq  sample2_R2.fastq

Method 2 – Use a range like [0-9], [A-Z], [a-z]:

ls sample[1-2]*

sample1_R1.fastq  sample1_R2.fastq  sample2_R1.fastq  sample2_R2.fastq

Method 3 – Exclude the unwanted sample ID by prefacing the character class with a ^, which results in inverted behaviour:

# This is saying: after 'sample', there should be anything EXCEPT a '3':
ls sample[^3]*

sample1_R1.fastq  sample1_R2.fastq  sample2_R1.fastq  sample2_R2.fastq

These examples may seem trivial, but you can use these techniques to easily operate on selections among 100s or 1000s of files.

A character class always matches a single character

So [10-13] does not mean “10 through 13”: it is read as “one character that is 1, 0-1, or 3”. To match multi-digit numbers, use separate patterns (e.g. sample1[0-3]*) or brace expansion (next section).

3 Brace expansion

Brace expansion with {} is another type of shell expansion, but it works very differently from wildcards: a wildcard expands only to names of files that already exist, whereas brace expansion expands to whatever you tell it to, whether or not such files exist. That makes brace expansion the tool for creating many files or dirs with mkdir and touch, and wildcards the tool for selecting existing ones.

For example, use .. within {} to indicate ranges of numbers or letters:

Create 30 directories, one for each day in September:

# Writing '01' keeps the leading zero, so that alphabetical order = date order:
mkdir -p 2026-09-{01..30}

ls 2026-09*

2026-09-01  2026-09-04  2026-09-07  2026-09-10  2026-09-13  2026-09-16  2026-09-19  2026-09-22  2026-09-25  2026-09-28
2026-09-02  2026-09-05  2026-09-08  2026-09-11  2026-09-14  2026-09-17  2026-09-20  2026-09-23  2026-09-26  2026-09-29
2026-09-03  2026-09-06  2026-09-09  2026-09-12  2026-09-15  2026-09-18  2026-09-21  2026-09-24  2026-09-27  2026-09-30

Create 6 empty files:

touch fig-1{A..F}.png

ls fig-*

fig-1A.png  fig-1B.png  fig-1C.png  fig-1D.png  fig-1E.png  fig-1F.png

As the example below shows, you can also:

Use a comma-separated list, and
Combine multiple brace expansions to get all combinations among values in the expansions

mkdir -p treatment-{Kr,Df,Tr}_temp-{lo,med,hi}

ls treatment-*

treatment-Df_temp-hi   treatment-Kr_temp-hi   treatment-Tr_temp-hi
treatment-Df_temp-lo   treatment-Kr_temp-lo   treatment-Tr_temp-lo
treatment-Df_temp-med  treatment-Kr_temp-med  treatment-Tr_temp-med

Exercise 1: Brace expansion and wildcards

Your sandbox still contains the sample1–sample3 FASTQ files. Predict the output of each of these commands, then run them to check:
```
ls sample{1..4}_R1.fastq
ls sample[1-4]_R1.fastq
```
Click to see the answer
```
ls: cannot access 'sample4_R1.fastq': No such file or directory
sample1_R1.fastq  sample2_R1.fastq  sample3_R1.fastq
```
```
sample1_R1.fastq  sample2_R1.fastq  sample3_R1.fastq
```
Brace expansion doesn’t check which files exist, so ls received four file names, including sample4_R1.fastq, which it couldn’t find. The character class is a wildcard and therefore only expands to existing files.

Using brace expansion and the touch command, create empty R1 and R2 FASTQ files for 100 samples with IDs from 001 to 100: sample<ID>_R1.fastq and sample<ID>_R2.fastq.

Click to see the answer

touch sample{001..100}_R{1,2}.fastq

ls sample???_R?.fastq

sample001_R1.fastq  sample026_R1.fastq  sample051_R1.fastq  sample076_R1.fastq
sample001_R2.fastq  sample026_R2.fastq  sample051_R2.fastq  sample076_R2.fastq
sample002_R1.fastq  sample027_R1.fastq  sample052_R1.fastq  sample077_R1.fastq
sample002_R2.fastq  sample027_R2.fastq  sample052_R2.fastq  sample077_R2.fastq
# [...output truncated...]

Now remove the six original FASTQ files (sample1–sample3) with a single rm command that leaves all 200 new files alone. Why would rm sample1* be a bad idea here?
Click to see the answer
```
ls sample?_R?.fastq    # Always check first!
rm sample?_R?.fastq
```
Because ? matches exactly one character, sample?_ matches sample1_ but not sample001_. (rm sample[1-3]_* would also work.)

rm sample1* would remove sample1_R1.fastq and sample1_R2.fastq, but also sample100_R1.fastq and sample100_R2.fastq, as well as sample1.txt if you still have it from the w4b exercise.
Bonus: Count the number of R1 files by first using ls with a globbing pattern that only selects R1 files, and then piping the ls output into wc -l.
Click to see the answer
```
ls *R1*fastq | wc -l
```
```
100
```
You may wonder how this works, since ls puts several files on each line. It only does that when it prints to your screen: when its output goes into a pipe, ls prints one file per line.
Bonus: Copy all FASTQ files except the two for sample100 into a new dir called selection, using a single cp command with a wildcard. (You will first need to create the new dir separately.)
Click to see the answer

First create the selection dir:
```
mkdir selection
```
Method 1 — Exclude sample numbers starting with a 1:
```
cp sample[^1]* selection/
```
Method 2 — Other way around; include sample numbers starting with a 0:
```
cp sample0* selection/
```

4 The `find` command to find files

4.1 Two problems you may run into

Consider the following two problems you may run into, adapted from Buffalo (2015). Say that you try to pipe ls output into the imaginary program process_fastq, which can operate on multiple FASTQ files at once, taking each as an argument:

ls *.fastq | process_fastq

Any spaces in the listed file names would lead to trouble here, since those file names would be erroneously split into separate arguments. A solution would be to use direct globbing:

process_fastq *.fastq

However, a different problem could occur here: if the number of globbed files is very high, you may get an error that the “argument list is too long”.

The find and xargs commands can deal with both of these problems. More generally, they are useful for:

find: file searching and processing beyond ls’s capabilities.
xargs: multi-file and parallel processing without loops.

Let’s create a small dummy project to try these commands, modeled on the maize SNP project from Buffalo (2015):

mkdir -p zmays-snps/{data/seqs,scripts,analysis}
touch zmays-snps/data/seqs/zmays{A,B,C}_R{1,2}.fastq

4.2 `find` basics

The basic syntax for find is:

find [path] [expression-1] [expression-2] ...

find will print all files by default, like ls — but unlike ls, it is recursive by default:

find zmays-snps

zmays-snps
zmays-snps/data
zmays-snps/data/seqs
zmays-snps/data/seqs/zmaysC_R1.fastq
zmays-snps/data/seqs/zmaysA_R1.fastq
zmays-snps/data/seqs/zmaysB_R1.fastq
zmays-snps/data/seqs/zmaysB_R2.fastq
zmays-snps/data/seqs/zmaysC_R2.fastq
zmays-snps/data/seqs/zmaysA_R2.fastq
zmays-snps/scripts
zmays-snps/analysis

Compare this with the corresponding ls output:

ls zmays-snps

analysis  data  scripts

Using the -name option, you can also glob like with ls:

cd zmays-snps
find data/seqs -name "zmaysB*fastq"

data/seqs/zmaysB_R1.fastq
data/seqs/zmaysB_R2.fastq

4.3 `find` functionality beyond that of `ls`

First, you can use -type as an expression to limit results by file type:

# "-type f" to only match regular files (not, e.g., dirs)
find data/seqs -name "zmaysB*fastq" -type f

data/seqs/zmaysB_R1.fastq
data/seqs/zmaysB_R2.fastq

In the previous command, the two expressions were implicitly connected with a logical and: the resulting matches had to be regular files and also had to have zmaysB*fastq in the name. You can connect expressions explicitly using the -and operator…

find data/seqs -name "zmaysB*fastq" -and -type f

data/seqs/zmaysB_R1.fastq
data/seqs/zmaysB_R2.fastq

…or the -or operator:

find data/seqs -name "zmaysA*fastq" -or -name "zmaysC*fastq"

data/seqs/zmaysA_R1.fastq
data/seqs/zmaysA_R2.fastq
data/seqs/zmaysC_R1.fastq
data/seqs/zmaysC_R2.fastq

You can negate expressions with ! — though you must quote the ! so it doesn’t get evaluated by the shell:

# Find regular files that do NOT match "zmaysC*fastq"
find data/seqs -type f "!" -name "zmaysC*fastq"

data/seqs/zmaysA_R1.fastq
data/seqs/zmaysA_R2.fastq
data/seqs/zmaysB_R1.fastq
data/seqs/zmaysB_R2.fastq

If the quoted "!" looks cryptic, you can use the equivalent -not instead, which doesn’t need quotes:

find data/seqs -type f -not -name "zmaysC*fastq"

As another negation example, say you have some files with temp in their names that you don’t want to include in the find output:

touch data/seqs/zmays{A,C}_R{1,2}-temp.fastq

find data/seqs -type f "!" -name "zmaysC*fastq" -and "!" -name "*-temp*"

data/seqs/zmaysA_R1.fastq
data/seqs/zmaysA_R2.fastq
data/seqs/zmaysB_R1.fastq
data/seqs/zmaysB_R2.fastq

The output is identical to that of the previous command, even though four new -temp files now exist: the zmaysC ones were excluded by the first negated expression, and the zmaysA ones by the second.

If ! gets evaluated by the shell, then what does it do?

So what does a ! mean to the shell? It accesses past commands — for example, to re-execute your most recent command, use the following:

!!

4.4 Selecting files by size and last-modified date

With -size, you can select files by their size, using + for “larger than” and - for “smaller than”, and a k, M, or G suffix for kilobytes, megabytes, or gigabytes. For example, to find unusually large FASTQ files in the practice dataset:

find /fs/ess/PAS3493/people/$USER/garrigos/fastq -size +24M

/fs/ess/PAS3493/people/jelmer/garrigos/fastq/S71_R1.fastq.gz
/fs/ess/PAS3493/people/jelmer/garrigos/fastq/S71_R2.fastq.gz

A related option is -empty, which is handy to spot output files that were created but never filled, e.g. because an analysis step failed:

# (All the dummy files you created with 'touch' are empty, so this lists them all)
find data/seqs -type f -empty

With -mmin (minutes) and -mtime (days), you can select files by when they were last modified:

# Files modified less than 60 minutes ago:
find data/seqs -mmin -60

# Files modified more than 30 days ago:
find /fs/ess/PAS3493/people/$USER -type f -mtime +30

4.5 Running commands on `find`’s results

Now, say that you instead want to remove the temporary files we saw above. We can do this using -exec followed by the rm command (in this case, using rm -i for interactive removal):

find data/seqs -name "*-temp.fastq" -exec rm -i {} \;

Here, {} is the placeholder for the file names we are passing on, and you have to indicate the end of our command with an escaped semicolon: \;.

-exec can be very useful to process file queries that are more complicated than what a simple globbing pattern allows for. With -exec, you can use any command to operate on the files that you found.

Finally, note that for the common operation of deleting files, a shortcut exists:

find data/seqs -name "*-temp.fastq" -delete

Exercise 2: `find`

The practice dataset contains gzip-compressed (.gz) files of more than one type. Using a single find command, list all .gz files in garrigos that are not FASTQ files.
Click to see the answer
```
find /fs/ess/PAS3493/people/$USER/garrigos -name "*.gz" "!" -name "*.fastq.gz"
```
```
/fs/ess/PAS3493/people/jelmer/garrigos/ref/annot.gtf.gz
```
Say that your working dir contains a file called notes.gz, and you forget the quotes around the pattern:
```
find /fs/ess/PAS3493/people/$USER/garrigos -name *.gz
```
What would happen, and why?

Click to see the answer

Recall from w4b that the shell expands wildcards before the command runs. The shell therefore replaces *.gz with notes.gz, the only matching file in your working dir, and find receives -name notes.gz. It then looks only for files that are named exactly notes.gz, and finds nothing in garrigos.

Quoting the pattern prevents the shell from expanding it, so find receives the * and does the matching itself.

5 `xargs` to repeat commands

xargs is to be read/pronounced as x-args, where “args” means “arguments”. This command passes arguments, supplied via standard in, on to another command. For example, first recreate the temporary files, then remove them with xargs:

touch data/seqs/zmays{A,C}_R{1,2}-temp.fastq

find data/seqs -name "*-temp.fastq" | xargs rm

The above command is equivalent to the following, where rm receives the very same list of files as its arguments:

rm data/seqs/*-temp.fastq

Unlike the glob, though, xargs won’t run into the argument list too long error: if the list of files is too long for a single command, xargs automatically splits it up and runs rm several times.

Two other useful xargs options are:

-I {} to put each item at a specific place in the command, e.g. when the file name should not be the last argument:
```
mkdir -p backup
find data/seqs -name "*_R1.fastq" | xargs -I {} cp {} backup/
```

-P to run several commands in parallel, combined here with -n 1 to process one file per command:

# Compress FASTQ files with gzip, up to 4 files at a time:
find fastq_dir -name "*.fastq" | xargs -n 1 -P 4 gzip

5.1 `xargs` versus `find -exec`

By using xargs instead of find -exec, you separate the process that finds the files you want to operate on, from the actual operations. This can be beneficial, such as when you want to carefully inspect a list of files before doing anything — for example:

Create some dummy files to practice on:

touch data/seqs/zmays{A,C}_R{1,2}-temp.fastq

Save the find results in a text file:

find data/seqs -name "*-temp.fastq" > files-to-delete.txt
cat files-to-delete.txt

data/seqs/zmaysC_R1-temp.fastq
data/seqs/zmaysA_R1-temp.fastq
data/seqs/zmaysA_R2-temp.fastq
data/seqs/zmaysC_R2-temp.fastq

Now pass that list of files to rm:
```
cat files-to-delete.txt | xargs rm
```

5.2 `find | xargs` and spaces in file names

As always, spaces in file names can trip you up — but you were promised a solution with find and xargs! However, out of the box, the below does in fact fail:

touch "samples A.txt"  "samples B.txt"

find . -name "samples [AB].txt" | xargs rm

rm: cannot remove './samples': No such file or directory
rm: cannot remove 'A.txt': No such file or directory
rm: cannot remove './samples': No such file or directory
rm: cannot remove 'B.txt': No such file or directory

The solution is to use -print0 with find, so it prints files with a “null byte” as the delimiter. Additionally, you need to tell xargs about the null-byte delimiter using -0:

find . -name "samples [AB].txt" -print0 | xargs -0 rm -v

removed './samples A.txt'
removed './samples B.txt'

Exercise 3: `xargs`

Predict the output of each of these two commands, then run them to check:
```
find data/seqs -name "*_R1.fastq" | xargs echo
find data/seqs -name "*_R1.fastq" | xargs -n 1 echo
```
Click to see the answer
```
data/seqs/zmaysA_R1.fastq data/seqs/zmaysB_R1.fastq data/seqs/zmaysC_R1.fastq
```
```
data/seqs/zmaysA_R1.fastq
data/seqs/zmaysB_R1.fastq
data/seqs/zmaysC_R1.fastq
```
In the first command, xargs runs echo once with all three file names as arguments, so they are printed on a single line. In the second command, -n 1 makes xargs run echo three times, with one file name each time.
Before running a command like find ... | xargs rm, how could you use echo to see exactly what would be run, without removing anything?
Click to see the answer

Put echo in front of the actual command:
```
find data/seqs -name "*_R1.fastq" | xargs echo rm
```
```
rm data/seqs/zmaysA_R1.fastq data/seqs/zmaysB_R1.fastq data/seqs/zmaysC_R1.fastq
```
Now xargs runs echo, which prints the full rm command instead of running it. This kind of “dry run” is a good habit before running xargs with a destructive command.

References

Buffalo, Vince. 2015. Bioinformatics Data Skills [Reproducible and Robust Research With Open Source Tools]. First edition. O’Reilly.

1 Introduction

1.1 Setting up

2 Other shell wildcards

3 Brace expansion

Exercise 1: Brace expansion and wildcards

4 The find command to find files

4.1 Two problems you may run into

4.2 find basics

4.3 find functionality beyond that of ls

4.4 Selecting files by size and last-modified date

4.5 Running commands on find’s results

Exercise 2: find

5 xargs to repeat commands

5.1 xargs versus find -exec

5.2 find | xargs and spaces in file names

Exercise 3: xargs