Week 7: Shell scripts

Author

Affiliations

Jelmer Poelstra

CFAES Bioinformatics Core, Ohio State University

Dept. of Plant Pathology, Ohio State University

---

---

1 Overview

This week, you will learn how to write shell scripts and use them to run programs with command-line interfaces (CLIs), like various bioinformatics tools.

Looking at the bigger picture, one key goal of this course is to enable you to submit shell scripts as “batch jobs” at OSC. This week, we will focus on shell scripts, and in two weeks, we will move on to submitting these scripts as batch jobs.

This is a pretty packed week and we may not reach the end of the in-class material on Wednesday. If not, we will finish the remainder at the start of next week. (The assignment will only cover material from the first class page.)

2 Learning goals

Shell scripting I: variables and arguments (Monday/Wednesday)

• Why it is useful to collect your commands into shell scripts that can be rerun easily
• The basics of shell scripts, including shell script header lines, using a script to run FastQC as an example
• Shell variables and how to use them
• Using command-line arguments with your own scripts
• Adding “logging” statements to shell scripts that run command-line programs
• Installing and using the Shellcheck VS Code extension

Shell scripting II: rerunning scripts with loops (Wednesday)

• How for loops work
• How to use a for loop to loop over files and run a shell script many times
• How to keep a record of your analysis steps in a Markdown file
• Self-study: How to use parameter expansion to search-and-replace text in variables
• Self-study: How to loop over samples (pairs of FASTQ files) rather than individual files

3 Exercises & assignments

• Graded Assignment: Software and shell scripts (Due 10/11)

4 Readings

• The last part of the week’s second class page is self-study.

5 Optional self-study material

On the course website

• Advanced shell scripting — command substitution, basename and dirname, renaming files with loops, conditionals (if statements), while loops, default values for arguments, standard output vs. standard error, and script options.

• Shell configuration — the $PATH, the ~/.bashrc configuration file, and aliases.

Other

• Buffalo (2015) — Chapter 12: “Bioinformatics Shell Scripting, Writing Pipelines, and Parallelizing Tasks” (OSU library link )

---

References

Buffalo, Vince. 2015. Bioinformatics Data Skills [Reproducible and Robust Research With Open Source Tools]. First edition. O’Reilly.