1.1 Terminal Basics

Terminal and shell basics

This guide covers essential terminal and shell basics for working on remote servers via SSH, including navigating the file system, managing files and directories, and running commands. The Terminal is a text-based interface that allows you to interact with your computer’s operating system using commands. It is especially useful for bioinformatics tasks, which often involve working on remote servers and running command-line tools.

Opening a terminal

To open a terminal on your local machine:

Mac: Use the built-in Terminal app (found in Applications > Utilities) or by using Spotlight Search (Cmd+Space, then type “Terminal”).
Windows: Use PowerShell or Command Prompt, or install Windows Subsystem for Linux (WSL) for a more Unix-like environment.
Linux: Use the built-in terminal application.

Navigating the file system

Command	Description
`pwd`	Print the current working directory
`ls`	List files and directories in the current directory
`ls -lh`	List files with detailed information in a human-readable format
`ls -1`	List files and directories in a single column
`cd <directory>`	Change to the specified directory
`cd ..`	Move up one directory level
`cd ../..`	Move up two directory levels
`cd ~`	Change to the home directory

Further you can use tree command to view the directory structure in a tree-like format. For example:

tree -L 2

This command will display the directory structure up to two levels deep.

Example output:

.
├── data
│   ├── fastq
│   └── reference
├── results
│   ├── alignments
│   ├── peaks
│   └── qc
├── scripts
│   ├── alignment.sh
│   └── qc.sh
└── README.md

8 directories, 3 files

Useful Keyboard Shortcuts

Shortcut	Description
`Tab`	Auto-complete file and directory names (This completes the name or opens suggestions)
`Ctrl+C`	Cancel the current command
`Ctrl+L`	Clear the terminal screen
`Ctrl+A`	Move the cursor to the beginning of the line
`Ctrl+E`	Move the cursor to the end of the line

Managing files and directories

Command	Description
`mkdir <directory>`	Create a new directory
`touch <file>`	Create a new empty file
`cp <source> <destination>`	Copy a file or directory
`ln -s <source> <name>`	Create a symbolic link to a file or directory
`ln -s <source> ./`	Create a symbolic link to a file or directory in the current directory
`mv <source> <destination>`	Move or rename a file or directory
`rm <file>`	Remove a file
`rm -r <directory>`	Remove a directory and its contents recursively Use with caution!
`cat <file>`	Display the contents of a file
`less <file>`	View the contents of a file one page at a time
`head <file>`	Display the first few lines of a file
`cat <file> \\| head`	Display the first few lines of a file using a pipe
`wc -l <file>`	Count the number of lines in a file
`tail <file>`	Display the last few lines of a file
`vi <file>`	Open a file in the vi text editor (For more details on using vi try here)
`wget <url>`	Download a file from the internet

Managing file compression

Command	Description
`gzip <file>`	Compress a file using gzip
`gunzip <file.gz>`	Decompress a gzip-compressed file
`zless <file.gz>`	View a gzip-compressed file one page at a time
`tar -cvf <archive.tar> <directory>`	Create a tar archive of a directory
`tar -xvf <archive.tar>`	Extract a tar archive
`tar -czvf <archive.tar.gz> <directory>`	Create a gzip-compressed tar archive of a directory
`tar -xzvf <archive.tar.gz>`	Extract a gzip-compressed tar archive

Printing output to the terminal

You can print output to the terminal using the echo command. This is useful for displaying messages or the values of variables. For example:

echo "Hello, World!"  # Print a message to the terminal
name="Doug"
echo "Hello, $name!"  # Print a message with a variable

Hello, Doug!

Keeping track of what you’ve done

If you want to look back or repeat a recent command, you can use the up ↑ and down ↓ arrow keys on your keyboard to scroll through your command history.

There is a cool command called history that shows you a list of all the commands you’ve run in the terminal. This can be useful for keeping track of what you’ve done and for re-running previous commands. To view your command history, simply type:

history

To export the history to a file, you can use:

history -w my_command_history.txt

Pipes and For loops

Pipes

Pipes (|) allow you to connect the output of one command to the input of another command. This is useful for chaining commands together to perform complex tasks. For example, to list all files in a directory and then count the number of files, you can use the following command:

ls | wc -l

This command lists all files in the current directory (ls) and then pipes the output to the wc -l command, which counts the number of lines (i.e., files) in the output.

Loops

You can also use loops to perform repetitive tasks. For example, to create multiple directories named dir1, dir2, and dir3, you can use the following loop:

for i in 1 2 3; do mkdir dir$i; done

If you had the files file1.txt, file2.txt, and file3.txt and wanted to compress them all using gzip, you could use the following loop:

for file in file1.txt file2.txt file3.txt; do gzip $file; done

Looping with wildcards

Writing out all the filenames can be tedious. Instead, you can use a wildcard (*) to match all files with a specific pattern. For example, to compress all .txt files in the current directory, you can use:

for file in *.txt; do gzip $file; done

You can also perform different operations within the loop. It is often useful to echo the command being run for logging purposes. For example, to compress all .txt files and print a message for each file, you can use:

for file in *.txt; do
  echo "Compressing $file"
  gzip $file
done

You can also use variables within loops. For example, if you wanted to make a name from the original filename, you could do something like:

for file in *.txt; do
  base_name=$(basename $file .txt) #The `basename` command extracts the filename without the directory path and the specified suffix (in this case, `.txt`).
  echo "Compressing $file to ${base_name}.gz"
  gzip $file
done

for file in *.txt; do
  base_name=${file%.txt} #The `${file%.txt}` syntax removes the `.txt` extension from the filename.
  echo "Compressing $file to ${base_name}.gz"
  gzip $file
done

Making notes with comments

You can add comments to your commands and scripts using the # symbol. Anything following the # on a line is ignored by the shell. This is useful for adding explanations or notes to your commands. For example:

# This command lists all files in the current directory
ls  
# This command counts the number of files
ls | wc -l

Running commands in the background

You can run commands in the background by appending an ampersand (&) at the end of the command. This allows you to continue using the terminal while the command is running. For example:

sleep 60 &  # This command will run in the background for 60 seconds

You can also use nohup to run a command that will continue running even if you close the terminal:

nohup sleep 60 &  # This command will run in the background for 60 seconds

To view the list of background jobs, you can use the jobs command:

jobs

Or top to see all running processes:

top

Cancelling commands

If you need to stop a running command, you can usually do so by pressing Ctrl+C in the terminal. This sends a SIGINT signal to the process, which typically causes it to terminate.

If a command is running in the background and you want to stop it, you can use the kill command followed by the process ID (PID) of the command. You can find the PID using the ps command or top. For example:

kill <PID>

If you are viewing a file with less and want to exit, you can press q to quit the viewer.