1.5 Folder Structures

Recommended Folder Structures

Part of the battle with bioinformatics is keeping your files organised! Here are some recommended folder structures to help you keep track of your data and analysis files. By no means is this a definitive structure, but it should give you a good starting point.

Date format

When including dates in folder or file names, it’s best to use the YYYYMMDD format. This ensures that files are sorted chronologically when listed alphabetically. For example, a folder created on January 16, 2024, would be named 20240116_ProjectName.

Equally, if you have multiple dates to include, you can separate them with underscores, for example: 20240116_20240316_ProjectName.

Or you can us a more concise format such as 16JAN2024 or 16JAN24, for example: 16JAN2024_Rad21_ChIPseq_mESCs.

Naming Conventions

When naming files and folders, it’s important to use clear naming conventions. This will help you and others understand the contents of the files and their purpose at a glance.

The maintakeaway is to use as much relevant information as possible in the file/folder name while keeping it concise.

I would recommend using numbers at the start of folder names to keep them in order, and for ease of navigating the folder systems. It may also be useful to create folders to group assays or or files from the same project together, for example:

.
├── 00_raw_fastqs/
├── 01_ChIP_seq/
├── 02_ATAC_seq/
├── 03_Multiple_enhancer_analysis/
├── 04_RNA_seq/
└── 05_Single_cell_analysis/

Within the 01_ChIP_seq/ folder, you could then have subfolders for each step of the analysis, for example:

.
├── 01_ChIP_seq/
    ├── 01_16JAN2024_mESCs_Rad21_ChIPseq_Rad21KO/
    ├── 02_10MAR2024_mFL_Structural_proteins_ChIPseq/
    ├── 03_01JUN2024_mESC_Histones_ChIPseq/

Within each sample folder, you could then have subfolders for each sample:

.
├──01_ChIP_seq/
    ├── 01_16JAN2024_mESCs_Rad21_ChIPseq_Rad21KO/
        ├── mESC_Rad21_ChIP_seq_WT_rep1/
        ├── mESC_Rad21_ChIP_seq_WT_rep2/
        ├── mESC_Rad21_ChIP_seq_Rad21KO_cloneA_rep1/
        ├── mESC_Rad21_ChIP_seq_Rad21KO_cloneB_rep1/

File naming guidelines

When naming files, consider the following guidelines:

Avoid spaces; use underscores (_) or hyphens (-) instead.
Include relevant metadata in the name, for example:

Order	Metadata Type	Examples	Notes
1	Species	human, mouse
2	Sample type	mESC, mFL, EB_CD71+	If applicable, include differentiation stage (e.g., d0, d7, E14.5)
3	Assay type	ChIP-seq, RNA-seq
4	Genotype or conditions	WT, KO, treated, untreated	If applicable, include time point or treatment condition (e.g., 0h, 24h) or a biological clone ID
5	Replicate number	rep1, rep2
6	File type/extension	.fastq.gz, .bam, .bed

This would result in file names such as:

mESC_CTCF_ChIPseq_WT_E14_rep1.fastq.gz
HUDEP2_d13_ATAC_HBAKO_clone1_rep1.fastq.gz
mESC_Rad21_ChIP_seq_Rad21KO_cloneB_rep1.bam

It is clear from these filenames what species, sample type, assay, condition, and replicate the files correspond to. This is not only helpful to others in the future whoc may want to revisit your data, but also to you when you come back to the files after some time!

Raw data folder

You may have noticed the 00_raw_fastqs/ folder in the example above. I would recommend having a top-level folder for raw data, which is never modified. This ensures that you always have the original data to go back to if needed.

Within this folder, you could have subfolders for each sequencing run or project, for example:

.
├── 00_raw_fastqs/
    ├── 01_ChIP_seq_runs/
        ├── 20240116_Rad21_ChIPseq_mESCs/
        ├── 20240310_Structural_proteins_ChIPseq_mFL/
        ├── 20240601_Histones_ChIPseq_mESCs/
    ├── 02_ATAC_seq_runs/

Within each sequencing run folder, you could then have the raw FASTQ files:

├── 00_raw_fastqs/
    ├── 01_ChIP_seq_runs/
        ├── 20240116_Rad21_ChIPseq_mESCs/
            ├── mESC_Rad21_ChIPseq_WT_rep1.R1.fastq.gz
            ├── mESC_Rad21_ChIPseq_WT_rep1.R2.fastq.gz
            ├── mESC_Rad21_ChIPseq_WT_rep2.R1.fastq.gz
            ├── mESC_Rad21_ChIPseq_WT_rep2.R2.fastq.gz

Linking raw data to analysis folders

To avoid duplicating large raw data files in multiple analysis folders, you can create symbolic links to the raw FASTQ files in your analysis folders. This way, you can access the raw data without taking up extra storage space.

To create a symbolic link, use the ln -s command in the terminal. For example, to link a raw FASTQ file from the 00_raw_fastqs/ folder to an analysis folder, you would run:

cd /path/to/01_ChIP_seq/01_16JAN2024_mESCs_Rad21_ChIPseq_Rad21KO/mESC_Rad21_ChIP_seq_WT_rep1/
ln -s /path/to/00_raw_fastqs/01_ChIP_seq_runs/20240116_Rad21_ChIPseq_mESCs/mESC_Rad21_ChIPseq_WT_rep1*.fastq.gz ./

This command creates symbolic links to the raw FASTQ files in the current directory (./).

Summary

In summary, having a clear and organized folder structure is crucial for managing bioinformatics data effectively. By following the recommended folder structures and naming conventions outlined above, you can ensure that your data is easy to navigate and understand for both yourself and others in the future.