# Version Control for RNA-seq Analysis: How to Use Git to Track Your Code, Data, and Results ## Key Takeaways - **Version control with Git is foundational for RNA-seq reproducibility by tracking code, configuration files, and documentation, not raw or intermediate data files.** This ensures that the exact software versions, parameter choices, and reference files used for analysis are recorded, preventing the loss of critical context over time. - **A well-structured Git repository separates code, configuration, and documentation from large data files (e.g., FASTQ, BAM, reference genomes) using a `.gitignore` file.** This prevents repository bloat and maintains efficient cloning and management, with data provenance tracked via accession numbers and download dates in versioned metadata files. - **Commits serve as discrete checkpoints in the RNA-seq analysis workflow, documenting logical units of work such as script additions, parameter modifications, or reference genome updates.** Descriptive commit messages are crucial for understanding the evolution of the analysis, akin to a detailed laboratory notebook. - **Branches facilitate experimentation with alternative analysis strategies (e.g., different normalization methods or imputation approaches) without compromising the main analysis pipeline.** Merging successful experimental branches into the main branch allows for systematic evaluation and integration of improvements. - **Tracking configuration files (e.g., `samples.tsv`, `parameters.yaml`, `reference.yaml`) and environment specifications (e.g., conda environment files, Dockerfiles) is critical for reproducing RNA-seq results.** These files capture sample metadata, tool-specific settings, reference genome versions, and software dependencies, ensuring that analyses can be rerun with identical inputs and environments. - **Integrating Git with workflow managers like Snakemake or Nextflow enhances reproducibility by versioning workflow definitions (e.g., Snakefile, Nextflow script) and capturing execution logs.** This provides a complete record of how analysis steps were executed, complementing Git's tracking of code and configuration. --- RNA-seq analysis involves multiple processing stages, from raw sequence reads to count tables and differential expression results. Each stage depends on specific software versions, parameter choices, and reference files. Without version control, reproducing an analysis after several months or sharing it with collaborators becomes difficult because the exact state of scripts and configurations is lost. Git provides a practical mechanism to record every change made to analysis code, configuration files, and documentation, creating a complete history of how results were produced. This article explains how to implement Git for RNA-seq projects, covering repository structure, tracking code and configuration files, and integrating with workflow managers. ## The Reproducibility Problem in RNA-seq Projects RNA-seq analysis pipelines typically include quality control, read alignment, transcript quantification, and differential expression testing. Each step can be executed with multiple tools, and each tool has version-specific behavior. A study published in GigaScience describing single-cell RNA-seq analysis noted that downstream functional analysis is made difficult by technical noise and dropout events, where excessive zero counts appear in expression matrices [1]. The choice of imputation method, clustering algorithm, or normalization approach can change biological conclusions. When scripts are modified without tracking, the connection between a published result and the exact code that generated it is broken. The FAIR_Bioinfo training course, designed to teach reproducible computational biology, uses differential gene expression analysis from RNA-seq data as its central example [2]. The course demonstrates that reproducibility requires versioned code, workflow management, virtual environments, and dynamic reporting that lists all user-selected parameters [2]. Git serves as the foundation for this system because it records the evolution of every file in the analysis. For researchers working with public data, version control also documents the provenance of inputs. The National Center for Biotechnology Information provides databases for sequence data and analysis services [3]. When you download raw reads from NCBI, recording the accession numbers and download dates in a versioned file ensures that collaborators can retrieve the same data. The European Bioinformatics Institute offers training pathways for bioinformatics data resources and practical analysis education [4]. These resources emphasize that data management and analysis documentation are core skills for computational biology. ## Core Git Concepts for RNA-seq Workflows Git tracks changes to files in a repository. A repository is a directory that contains your analysis files plus a hidden `.git` folder storing the complete history. Three core operations form the daily workflow: `git add` stages changes, `git commit` records them permanently, and `git push` uploads commits to a remote server such as GitHub or GitLab. ### Commits as Analysis Checkpoints Each commit should represent a logical unit of work. For RNA-seq analysis, useful commit points include: - Initial creation of the workflow skeleton - Addition of a new preprocessing step - Modification of alignment parameters - Update of reference genome version - Fix of a bug in differential expression filtering A commit message should describe what changed and why. For example, "Switch STAR alignment to v2.7.10a and update genome index paths" provides context that a message like "update script" does not. The commit history becomes a lab notebook for your computational work. ### Branches for Experimentation Branches allow you to test alternative analysis strategies without disrupting the main workflow. For example, you might create a branch to test a different normalization method or imputation approach. The main branch retains the canonical analysis, while experimental branches contain variations. If the experimental approach proves superior, you can merge it into the main branch. If not, you can discard the branch without losing the main workflow. The Carpentries offers foundational lessons in Git and shell computing for researchers [5]. These lessons teach the basic commands and workflows that support reproducible data analysis. For researchers new to version control, working through these lessons before starting an RNA-seq project reduces the learning curve. ## Structuring a Git Repository for RNA-seq Analysis A well-organized repository separates code, configuration, data references, and results. This separation prevents large data files from bloating the repository while ensuring that all analysis components are versioned. ### Recommended Directory Layout ``` rnaseq-project/ ├── .gitignore ├── README.md ├── config/ │ ├── samples.tsv │ ├── reference.yaml │ └── parameters.yaml ├── code/ │ ├── 01_quality_control.sh │ ├── 02_alignment.sh │ ├── 03_quantification.R │ └── 04_differential_expression.R ├── data/ │ ├── raw/ (gitignored, referenced by accession) │ └── processed/ (gitignored, regenerable) ├── results/ │ ├── figures/ (gitignored) │ └── tables/ (gitignored) └── docs/ ├── analysis_plan.md └── protocol_log.md ``` The `config` directory holds sample metadata and parameter files. The `code` directory contains all scripts in execution order. The `data` and `results` directories are typically excluded from version control because they contain large files. The `docs` directory holds the analysis plan and a running protocol log. ### What to Track and What to Ignore Track these file types in Git: - Scripts (shell, Python, R, Perl) - Configuration files (YAML, JSON, TSV) - Environment definitions (conda environment files, Dockerfiles) - Workflow definitions (Snakefile, Nextflow script) - Documentation (README, analysis plan, protocol log) - Small reference files under 100 MB Ignore these file types: - Raw sequencing data (FASTQ files) - Large reference genomes and indexes - Intermediate files (BAM, SAM, BED) - Output tables and figures - Temporary files and logs The `.gitignore` file lists patterns for files to exclude. A typical RNA-seq `.gitignore` includes entries for `*.fastq`, `*.bam`, `*.bai`, `*.gtf`, `*.fa`, and `*.fna`. This prevents accidental commits of large binary files that would slow the repository and make cloning difficult. ## At a Glance: Version Control Decisions for RNA-seq Projects The table below summarizes the key decisions researchers face when implementing version control for RNA-seq analysis. Each decision affects reproducibility, collaboration, and long-term data management. | Decision Point | Recommended Approach | Common Mistake | Reproducibility Impact | |---|---|---|---| | Repository scope | Track scripts, configs, environment files, and documentation | Committing raw FASTQ or BAM files | Large files bloat the repository and obscure code history | | Data handling | Record accession numbers and download dates in versioned metadata files | Storing raw data only on local machines without documentation | Collaborators cannot locate or retrieve the original data | | Environment specification | Commit conda environment files, Dockerfiles, or requirements.txt | Tracking scripts without software versions | Analysis may fail or produce different results on another system | | Workflow integration | Version Snakefile or Nextflow script alongside configuration | Running workflows without committing workflow definitions | Execution steps cannot be reconstructed from the repository | | Release management | Tag repository versions at manuscript submission or publication | Leaving the repository without version tags | Readers cannot identify the exact code version used for results | ## Tracking Configuration Files and Sample Metadata Configuration files are the backbone of reproducible RNA-seq analysis. They record sample identifiers, file paths, reference genome versions, and analysis parameters. Tracking these files in Git ensures that every analysis run can be traced to its exact inputs. ### Sample Metadata Table A sample metadata table (often named `samples.tsv`) lists each sample with its associated files and experimental conditions. A minimal table includes: | sample_id | condition | replicate | fastq_r1 | fastq_r2 | |-----------|-----------|-----------|----------|----------| | WT_01 | control | 1 | data/raw/WT_01_R1.fastq.gz | data/raw/WT_01_R2.fastq.gz | | WT_02 | control | 2 | data/raw/WT_02_R1.fastq.gz | data/raw/WT_02_R2.fastq.gz | | KO_01 | treatment | 1 | data/raw/KO_01_R1.fastq.gz | data/raw/KO_01_R2.fastq.gz | | KO_02 | treatment | 2 | data/raw/KO_02_R1.fastq.gz | data/raw/KO_02_R2.fastq.gz | This table is versioned in Git. When a sample is added or removed, the change is recorded. The table also serves as the input for workflow managers that iterate over samples. ### Parameter Files Parameter files store tool-specific settings. For example, a `parameters.yaml` file might contain: ```yaml trim_galore: quality: 20 length: 36 star: alignIntronMax: 100000 outFilterMismatchNmax: 10 featureCounts: strandSpecific: 2 countType: exon ``` Versioning parameter files allows you to compare results across parameter changes. If a collaborator asks why your results differ from theirs, the parameter file provides the answer. ### Reference Genome Tracking Reference genome versions affect alignment and quantification results. Record the genome build, annotation version, and download source in a `reference.yaml` file. For example: ```yaml genome: organism: Homo sapiens build: GRCh38 annotation: GENCODE v43 source: https://www.gencodegenes.org/ download_date: 2024-01-15 ``` This file is tracked in Git. When you update the reference, the commit records the change. The Bioconductor project provides documentation for reproducible genomic analysis, including guidance on managing reference data [6]. ## Integrating Git with Workflow Managers Workflow managers such as Snakemake and Nextflow automate the execution of analysis pipelines. They also provide built-in reproducibility features that complement Git. The scPASU protocol, a Snakemake workflow for quantifying polyadenylation site usage from 3' single-cell RNA-seq data, demonstrates how workflow managers structure analysis steps [7]. The protocol is configurable for organism-specific and sample-specific parameters, and it supports discovery of polyadenylation sites [7]. ### Snakemake and Git Snakemake workflows define rules with inputs, outputs, and shell or script commands. The Snakefile is a text file that can be versioned in Git. When combined with a conda environment file, Snakemake provides a reproducible execution environment. The FAIR_Bioinfo course uses Snakemake with Docker containers to create a fully reproducible RNA-seq analysis [2]. The entire versioned code is available as open source, demonstrating the integration of Git with workflow management [2]. ### Nextflow and Git Nextflow pipelines are also versioned in Git. The nf-core community maintains standardized pipelines with documented usage and configuration practices [8]. These pipelines are distributed through Git repositories, allowing users to track changes and contribute improvements. The NextLongIso pipeline, a Nextflow implementation for long-read RNA-seq analysis, is freely available on GitHub and Zenodo [9]. It integrates transcript discovery with downstream regulatory analyses, including alternative splicing and isoform switching [9]. The pipeline's availability through version-controlled repositories enables users to access specific releases and reproduce published analyses. ### Workflow Manager Benefits for Version Control Workflow managers generate execution logs that record which version of each tool ran and what parameters were used. These logs complement Git history by providing run-specific details. When a workflow completes, the log file can be committed to the repository or stored alongside results. This creates a complete record of the analysis execution. The Oqtans workbench, integrated into Galaxy, provides persistent storage, data exchange, and documentation of intermediate results and analysis workflows [10]. It is available as a git repository containing all installed software [10]. This example shows how version control extends beyond scripts to include the entire analysis environment. ## Practical Steps for Implementing Git in an RNA-seq Project Implementing version control requires a systematic approach. The following steps provide a practical path for researchers new to Git. ### Step 1: Initialize the Repository Create a project directory and initialize Git: ```bash mkdir rnaseq-project cd rnaseq-project git init ``` Create the directory structure described earlier and add a `.gitignore` file. ### Step 2: Create the Initial Commit Add the README, analysis plan, and configuration templates: ```bash git add README.md docs/analysis_plan.md config/ git commit -m "Initialize project structure and analysis plan" ``` The README should describe the project purpose, data sources, and how to run the analysis. ### Step 3: Add Scripts Incrementally Add each script as it is created or modified: ```bash git add code/01_quality_control.sh git commit -m "Add FastQC and MultiQC quality control script" ``` Commit scripts in logical units. Avoid committing multiple unrelated changes in a single commit. ### Step 4: Track Configuration Changes When parameters change, commit the updated configuration: ```bash git add config/parameters.yaml git commit -m "Update STAR alignment parameters for paired-end reads" ``` ### Step 5: Record Data Accessions Create a `data/accessions.tsv` file listing all public data identifiers: | accession | database | sample | download_date | |-----------|----------|--------|---------------| | SRR1234567 | SRA | WT_01 | 2024-02-01 | | SRR1234568 | SRA | WT_02 | 2024-02-01 | Commit this file so data provenance is documented. ### Step 6: Use Branches for Alternative Analyses Create a branch for testing a different normalization method: ```bash git checkout -b test-deseq2-vs-edger ``` Run the alternative analysis and compare results. If the alternative is adopted, merge the branch: ```bash git checkout main git merge test-deseq2-vs-edger ``` ### Step 7: Tag Release Versions When a manuscript is submitted or a report is generated, create a tag: ```bash git tag -a v1.0 -m "Analysis version for manuscript submission" ``` Tags provide named checkpoints that can be referenced in publications. ## Records and Measurements for Version Control Version control generates records that serve as evidence of analysis provenance. These records include commit logs, diff outputs, and release tags. ### Commit Log as Analysis History The commit log provides a chronological record of all changes: ```bash git log --oneline ``` This output shows each commit with its message. For RNA-seq projects, the log should read like a protocol narrative, showing the progression from raw data processing to final results. ### Diff Output for Change Assessment The `git diff` command shows differences between versions: ```bash git diff HEAD~1 HEAD -- config/parameters.yaml ``` This output reveals exactly what changed in a parameter file. When troubleshooting a result change, diff output identifies the responsible modification. ### Tagged Releases for Publication Tagged releases provide stable references for publications. A manuscript can state that the analysis used version v1.0 of the repository. Collaborators can retrieve that exact version: ```bash git checkout v1.0 ``` This ensures that the analysis code matches what was described in the publication. ## Common Failure Patterns in RNA-seq Version Control Researchers encounter several recurring problems when implementing version control for RNA-seq projects. Recognizing these patterns helps avoid them. ### Committing Large Data Files Adding FASTQ or BAM files to Git creates repositories that are slow to clone and difficult to manage. Git is designed for text files, not large binary data. The solution is to use `.gitignore` to exclude data files and instead track accession numbers or file paths. For data that must be preserved, use dedicated data repositories such as NCBI databases [3]. ### Uninformative Commit Messages Commit messages like "update" or "fix" provide no context. A commit message should explain the reason for the change. For example, "Update featureCounts to count only primary alignments" tells future readers what changed and why. ### Committing Generated Outputs Results files such as count tables and figures should not be committed to Git. They are regenerable from the code and configuration. Committing them creates merge conflicts and repository bloat. Instead, document how to regenerate results and store final outputs in a separate location. ### Ignoring Environment Specification Tracking scripts without tracking the software environment leads to irreproducible results. A script that runs with one version of a tool may fail or produce different results with another version. Include environment files in the repository: - `environment.yaml` for conda - `Dockerfile` for container images - `requirements.txt` for Python packages The Bioconductor project provides installation and package documentation that supports reproducible genomic analysis [6]. Recording package versions ensures that the analysis environment can be reconstructed. ### Forgetting to Commit Configuration Files Some researchers commit only scripts and leave configuration files untracked. This creates a gap in the analysis record. Configuration files contain the parameters that determine analysis outcomes. They must be versioned alongside scripts. ## Quality Controls for Versioned RNA-seq Analysis Version control supports quality control by enabling systematic comparison of analysis versions. Several quality checks should be part of the workflow. ### Verify Repository Completeness Before sharing or publishing, verify that the repository contains all necessary components: - All analysis scripts - Configuration files with parameters - Environment specifications - Sample metadata - Data accession records - README with execution instructions The Galaxy Training Network provides accessible workflow training and reproducibility context [11]. Their materials emphasize that complete documentation is essential for reproducible analysis. ### Test Clean Checkout Clone the repository to a fresh location and attempt to run the analysis: ```bash git clone https://github.com/yourname/rnaseq-project.git cd rnaseq-project ``` If the analysis runs successfully from a clean checkout, the repository is complete. If not, missing files or undocumented steps are revealed. ### Compare Results Across Versions When parameters change, compare results from the old and new versions. Differential expression results should be examined for consistency. The CTEC method for single-cell RNA-seq clustering demonstrates that different preprocessing and feature extraction strategies can produce different cluster assignments [12]. This finding underscores the importance of documenting analysis choices and comparing results when methods change. ### Document Tool Versions Record the version of every tool used in the analysis. This information can be captured in the environment file or in a `versions.yaml` file: ```yaml tools: fastqc: 0.12.1 star: 2.7.10a featurecounts: 2.0.6 deseq2: 1.42.0 ``` The LncRAnalyzer workflow for long non-coding RNA discovery uses retrained models for 60 species and integrates multiple filtering approaches [13]. The pipeline is publicly available on GitLab, demonstrating how version-controlled workflows support complex analyses [13]. Recording tool versions ensures that the analysis can be reproduced with the same software. ## Limitations of Git for RNA-seq Analysis Git provides version control for code and configuration, but it has limitations that researchers should understand. ### Git Does Not Store Large Data Raw sequencing data and large reference files cannot be practically stored in Git. These files must be stored elsewhere, such as NCBI databases or institutional storage systems [3]. Git tracks references to these files, not the files themselves. ### Git Does Not Capture Computational Environment Git tracks file changes but does not record the state of the operating system or installed software. Environment files and container definitions address this limitation, but they require discipline to maintain. The FAIR_Bioinfo course addresses this by using Docker containers in addition to Git [2]. ### Git History Can Be Rewritten Commands like `git rebase` and `git reset` can rewrite history. For collaborative projects, this can cause confusion. The solution is to avoid rewriting history on shared branches and to use `git push --force` only when necessary. ### Git Does Not Validate Results Version control ensures that code changes are tracked, but it does not verify that results are biologically correct. Quality control of RNA-seq data remains a separate responsibility. The ncRNAseq protocol, which modifies library preparation to capture mid-sized non-coding RNAs, includes a two-step alignment strategy to correctly assign reads [14]. This example shows that analysis quality depends on both the code and the biological validity of the approach. ## Safety and Regulatory Context for Research Data Version control supports compliance with data management requirements from funding agencies and journals. Many journals require that analysis code be deposited in a public repository. Git repositories on GitHub or GitLab satisfy this requirement. ### Data Privacy Considerations For research involving human subjects, raw sequencing data may be subject to privacy restrictions. Version-controlled repositories should not contain sensitive data. Instead, repositories should reference controlled-access data through accession numbers. The NCBI provides databases for sequence data with varying levels of access control [3]. ### Data Management Plans Funding agencies increasingly require data management plans that describe how data and code will be preserved. A version-controlled repository provides evidence of responsible data management. The European Bioinformatics Institute offers training on data-resource management and practical analysis education [4]. These resources help researchers meet data management expectations. ### Professional Escalation Criteria When version control issues cannot be resolved locally, escalate to appropriate support: - For Git command problems, consult institutional research computing support - For workflow manager issues, consult the workflow documentation or community forums - For data access problems, contact the data repository help desk - For reproducibility failures, consult the FAIR_Bioinfo course materials or The Carpentries lessons [5][2] ## Integrating Version Control with RNA-seq Analysis Tools Different RNA-seq analysis tools have specific considerations for version control. ### Quality Control Tools Quality control tools such as FastQC and MultiQC generate reports. The scripts that run these tools should be versioned. The reports themselves are outputs that can be regenerated. The Galaxy Training Network provides tutorials for quality control workflows [11]. These tutorials demonstrate how to structure quality control steps in reproducible workflows. ### Alignment Tools Alignment tools such as STAR and HISAT2 require reference genome indexes. The index files are large and should not be committed to Git. Instead, document the reference version and the command used to build the index. The `reference.yaml` file described earlier records this information. ### Quantification Tools Quantification tools such as featureCounts and Salmon generate count tables. The parameters used for quantification should be versioned. The AGTAR approach for transcriptome assembly and abundance estimation uses a genetic algorithm [15]. This method demonstrates that novel analysis approaches require careful documentation of parameters and implementation details. ### Differential Expression Tools Differential expression tools such as DESeq2 and edgeR are R packages. The R scripts that run these tools should be versioned. The Bioconductor project provides documentation for these packages and their reproducible use [6]. Recording package versions in the environment file ensures that the analysis can be reproduced. ### Single-cell RNA-seq Tools Single-cell RNA-seq analysis has additional complexity due to dropout events and technical noise [1]. Tools such as scNTImpute and CMF-Impute address these challenges through imputation methods [1][16]. The source code for these tools is available on GitHub, demonstrating the importance of version control in tool development [1][16]. When using these tools, record the version and parameters in the analysis repository. ## Practical Workflow for a Versioned RNA-seq Project The following workflow integrates Git into the complete RNA-seq analysis process. ### Phase 1: Project Setup 1. Create the project directory structure 2. Initialize Git 3. Create the README and analysis plan 4. Create the `.gitignore` file 5. Make the initial commit ### Phase 2: Data Acquisition 1. Download raw data from public databases 2. Record accession numbers in `data/accessions.tsv` 3. Record download dates and database versions 4. Commit the accession file ### Phase 3: Quality Control 1. Write the quality control script 2. Run FastQC and MultiQC 3. Review quality reports 4. Commit the script and any configuration changes ### Phase 4: Alignment and Quantification 1. Write the alignment script 2. Write the quantification script 3. Record reference genome versions 4. Run the analysis 5. Commit scripts and configuration files ### Phase 5: Differential Expression 1. Write the differential expression script 2. Run the analysis 3. Review results 4. Commit the script and configuration ### Phase 6: Reporting 1. Generate figures and tables 2. Write the methods section 3. Tag the repository version 4. Deposit the repository in a public archive ## Common Failure Patterns and Solutions ### Failure: Repository Becomes Too Large Symptom: Cloning the repository takes too long or fails. Solution: Remove large files from the repository history. Use `git filter-branch` or the BFG Repo-Cleaner to remove large files from history. Add patterns to `.gitignore` to prevent future commits of large files. ### Failure: Results Cannot Be Reproduced Symptom: Running the analysis from a clean checkout produces different results. Solution: Check that all configuration files are committed. Verify that the environment specification is complete. Compare the commit history with the analysis log to identify missing components. ### Failure: Merge Conflicts in Configuration Files Symptom: Multiple collaborators edit the same parameter file. Solution: Use smaller configuration files with clear ownership. Communicate changes through commit messages. Resolve conflicts by discussing the intended parameter values. ### Failure: Commit History Is Unclear Symptom: The commit log does not explain the analysis progression. Solution: Write descriptive commit messages that explain the reason for each change. Use branches for experimental work and merge with clear messages. ## Welfare and Ethical Considerations for Research Data Version control supports ethical research practices by ensuring that analysis methods are transparent and reproducible. This transparency allows other researchers to verify results and build upon published work. ### Data Sharing Responsibilities Many journals require that data and code be deposited in public repositories. Git repositories provide a mechanism for sharing analysis code. The NCBI provides databases for sequence data sharing [3]. Researchers should follow journal and funding agency requirements for data deposition. ### Avoiding Analysis Errors Version control helps prevent errors by providing a record of changes. When an error is discovered, the commit history identifies when the error was introduced. This information supports correction and prevents similar errors in future analyses. ### Professional Conduct Reproducible analysis is a professional responsibility in bioinformatics. The FAIR_Bioinfo course emphasizes that reproducibility guarantees the validity of scientific results and simplifies project dissemination [2]. By implementing version control, researchers demonstrate commitment to rigorous scientific practice. ## A Decision Framework for Choosing What to Version in RNA-seq Projects Deciding what belongs in a Git repository requires a consistent evaluation method instead of a fixed rule. Researchers often struggle with edge cases such as small intermediate files, custom reference indexes, or parameter files that change between runs. A structured decision framework helps classify each file type and reduces the risk of either bloating the repository with unnecessary binaries or omitting critical configuration details. ### The Three-Question Test for File Inclusion Apply three questions to every file before deciding whether to track it in Git. The first question asks whether the file is text-based and human-readable. Scripts, configuration files, documentation, and metadata tables qualify. Binary files such as FASTQ archives, BAM alignments, and compressed count matrices fail this test and should be excluded. The second question asks whether the file can be regenerated from other tracked files. If a script produces the file when executed with the tracked configuration, the file itself does not need versioning. The third question asks whether the file records decisions or provenance that cannot be reconstructed from other sources. Sample metadata tables, accession records, and parameter files pass this test because they document choices made by the researcher. A file that passes the first and third questions but fails the second should be tracked. A file that passes the first question but fails the third can be tracked or ignored depending on its size and stability. A file that fails the first question should never be tracked directly. This framework handles the common edge cases that arise in RNA-seq projects, including custom scripts that generate reference indexes and configuration files that vary between sequencing batches. ### Categorizing Files by Regeneration Cost Regeneration cost provides a practical measure for classification. Files that require hours of computation or access to restricted data sources have high regeneration costs. Files that require seconds to produce from existing inputs have low regeneration costs. The framework recommends tracking files with low regeneration cost that document analysis decisions, while excluding files with high regeneration cost that can be referenced through metadata. For example, a STAR genome index built from a reference FASTA and annotation GTF requires substantial compute time and disk space. The index itself should not be committed to Git. However, the exact commands and reference versions used to build the index should be tracked in a configuration file. This approach preserves the ability to rebuild the index while keeping the repository small. The FAIR_Bioinfo training course demonstrates this principle by using Docker containers to capture the computational environment while keeping the versioned code repository focused on scripts and configuration [2]. ### Handling Configuration Drift Between Batches RNA-seq projects often process samples in batches with slightly different parameters. A common failure is overwriting a single configuration file with each new batch, losing the record of previous settings. The decision framework addresses this by recommending versioned configuration directories that separate stable analysis parameters from batch-specific settings. Create a `config/` directory with subdirectories for each analysis stage. Store stable parameters in files such as `config/alignment.yaml` and `config/quantification.yaml`. Store batch-specific information in files named by batch identifier, such as `config/batches/batch_2024_01.yaml`. When a new batch arrives, create a new batch file instead of modifying the existing one. This practice preserves the complete history of parameter changes and allows direct comparison between batches. The scPASU protocol for quantifying polyadenylation site usage demonstrates this pattern by making its Snakemake workflow configurable for organism-specific and sample-specific parameters [7]. ### A Scoring Matrix for File Classification A scoring matrix provides a systematic method for classifying files that fall into ambiguous categories. Score each file on three criteria using a simple scale. | Criterion | Score 0 | Score 1 | Score 2 | |-----------|---------|---------|---------| | Text format | Binary or compressed | Mixed text and binary | Plain text or markup | | Regeneration cost | Hours or restricted access | Minutes with available inputs | Seconds from tracked files | | Decision documentation | No analysis decisions recorded | Partial parameter information | Complete parameter and provenance record | Files with a total score of 4 or higher should be tracked in Git. Files with a score of 3 or lower should be excluded and referenced through metadata. This matrix resolves edge cases consistently. A small custom Python script that filters low-quality reads scores 2 for text format, 1 for regeneration cost, and 2 for decision documentation, giving a total of 5 and a recommendation to track. A BAM file scores 0 for text format, 0 for regeneration cost, and 0 for decision documentation, giving a total of 0 and a clear exclusion. ### Applying the Framework to Common RNA-seq File Types The framework produces consistent recommendations for the file types encountered in standard RNA-seq analysis. Raw FASTQ files score 0 on all criteria and are excluded. Quality control reports in HTML format score 1 for text format, 1 for regeneration cost, and 1 for decision documentation, giving a total of 3 and a recommendation to exclude. However, the MultiQC configuration file that determines which metrics appear in the report scores 2 for text format, 2 for regeneration cost, and 2 for decision documentation, giving a total of 6 and a clear recommendation to track. Alignment parameter files score 2 for text format, 2 for regeneration cost, and 2 for decision documentation. These files must be tracked because they record the exact settings that determine alignment outcomes. The AGTAR approach for transcriptome assembly and abundance estimation uses an adapted genetic algorithm with multiple tunable parameters [15]. The publication metadata for this method emphasizes that novel analysis approaches require careful documentation of parameters and implementation details [15]. The scoring matrix ensures that these parameters are preserved. Differential expression result tables score 1 for text format, 1 for regeneration cost, and 1 for decision documentation. These files are regenerable from the tracked scripts and configuration, so they should be excluded from Git. The scripts that generate them score highly on all criteria and are tracked. This separation keeps the repository focused on the analysis logic instead of the outputs. ### Recording the Decision Process The decision framework itself should be documented in the repository. Create a `docs/versioning_policy.md` file that explains which file types are tracked and which are excluded. This documentation serves two purposes. First, it helps collaborators understand why certain files appear in the repository and others do not. Second, it provides a reference point when new file types are introduced to the project. The versioning policy should include the scoring matrix and examples of its application to the specific project. For instance, a policy might state that all files with a score of 4 or higher are tracked, that raw sequencing data is always excluded, and that batch-specific configuration files are stored in a dedicated directory. The policy should be committed to Git and updated when the project requirements change. The European Bioinformatics Institute offers training pathways for bioinformatics data resources and practical analysis education [4]. These resources emphasize that data management and analysis documentation are core skills for computational biology. A written versioning policy demonstrates this skill and provides a concrete record of the decisions made during the project. ### Auditing the Repository Against the Framework Conduct a periodic audit of the repository to verify that files are classified correctly. The audit process involves listing all files in the repository, applying the scoring matrix to each file, and comparing the results with the actual tracking status. Files that are tracked but score below the threshold should be removed from Git history if possible. Files that are excluded but score above the threshold should be added to the repository. The audit also checks for missing files that should be tracked. Compare the repository contents with the analysis scripts and configuration files used in the most recent run. Any file that was modified during the analysis but does not appear in the repository should be added. This comparison requires a record of the files used in each analysis run, which can be obtained from workflow manager logs or shell history. The Galaxy Training Network provides accessible workflow training and reproducibility context [11]. Their materials emphasize that complete documentation is essential for reproducible analysis. A regular audit ensures that the repository remains complete and that the versioning policy is followed consistently. ### Handling Files That Change Frequently Some configuration files change with every analysis run. For example, a file that records the current sample list may be updated each time new samples are added. The decision framework recommends tracking these files but committing changes at meaningful intervals instead of after every modification. A sample metadata table should be committed when a new batch of samples is added or when experimental conditions change. This practice keeps the commit history readable while preserving the complete record of changes. Frequently changing files that do not document decisions should be excluded. Temporary files, process logs, and status files fall into this category. The `.gitignore` file should include patterns for these files to prevent accidental commits. The versioning policy should list the specific patterns used in the project. ### Comparing the Framework with Alternative Approaches Alternative approaches to file classification exist, but they have limitations that the scoring matrix addresses. A simple rule that tracks all text files and excludes all binary files fails for edge cases such as large text-based count matrices that are regenerable. A rule that tracks all files under a certain size fails for small binary files that contain critical intermediate results. The scoring matrix provides a more nuanced approach that considers format, regeneration cost, and decision documentation together. The Bioconductor project provides official documentation for reproducible genomic analysis, including guidance on managing analysis files [6]. Their materials emphasize that reproducibility requires attention to both code and data management. The scoring matrix complements this guidance by providing a concrete method for classifying files within a Git repository. ### Implementing the Framework in a New Project When starting a new RNA-seq project, implement the framework during the initial repository setup. Create the versioning policy document before adding any files to the repository. This ensures that the classification decisions are made deliberately instead of reactively. The policy should be reviewed at the start of each new analysis phase and updated when new file types are introduced. The initial commit should include the versioning policy, the README, and the directory structure. Subsequent commits should follow the classification decisions recorded in the policy. This approach creates a consistent repository that is easy to navigate and maintain. ### Troubleshooting Classification Disputes Disagreements about whether a file should be tracked can arise in collaborative projects. The scoring matrix provides an objective basis for resolving these disputes. When a collaborator questions a classification decision, apply the matrix to the file in question and compare the score with the threshold. This process removes personal preference from the decision and focuses the discussion on the file's characteristics. If the matrix produces a borderline score, consider the specific context of the project. A file that scores 3 in one project may score 4 in another if it documents decisions that are more critical. The versioning policy should note these context-dependent cases and provide guidance for handling them. ### Records Generated by the Framework The framework generates several records that support reproducibility. The versioning policy document records the classification decisions and their rationale. The commit history records when files were added or removed from the repository. The audit log records the results of periodic reviews. Together, these records provide a complete account of how the repository was structured and maintained. These records complement the other documentation in the repository. The sample metadata table records experimental design. The parameter files record analysis settings. The versioning policy records the file management decisions. This layered documentation ensures that every aspect of the analysis is preserved. The nf-core community maintains standardized pipelines with documented usage and configuration practices [8]. Their documentation emphasizes that reproducibility requires attention to both pipeline code and configuration management. The decision framework described here provides a method for applying these principles to individual RNA-seq projects. ## Frequently Asked Questions ### What is the difference between Git and GitHub? Git is a version control system that runs locally on your computer. It tracks changes to files in a repository. GitHub is a web-based hosting service for Git repositories. It provides remote storage, collaboration features, and a web interface. You can use Git without GitHub by keeping repositories only on your local machine, but remote hosting enables collaboration and backup. ### How do I add a large reference genome to my Git repository? Large reference genomes should not be added to Git. Instead, store the reference genome in a separate location and record its version and download source in a configuration file. The configuration file is tracked in Git, while the genome file itself is not. This approach keeps the repository small while preserving the information needed to reproduce the analysis. ### What should I include in my .gitignore file for an RNA-seq project? Your `.gitignore` file should exclude raw sequencing data, alignment files, reference genomes, and generated outputs. Common patterns include `*.fastq`, `*.fastq.gz`, `*.bam`, `*.bai`, `*.sam`, `*.gtf`, `*.fa`, `*.fna`, `*.fai`, and `*.bw`. Also exclude temporary files and logs. Keep scripts, configuration files, and documentation tracked in Git. ### How do I record the version of R packages used in my analysis? Create an environment file that lists all R packages and their versions. The `sessionInfo()` function in R provides version information for all loaded packages. You can save this output to a text file and commit it to the repository. Alternatively, use the `renv` package to create a lockfile that records package versions. ### Can I use Git with Snakemake or Nextflow? Yes, Git integrates well with workflow managers. The workflow definition file, such as a Snakefile or Nextflow script, is a text file that can be versioned in Git. Configuration files and environment specifications are also tracked. The workflow manager generates execution logs that complement the Git history by providing run-specific details. ### How do I share my analysis repository with collaborators? Push your repository to a remote hosting service such as GitHub, GitLab, or Bitbucket. Add collaborators to the repository so they can clone it and contribute changes. Use branches for experimental work and pull requests for code review. The nf-core community provides examples of collaborative pipeline development using Git [8]. ### What is a tag and when should I use one? A tag is a named reference to a specific commit. Tags are used to mark important points in the repository history, such as versions used for manuscript submission or publication. Create a tag with `git tag -a v1.0 -m "Analysis version for manuscript submission"`. Tags provide stable references that can be cited in publications. ### How do I handle data that cannot be shared publicly? For data with privacy restrictions, do not include the data files in the repository. Instead, record accession numbers or controlled-access identifiers in a metadata file. The repository documents the analysis code and configuration, while the data remains in a controlled-access repository. The NCBI provides databases with varying levels of access control [3]. ## Related Bioinformatics Guides - [RNA-Seq Data Analysis in Galaxy: A User-Friendly Platform](/knowledge/bioinformatics/rna-seq-data-analysis-in-galaxy-a-user-friendly-platform) - [RNA-Seq Data Analysis Workflow: From Raw Reads to Insights](/knowledge/bioinformatics/rna-seq-data-analysis-workflow-from-raw-reads-to-insights) - [Alternative Splicing Analysis from RNA-Seq Data](/knowledge/bioinformatics/alternative-splicing-analysis-from-rna-seq-data) - [Genomic Data Analysis Tools: A Comparative Guide for Researchers](/knowledge/bioinformatics/genomic-data-analysis-tools-a-comparative-guide-for-researchers) - [Lipidomic Analysis: A Beginner's Guide to Workflows and Data Interpretation](/knowledge/bioinformatics/lipidomic-analysis-a-beginner-s-guide-to-workflows-and-data-interpretation) ## Related Clinical & Scientific Guides * [A Practical Guide to Detecting Antimicrobial Resistance Genes in Shotgun Metagenomic Data](/knowledge/bioinformatics/a-practical-guide-to-detecting-antimicrobial-resistance-genes-in-shotgun-metagenomic-data) * [Computational Immunology: Modeling the Immune System](/knowledge/bioinformatics/computational-immunology-modeling-the-immune-system) * [How to Set Hard Filters for Germline Variant Calling: A Practical Guide to GATK Best Practices](/knowledge/bioinformatics/how-to-set-hard-filters-for-germline-variant-calling-a-practical-guide-to-gatk-best-practices) ## References and Further Reading [1] [Imputation method for single-cell RNA-seq data using neural topic model.](https://pubmed.ncbi.nlm.nih.gov/38000911). GigaScience, 2022. [2] [FAIR_Bioinfo: a turnkey training course and protocol for reproducible computational biology](https://doi.org/10.21105/JOSE.00068). 2021. [3] [NCBI Data Resources](https://www.ncbi.nlm.nih.gov/). National Center for Biotechnology Information. [4] [EMBL-EBI Training](https://www.ebi.ac.uk/training). European Bioinformatics Institute. [5] [The Carpentries Lessons](https://carpentries.org/lessons). The Carpentries. [6] [Bioconductor](https://bioconductor.org/). Bioconductor Project. [7] [scPASU: A computational protocol for quantifying polyadenylation site usage and alternative polyadenylation from 3' scRNA-seq data.](https://doi.org/10.1016/j.xpro.2026.104544). 2026. [8] [nf-core Documentation](https://nf-co.re/docs). nf-core. [9] [NextLongIso: a comprehensive Nextflow pipeline for multi-dimensional long-read RNA-seq analysis.](https://pubmed.ncbi.nlm.nih.gov/42544764). Bioinformatics (Oxford, England), 2026. [10] [Oqtans: the RNA-seq workbench in the cloud for complete and reproducible quantitative transcriptome analysis.](https://pubmed.ncbi.nlm.nih.gov/24413671). Bioinformatics (Oxford, England), 2014. [11] [Galaxy Training Network](https://training.galaxyproject.org/). Galaxy Project. [12] [CTEC: a cross-tabulation ensemble clustering approach for single-cell RNA sequencing data analysis.](https://pubmed.ncbi.nlm.nih.gov/38552307). Bioinformatics (Oxford, England), 2024. [13] [LncRAnalyzer: a robust workflow for long non-coding RNA discovery using RNA-Seq.](https://pubmed.ncbi.nlm.nih.gov/41103112). The Plant journal : for cell and molecular biology, 2025. [14] [ncRNAseq: simple modifications to RNA-seq library preparation allow recovery and analysis of mid-sized non-coding RNAs.](https://pubmed.ncbi.nlm.nih.gov/34841883). BioTechniques, 2022. [15] [AGTAR: A novel approach for transcriptome assembly and abundance estimation using an adapted genetic algorithm from RNA-seq data](https://doi.org/10.1016/j.compbiomed.2021.104646). Computers in Biology and Medicine, 2021. [16] [CMF-Impute: an accurate imputation tool for single-cell RNA-seq data.](https://pubmed.ncbi.nlm.nih.gov/32073612). Bioinformatics (Oxford, England), 2020. > This article is educational and does not replace validated analysis plans, institutional policy, clinical interpretation, or specialist review.