# How to Use Globus for Genomics Data Sharing: A Practical Guide By the end of this guide you will be able to put a directory of sequencing data on a Globus collection, grant a collaborator read or write access without giving them a shell account, move a multi terabyte project folder between two institutions with checksum verification, and script the whole workflow from the command line. You will also know which settings matter for genomics data storage, which ones you can safely ignore, and the mistakes that most often cause failed or insecure transfers. To follow along you need a laptop or workstation with a few gigabytes of free space, a set of test files (FASTQ, BAM, or anything else, real or synthetic), and about thirty minutes. If you want to run your own server side collection, you also need SSH access to a Linux server where you can install software. If your institution or sequencing core already runs a Globus collection, you can skip that part entirely and use theirs. All commands in this guide were written for the Globus command line interface (CLI) on Linux and macOS. The web interface steps use the current Globus web app and may shift slightly as Globus updates its interface. ## What Globus Does and Why It Fits Genomics Data Globus, operated by the University of Chicago as part of the Globus research data platform, is a hosted service for moving and sharing research data. You register a storage system with Globus, and the service then moves files between registered systems on your behalf. The transfer engine runs outside your machine, which matters for genomics in three concrete ways. First, genomic files are large and transfers fail. A whole genome alignment in CRAM format routinely runs 10 to 100 GB per sample, and a single sequencing run can produce directory trees with hundreds of thousands of FASTQ files. A transfer started with scp over a laptop Wi-Fi connection that dies at hour six has to start over. Globus transfers resume from where they stopped, keep going after your laptop disconnects, and verify file integrity with checksums by default when both endpoints support it. Second, genomic collaborations usually cross institutions. A collaborator at another university should be able to pull your processed VCFs without you creating a local account for them, opening a firewall port, or emailing a 40 GB archive in pieces. Globus handles authorization through identity providers, so your collaborator logs in with their own institutional credentials or a free Globus ID. Third, funders increasingly expect sharing infrastructure. The NIH Genomic Data Sharing Policy requires investigators to submit human genomic data to an approved repository and to share data from funded projects. The 2023 NIH [Data Management](/blog/guides/data-management-basics-principles-processes-and-best-practices) and Sharing Policy extends explicit sharing expectations across NIH funded data more broadly. Globus does not deposit data into repositories for you, but it is the standard plumbing that institutional and repository endpoints use to receive submissions and hand data back to you. The FAIR principles, published as findable, accessible, interoperable, and reusable guidance for scientific data management, give a useful checklist, and a well configured Globus collection with documented permissions directly addresses the accessible part of that checklist. ### The vocabulary you need before clicking anything Globus documentation uses a few terms precisely, and mixing them up causes most beginner errors. **Collection** (older docs say endpoint). A registered storage location that Globus can read from and write to. Everything else hangs off this concept. **Mapped collection**. A full storage system, usually a server or cluster filesystem, mounted into Globus by an administrator. On a mapped collection, each user sees their own paths, enforced by the connector between Globus and the filesystem. Typical examples are an HPC scratch filesystem or a university research storage service. **Guest collection**. A share carved out of a mapped collection. It points at one directory on the underlying system and has its own access control list. Guest collections are how you share one project folder without exposing the rest of the filesystem. **Globus Connect Personal (GCP)**. Free software you install on a laptop, desktop, or single lab server so it becomes a collection. It makes outbound connections only, so it works behind home routers and campus firewalls without port forwarding. **Globus Connect Server (GCS)**. The server version of the endpoint software, installed by an administrator with root access. Version 5 is the current generation. **Access control list (ACL)**. The list of who can read or write a collection or guest collection. Globus enforces these rules at the service level, independent of local Unix permissions. **Task**. A single Globus transfer or delete operation. Every task gets an ID you can use to monitor progress or troubleshoot later. **Sync**. Globus can skip files that already exist at the destination, at several levels of strictness, which turns a first transfer plus updates into an incremental pipeline. ## Before You Start Work through this checklist once. It prevents most of the errors described later in this guide. 1. Know which identity you will use. If your university is a member of InCommon or a similar federation, use your institutional login for anything involving human genomic data, because access records then tie to a real, verifiable identity. A Globus ID works fine for non-sensitive transfers. 2. Decide whether you will run your own collection or use an existing one. Ask your HPC or research computing staff whether they provide a Globus collection. Using theirs is almost always better for genomics data storage because those servers have fast disk, backup policies, and staff who fix problems. 3. Prepare test data that you do not mind moving. Do not learn transfer semantics on the only copy of a patient cohort. 4. Install the CLI now, even if you plan to use the web app, because scripted verification in Step 7 depends on it. ## Step 1. Create Your Globus Account Go to globus.org and choose Sign Up or Log In. You can authenticate with a Google account, an ORCID, a free Globus ID, or your institutional credentials through the federation. Pick one identity and stick with it, because collections you create and ACLs others grant you attach to that identity. Two practical notes. First, if you later need access to a collaborator's collection and you authenticate with a personal Gmail address, their compliance office may reject the access request. Second, a Globus ID is separate from any institutional identity, and some people accidentally create two accounts this way. Globus lets you link multiple identities to one account from the Account page in the web app, which is worth doing the day you create the account rather than the day a deadline is close. ## Step 2. Install Globus Connect Personal on Your Workstation GCP turns a single machine into a collection named after you, such as `yourname#laptop`. Download the installer for your operating system from globus.org/globus-connect-personal. On a personal Linux machine or an Apple silicon Mac, the flow is download, run setup, click through a browser authorization, then start the app. On Linux servers where you want a headless setup, the tarball includes a setup command that prints a URL you open in any browser. Install the CLI first, since the setup process for headless installs uses it: ```bash pip install globus-cli globus login ``` The `globus login` command prints a URL, you open it in a browser, log in, and paste the resulting code back into the terminal. Expected output of `globus login` on success is a single line: ``` OK. Login successful! ``` Confirm the identity the CLI is using: ```bash globus whoami ``` Expected output: ``` Username: you@university.edu Name: Your Name Id: 1a2b3c4d-5e6f-7890-abcd-ef1234567890 Email: you@university.edu ``` Now download and unpack GCP, then run setup. The exact file name changes with the version, so use whatever the download page gives you: ```bash tar xzf globusconnectpersonal-*.tgz cd globusconnectpersonal-*/ ./globusconnectpersonal -setup ``` The setup command prints a long URL. Open it, approve the request, and paste the returned key back into the terminal. On success you see a confirmation that the setup key was accepted. Start the endpoint: ```bash ./globusconnectpersonal -start & ``` From now on, your machine is reachable as a Globus collection whenever GCP is running. GCP makes only outbound connections to Globus, so no router changes are needed. ### Configure what GCP is allowed to share By default, GCP restricts readable and writable paths. On Linux and macOS this is controlled by a config file in the GCP directory. Open it and add the directory that holds your genomics data: ``` [_GLOBAL] Label = Genomics Workstation [Endpoint] Name = yourname#genomics-workstation [SharedEndpoint] ; Path = Alias /share/projects/ = /share/projects/ ``` Then restart GCP. If a later transfer fails with a path error even though the file clearly exists, the cause is almost always this allowlist, not the file itself. On the graphical versions for Windows and macOS you set shared directories through the app's preferences panel instead. ## Step 3. Use an Institutional Collection or Run Your Own Server Endpoint For anything beyond a one-off transfer, prefer a mapped collection on a server. Check whether your institution already runs one. The CLI can search the public endpoint catalog: ```bash globus endpoint search "university research storage" --limit 5 ``` Expected output is a table of matching collections with their IDs, owners, and display names. You can also simply log in to app.globus.org, open the File Manager, and browse. Your own institutional collections appear under "Groups and Shared With You" once your identity matches your organization. If you must run your own server endpoint, install Globus Connect Server version 5 following the official installation guide for your storage type, POSIX filesystem, S3-compatible object store, and others are supported. This requires root access and, for sharing to work, an organization subscription that allows guest collections. In practice, running your own GCSv5 endpoint is a job for a research computing group, not an individual lab member, so the rest of this guide assumes either an institutional mapped collection or your GCP endpoint. ## Step 4. Create a Guest Collection for Sharing ![Five step flow for creating a guest collection, naming it, granting a collaborator access, and revoking that access later.](/article-images/illustrations/globus-for-genomics-data-sharing-tutorial-guest-collection-setup.webp) This is the step where genomics projects become shareable. The goal is a single directory, for example `/projects/rgb_joint_calls/release_v2`, that named collaborators can read without seeing anything else on the system. In the web app at app.globus.org: 1. Open the File Manager and navigate to your mapped collection, for example `university#hpc-scratch`. 2. Move into the directory you want to share. 3. Click "Share" or "Add Guest Collection" and choose "Create New Guest Collection." 4. Give it a descriptive name such as "RGB Joint Calling v2 Release." The name is visible to collaborators, so make it meaningful. 5. Save. The new guest collection gets its own UUID. 6. Open the guest collection, go to the Share or Permissions tab, and click "Add Share," then "Grant Access." 7. Enter your collaborator's email address or Globus identity, choose a role, and save. Role choices matter: - **Access (read)** lets the person browse and download. Use this for collaborators receiving your results. - **Access (write)** lets them upload and delete. Use this for a sequencing center drop box or a two-way analysis handoff, and remove it when the exchange is done. - **Manager** lets them modify the ACL itself. Grant this rarely. Permissions can target an individual identity, a Globus Group, or everyone in an organization. For recurring multi-institution projects, create a Globus Group, add the project members, and grant the group one ACL entry. When a postdoc leaves the project you remove them from the group once and every collection the group can reach updates together. One structural fact worth internalizing: a guest collection is bounded by its mapped collection's sharing policy. On some institutional collections, sharing is disabled or restricted to certain paths. If the "Share" option is grayed out, that is a policy decision by your computing center, and the fix is an email to them, not a workaround. ## Step 5. Start a Transfer in the Web App With both endpoints registered, a transfer takes about two minutes of clicking. 1. Open app.globus.org and click File Manager. 2. In the left panel, select your source collection and navigate to the data. In the right panel, select the destination collection and navigate to the target directory. If the destination directory does not exist, use the New Folder button. 3. Click "Transfer or Sync" to link the two panels. 4. Select files or directories in the left panel and click the arrow. For a genomics project this is usually one project directory with `-recursive` semantics, which the web app applies automatically for directories. 5. Open "Transfer Options" and set a sync level and a label. Labels are free text and are the only reliable way to remember six months later what task `a1b2c3d4` was, so use labels like "CoV2_2026_batch3_raw_to_hpc." 6. Click "Start Transfer." Globus queues the task and runs it asynchronously. Expected on-screen result: a green confirmation with a link to the task page. The task page shows progress, files completed, and per-file errors. A 50 GB directory over a 1 Gbps connection typically takes on the order of ten minutes, but the honest answer is that speed depends on both endpoints, their disk systems, and the network path between them. ## Step 6. Automate With the Command Line The CLI mirrors everything the web app does and adds scripting. The core command is `globus transfer`. Transfer one directory, recursively, with checksum-level sync and a label: ```bash globus transfer \ a1b2c3d4-1111-2222-3333-444455556666:/data/rgb/batch3 \ e7f8a9b0-9999-8888-7777-666655554444:/scratch/rgb/batch3 \ --recursive \ --sync-level checksum \ --label "rgb batch3 raw reads to hpc" \ --notify failed ``` Expected output: ``` Message: transfer accepted, task submitted Task ID: 9f8e7d6c-1212-3434-5656-78789a9b0c1d ``` The task ID is your handle for everything that follows: ```bash globus task show 9f8e7d6c-1212-3434-5656-78789a9b0c1d ``` Expected output includes a status field (`ACTIVE`, `SUCCEEDED`, or `FAILED`), counts of files transferred, bytes transferred, and an errors count. To block until completion inside a script: ```bash globus task wait 9f8e7d6c-1212-3434-5656-78789a9b0c1d echo "exit code: $?" ``` `globus task wait` exits 0 on success and nonzero on failure, so you can chain it in a pipeline that triggers downstream steps. A minimal but production-sensible weekly sync looks like this: ```bash #!/usr/bin/env bash set -euo pipefail SRC="a1b2c3d4-1111-2222-3333-444455556666:/data/rgb/current" DST="e7f8a9b0-9999-8888-7777-666655554444:/scratch/rgb/current" LABEL="rgb nightly sync $(date +%F)" TASK_ID=$(globus transfer $SRC $DST \ --recursive \ --sync-level checksum \ --label "$LABEL" \ --notify failed \ | grep -oP 'Task ID: \K[a-f0-9-]+') globus task wait "$TASK_ID" globus task show "$TASK_ID" | grep -E "status|errors" ``` The endpoint IDs are the long UUIDs from `globus endpoint list`. Run `globus endpoint list` once, paste the UUIDs into a config file or environment variables, and stop typing them. ### Choosing a sync level ![Four Globus sync levels side by side with what each checks, where it fits in genomics, and what the check costs.](/article-images/illustrations/globus-for-genomics-data-sharing-tutorial-sync-level-choice.webp) The sync level decides what Globus considers "already done" at the destination. This choice is where transfer semantics meet bioinformatics data management discipline. | Sync level | What Globus checks | Sensible use in genomics | |, - |, - |, - | | exists | File exists at destination | Mirroring scratch space where content never changes | | size | File exists with same size | Fast re-runs of immutable archives | | mtime | Size and modification time match | Incremental sync of working directories | | checksum | Size and Globus-verified checksum match | Raw reads, BAMs, VCFs, anything downstream of publication | For raw and processed genomic data destined for analysis or publication, use `checksum`. The cost is one read pass per file on both ends, which is trivial compared with the cost of silently propagating a truncated FASTQ. For throwaway scratch mirroring, `exists` is faster and fine. By default Globus also verifies checksums on files it transfers, not just files it skips, whenever both endpoints support checksumming. You get integrity verification without asking for it. The `-preserve-mtime` flag keeps original modification timestamps, which keeps pipeline Makefiles and Snakemake timestamps sane on the receiving end. ## Step 7. Verify the Result A task reporting SUCCEEDED means Globus moved what it was asked to move. Verifying the data is correct is a separate habit, and in genomics it is non-negotiable. Three layers of verification, cheapest first: 1. Task-level checks. On the task page or via `globus task show`, confirm zero errors and the expected file and byte counts. An errors count of 3 on a 40,000 file transfer is three samples you may be missing downstream. 2. Manifest checks. Generate a manifest before transfer and check it after: ```bash # On the source, before transfer cd /data/rgb/batch3 find . -type f -name "*.fastq.gz" -exec md5sum {} \; > manifest.md5 # On the destination, after transfer cd /scratch/rgb/batch3 md5sum -c manifest.md5 | tail -5 ``` Expected output: ``` sample_RG01_R1.fastq.gz: OK sample_RG01_R2.fastq.gz: OK sample_RG02_R1.fastq.gz: OK sample_RG02_R2.fastq.gz: OK sample_RG03_R1.fastq.gz: OK ``` Every line must end in OK. Any FAILED line means retransfer that file and investigate why it diverged. 3. Downstream checks. For critical releases, run a tool-level sanity pass, for example `samtools quickcheck *.bam` on delivered BAMs, before telling collaborators the data is ready. This catches corruption that predates the transfer, which is a real failure mode that Globus cannot detect because the source file was already bad. ## Step 8. Share Results and Collect Data From Collaborators Sharing in the other direction uses the same guest collection machinery. **Distributing results to a collaborator.** Create a guest collection on your project directory, grant their identity read access, and send them the collection link. They open it in the Globus web app, log in with any free identity, and download files in a browser or initiate a transfer to their own collection. No VPN, no FTP credentials, no 40 GB email attachments. **Collecting incoming data.** Create a guest collection pointing at an inbox directory such as `/projects/rgb/inbox`, grant your collaborator write-only-style access (write access without read is possible on GCSv5, and is the polite default for drop boxes), and give them the link. Their upload lands in your inbox without them seeing anything else. **Public or semi-public release data.** For data accompanying a paper, a guest collection with an HTTPS link lets reviewers and readers download files in a browser after a quick login. For genuinely public releases, deposit in a repository such as SRA or an institutional archive, because repositories provide [persistent identifiers](/knowledge/bioinformatics/persistent-identifiers-in-bioinformatics-selection-criteria-and-implementation-workflow) that a Globus collection does not. The two approaches compose: use Globus to push the deposit to the repository's endpoint, then publish the repository accession. **Housekeeping.** ACLs persist until removed. When a collaboration ends, revoke access from the guest collection's permissions tab, or remove the person from your Globus Group if that is how you granted access. Audit a quarter's worth of active shares twice a year. Access you forgot about is the most common way a casually shared genomics directory becomes a problem. ## Genomics-Specific Tips for Bioinformatics Data Management Globus moves bytes, but the surrounding habits determine whether your genomics data storage is usable a year from now. **Keep raw data immutable.** Treat raw FASTQ and BCL directories as read-only after initial QC. Transfer them once with checksum-level sync, then never modify them in place. Processed results live in separate, versioned directories such as `release_v1`, `release_v2`. When you share, share the release directory, not the working directory. **Ship manifests with every batch.** The `md5sum` manifest from Step 7 costs seconds to generate and saves hours of argument about whether a truncated file came from your sequencer or the network. Include it in every guest collection you share. **Name transfers like you name samples.** Task labels are your transfer log. "batch3_to_hpc" beats "files" the same way sample sheet naming beats IMG_4032. If you run scheduled syncs, put the date in the label. **Batch small files thoughtfully.** Globus handles many-file transfers, but millions of small files are slower per byte than a few large ones on every transport, not just Globus. If you routinely move tens of millions of tiny index or fragment files, tar them first. FASTQ directories with tens of thousands of files transfer fine as-is. **Plan for repository deposition from the start.** The NIH Genomic Data Sharing Policy expects submission of human genomic data to an approved repository, and the broader Data Management and Sharing Policy expects a plan describing how data will be shared. Practical implication for your directory structure: keep the eventual submission package (read groups, sample metadata, consent documentation) in a directory layout that can be transferred wholesale to a repository endpoint when the time comes. Retro-fitting metadata after two years of ad hoc folders is the slowest possible way to meet a grant milestone. **Treat FAIR as a sharing checklist.** The FAIR principles emphasize that data should be retrievable by identifier and accessible through a well defined protocol. A Globus guest collection satisfies retrievable-by-authorized-protocol for your active project window, while a repository accession satisfies findable and persistent for the long term. Use each tool for the stage it fits. ## Key Options Reference | Option or concept | Where it applies | What it does | When to use it | |, - |, - |, - |, - | | `-recursive` | `globus transfer` | Includes directory contents | Almost always, for project directories | | `-sync-level checksum` | `globus transfer` | Skips only files with matching checksum | Raw reads, BAMs, VCFs, release data | | `-sync-level exists` | `globus transfer` | Skips files present at destination | Scratch mirroring, cheap incremental syncs | | `-preserve-mtime` | `globus transfer` | Keeps original timestamps | Pipelines that depend on file times | | `-label "text"` | `globus transfer` | Names the task in logs | Every transfer, no exceptions | | `-notify failed` | `globus transfer` | Emails only on failure | Scheduled jobs, so success is quiet | | Guest collection ACL | Sharing tab | Who can read or write a directory | Grant per project, revoke when done | | Globus Group | Web app, account area | Manage many people with one ACL | Multi-institution, long-running projects | | `globus task wait` | CLI | Blocks until the task finishes | Scripts that trigger downstream steps | | HTTPS access | Collection settings | Browser downloads from a collection | Reviewers, small-file handouts | ## Common Mistakes and Limitations **Sharing your home directory root.** A guest collection should point at one project directory. Granting access at the top of your home directory exposes `.ssh`, credential files, and every other project. Create a dedicated share directory and put guest collections under it. **Using sync level exists for critical data.** With `exists`, a file interrupted mid-write at destination and never completed still counts as "already there." For raw and processed genomics data, checksum is the correct level, and the extra read pass is cheap insurance. **Assuming GCP runs without you.** Globus Connect Personal is an application on your machine. If your laptop sleeps, is off, or the app was quit, transfers to and from it wait. Scheduled transfers to a laptop endpoint fail for boring reasons. For anything scheduled, use a server-side collection. **Ignoring the errors count.** A task that reports SUCCEEDED with 12 errors did not deliver 12 files. This bites people who transfer 30,000 small files where a few hit transient destination errors. Check the per-file error list on the task page and retransfer just the failures, which sync levels make cheap. **Symlinks and odd file names.** Globus transfers symlinks as symlinks rather than following them to data elsewhere, so a directory full of links to a shared reference directory arrives as links, not genomes. Also, destination filesystems vary in case sensitivity and tolerated characters, and a source file named `Sample [batch1].fastq.gz` can fail or collide at the destination. Sanitize names at the sequencer export step. **Forgetting that ACLs are separate from Unix permissions.** A collaborator with Globus write access to a guest collection writes through the Globus service, and the files land with the permissions of the account the mapped collection runs under. Conversely, giving someone a Globus ACL does not give them SSH access. Plan ownership and group-write bits on the destination side for your local pipelines. **Treating Globus as backup.** Globus moves and shares data. Deleting a file at the source and deleting it through a guest collection delete the same file. Your genomics data storage still needs a real backup policy with versioning or snapshots, which Globus does not replace. **Human genomic data compliance is yours, not the tool's.** Globus enforces who can reach a collection, and organizations with a subscription can operate managed collections under a business associate agreement for regulated data. It does not de-identify samples, decide consent scope, or approve your data use. For human data, follow your institution's compliance process and the NIH Genomic Data Sharing Policy expectations on controlled access, and involve your compliance office before the first share, not after. **CLI sessions expire.** The CLI caches credentials and they eventually lapse. If a script that worked for months starts failing with an authorization error, run `globus login` again interactively before assuming anything is broken. **One individual case note, stated once.** Transfer, sharing, and compliance decisions for human genomic data depend on your institution, consent terms, and funder, and the right answer is project specific. Consult your institution's research compliance office for those decisions. ## Frequently Asked Questions ### Is Globus free to use? Yes for the core use case in this guide. Globus Connect Personal, the web app, the CLI, and standard transfers between collections are free, funded through subscriptions that research organizations pay for advanced features. Check your institution first, because you may already be covered. ### How is Globus different from scp or rsync? Globus runs transfers as asynchronous tasks on a service, so they survive disconnects, verify integrity, and are monitored from anywhere, while scp and rsync die when your session does. Rsync remains excellent for one machine to another over a stable link, Globus wins when transfers are large, cross-institutional, shared with third parties, or scheduled. ### Can I use Globus with my university HPC cluster? Almost certainly, either through a collection your HPC staff already operates or by asking them to install Globus Connect Server version 5. Moving data between your workstation and cluster scratch through Globus replaces rsync-over-jump-host workflows and does not require an interactive shell session to keep running. ### How do collaborators without any account download my files? They open your guest collection link, log in with any free identity such as Globus ID, Google, or their institutional credentials, and download through the browser or transfer to their own collection. Sending them the link plus the access grant is all you do on your side. ### How fast will my transfers be? Speed depends on both endpoints, their disk systems, and the network between them, not on your laptop alone. Globus is tuned for long-haul research network paths and commonly performs far better than a single-stream scp between the same two sites, but a transfer is only as fast as the slowest endpoint's disk or uplink. ### Can two people share directly between their laptops? Yes. Both install Globus Connect Personal, both endpoints appear in the web app, and either side can push to the other, provided both machines are on and running GCP. This works through home routers because GCP makes only outbound connections. ### Does Globus see or store my file contents? Globus coordinates transfers and does not keep copies of your file contents on its servers, and transfers over the wire use TLS encryption. Metadata such as file listings for the collections you use is visible to the service in order to function, so for maximally sensitive local data, keep read paths restricted in your GCP config. ### What do I need to do differently for human genomic data? Use an institutional identity, restrict shares to named individuals or vetted groups, revoke access when collaborations end, and route the project through your institution's controlled-access process in line with the NIH Genomic Data Sharing Policy before sharing outside the study team. Involve your compliance office early, and keep controlled-access data off casual personal endpoints.