# Ensembl BioMart Export Fails or Times Out: How to Fix Most Ensembl BioMart export failures come from one cause: the query asks the web service for more than it is allowed to compute in a single request. BioMart runs your filters and attributes against the Ensembl databases on a server that enforces a time and memory budget per query. When a job exceeds that budget, the service stops it, and you see an endless spinner, an empty downloaded file, a browser timeout message, or an HTTP 504 gateway timeout error. The first thing to check is the Count button. Build your query as usual in the BioMart interface, click Count, and read the number it reports. Count tells BioMart to calculate how many rows your query would return without exporting anything. If Count returns a modest number, the service is up and your query is valid, so the problem is probably the export format, an expired session, or your browser. If Count returns tens of thousands of rows or hangs itself, query size is the problem, and the fix is to shrink the query or move the job to a tool built for large exports. This article explains what happens during an Ensembl BioMart export timeout, works through the causes from most common to rarest, gives a diagnostic sequence you can follow in about ten minutes, and shows the batching scripts and bulk-download routes that solve the problem permanently. ## What Happens During an Ensembl BioMart Export Timeout ![Flow of a BioMart query: dataset, filters and attributes become SQL joins, the server budget runs out, and you get a timeout or empty file.](/article-images/illustrations/ensembl-biomart-export-timeout-timeout-mechanism.webp) BioMart is the batch query and export interface connected to Ensembl, the genome database produced by EMBL-EBI, the [European Bioinformatics Institute](/knowledge/bioinformatics/the-european-bioinformatics-institute-embl-ebi). You can open it at https://www.ensembl.org/biomart. Where the [Ensembl genome browser](/blog/guides/ensembl-genome-browser) shows one gene or one region at a time, BioMart answers questions such as "give me the coordinates, symbols, and mouse orthologs for every gene on human chromosome 22" and returns the answer as a table or a file. Three terms describe every BioMart query: - A **dataset** is the species-specific collection you query, for example the Homo sapiens genes dataset. - A **filter** restricts which rows come back, for example a chromosome, a gene type such as protein_coding, or a list of Ensembl gene IDs you paste in. - An **attribute** is a column you want returned, for example Gene stable ID, Gene name, or Mouse gene stable ID. When you click Results, BioMart translates your choices into a database query. Every attribute you add is another table the server must join to your rows. Ortholog attributes, which pull the matching gene in another species, join across large cross-species tables. Gene Ontology attributes join the annotation tables. Sequence export joins coordinate lookups for every region you requested. A query that looks small on screen can therefore trigger a long chain of joins across hundreds of thousands of rows. The web service enforces a maximum processing time and memory allowance per query. If the job is still running when that budget runs out, the service kills it. Depending on how far the query got, you see different symptoms: the page spins and then fails, the browser reports the site took too long to respond, the download finishes but the file is empty or truncated, or the server returns a 504 error, which means an intermediate server gave up waiting for the answer. The biomaRt package in R talks to the same BioMart service over HTTP, the standard web protocol, so the same limits apply there. A query that fails in the browser will usually fail in biomaRt as well, often with a message saying the query returned an invalid result. The causes and fixes below cover both routes. ## First Checks: A Ten-Minute Diagnostic Sequence ![Decision path asking whether Ensembl is up, whether Count is modest, and whether a minimal query works, ending in batching or FTP.](/article-images/illustrations/ensembl-biomart-export-timeout-ten-minute-triage.webp) Work through these steps in order. Each one either fixes a common failure or narrows the cause. 1. **Check whether Ensembl itself is up.** Open the Ensembl home page. A banner or a maintenance notice means the problem is on their side. Ensembl announces scheduled downtime in advance on the home page and the Ensembl blog, usually around release updates. 2. **Run Count.** Click Count instead of Results. If Count returns a number you can act on, the service is reachable and your query is structurally valid. 3. **Compare Count with your expectation.** If you pasted 500 gene IDs and Count reports 40,000 rows, a filter is not doing what you think, most often because the ID list was not applied or the wrong filter type was used. 4. **Preview rows.** On the Results page, look at the first rows shown on screen. If the preview works but the full export fails, size is confirmed as the cause. 5. **Test a minimal query.** Rebuild the query with one known ID and two attributes. If that works, add attributes back in small groups until the failure reappears. The last group you added contains the expensive attribute. 6. **Switch the export format to TSV.** TSV, tab-separated values, is the plainest format BioMart offers. If TSV succeeds where HTML or XLS failed, the format was the problem. 7. **Retry in a private browser window with extensions disabled.** This rules out cached pages, pop-up blockers, and cookie interference. 8. **If every small query works, stop fighting the web form.** Batch your IDs with biomaRt, use the REST API, or download the bulk files from the Ensembl FTP site. The sections below give working code for each route. ## Troubleshooting Table: Symptom, Likely Cause, Confirmation, and Fix | Symptom | Likely cause | How to confirm | Fix | |--|--|--|--| | Spinner runs for minutes, then a blank page or a browser timeout | Query too large for the web service | Count returns a very large number, or Count itself hangs | Cut attributes, batch IDs, then move the job to biomaRt, the REST API, or FTP files | | Export finishes but the file is empty or truncated | Server stopped the query before writing output, or export was limited to the first rows | Compare the line count in your file with the Count value, and check the export checkboxes on the Results page | Export as TSV, select export all results, reduce query size | | HTML or XLS export fails but TSV works | Format overhead on large results | Repeat the identical query with TSV selected | Use TSV for anything large and convert locally | | biomaRt reports the query returned an invalid result | Timeout, dropped connection, or a version mismatch after an Ensembl release | Run listMarts() and listDatasets(), then test a one-gene query | Update biomaRt, batch the query, retry with increasing waits | | Zero results with no error message | Wrong dataset or ID type for the filter | Test one ID you know is correct, and check the dataset name in the left panel | Match IDs to the right filter and species, strip version suffixes | | Immediate error when you click Results | Malformed input list, such as blank lines, mixed species, or symbols in a stable ID filter | Inspect the pasted list and test a single ID | One ID per line, trim whitespace, remove version suffixes | | Works from home, fails on a campus or corporate network | Proxy, VPN, or firewall interfering with the long response | Try another network or a cloud machine, or ask IT whether the domain is filtered | Run the job from an allowed host, or download FTP files on an unrestricted machine | | The whole site is unreachable, including the home page | Scheduled maintenance or a release switchover | Check the Ensembl home page banner or the blog | Wait for the notice to clear, use the archive site for older releases | | File has fewer rows than Count reported | Export limited to the first rows, or the unique-results option removed duplicates | Compare the file line count with Count and review the checkboxes | Tick export all results and review the unique-results setting | | Export succeeds but takes many minutes | Borderline query size | Time a reduced version of the same query | Batch anyway, and schedule large jobs off peak | ## Ensembl BioMart Export Timeout Causes, From Most Common to Rarest ### 1. The query is too large This is the cause behind the majority of Ensembl BioMart export timeout reports. A query such as "all protein-coding genes, with gene ID, symbol, description, GO terms, protein domains, and orthologs in five species" asks the server to join many large tables for roughly twenty thousand human genes. The job exceeds the per-query processing budget, the service stops it, and the export fails or returns nothing. The mechanism matters because it tells you what to change. The cost of a BioMart query grows with the number of rows your filters select, multiplied by the number and type of attributes. Reducing either side helps. The Count button is the direct measure of the row side, and the attribute list is the direct measure of the join side. A query that returns a Count in the low thousands with a handful of simple attributes almost always exports cleanly, while the same filters with a dozen heavy attributes often fail. The fix hierarchy, from least to most effort, is: remove attributes you do not need, restrict the filters further, split the query into chromosome-sized or list-sized chunks, and finally move the job off the web form entirely using biomaRt, the REST API, or the bulk files described later. ### 2. Too many attributes, especially orthologs and GO terms Each attribute is a join, and joins are not equal. Gene stable ID and gene name are cheap. Ortholog attributes require a cross-species join for every row. GO term attributes pull the annotation tables. Variation attributes are among the most expensive because they connect to the largest tables in Ensembl. Adding orthologs for five species multiplies the work fivefold. The practical rule is to request only the columns you will actually use, and to add attributes in small groups during testing. If a query with IDs, symbols, and coordinates exports in seconds but fails after you add mouse and zebrafish orthologs, you have found the expensive part. You can then split the job: export the core table first, export orthologs as a separate query keyed on the same gene IDs, and join the two files locally in R, Python, or a spreadsheet. ### 3. A large pasted ID list Pasting thousands of IDs into a filter is a common pattern after differential expression analysis, and it is a common trigger for an Ensembl BioMart export fails or times out situation. The list itself becomes part of the query the server must process, so a filter with twenty thousand IDs is a heavy query even with only two attributes. Ensembl does not publish a fixed maximum list length, because the practical limit depends on everything else in the query. Lists in the hundreds behave reliably. For thousands of IDs, split the list into batches of a few hundred and run them in sequence, keeping the attributes minimal. The biomaRt batching script later in this article automates exactly this, including retries when an individual batch times out. ### 4. The wrong export format The Results page offers several formats, including HTML, XLS, and TSV. HTML and XLS require the server to format every row, which consumes memory and time on top of the database work. TSV is a plain stream of tab-separated text and is the most robust choice for large exports. A useful confirmation is to rerun the identical query with TSV selected. If TSV succeeds where the formatted export failed, you have your answer. Make TSV the default for anything with more than a few thousand rows, and convert to other formats on your own machine, where a failure costs you nothing. ### 5. Input ID problems that stall or empty a query These problems usually produce zero results rather than a timeout, but they are worth ruling out because they waste diagnostic time. The common variants are: - **Versioned IDs.** Ensembl identifiers carry a version after a dot, for example ENSG00000139618.12. Many BioMart filters expect the unversioned form, and Ensembl provides separate filters for IDs with versions. Strip the suffix or select the matching filter. - **Wrong ID type.** Pasting HGNC gene symbols into the Gene stable ID filter returns nothing. Use the symbol filter for symbols and the stable ID filter for Ensembl IDs. - **Whitespace and line endings.** Lists copied from Excel on Windows often carry trailing carriage returns. One ID per line, trimmed, avoids this. - **Mixed species.** A list of mouse IDs pasted into the human dataset returns nothing or unpredictable results. Check the dataset name in the left panel before you paste. - **Old IDs.** Ensembl stable IDs mostly persist between releases, but versions increment and some IDs retire or merge. A list built years ago can return fewer hits on the current release. The archive site at https://archive.ensembl.org lets you query the release your list came from. ### 6. The wrong portal, dataset, or release Ensembl covers vertebrates. Its sister resource, Ensembl Genomes, covers plants, fungi, bacteria, protists, and invertebrate metazoa, and runs its own BioMart at https://biomart.ensemblgenomes.org. Querying a plant or bacterial species on the vertebrate BioMart fails, and the reverse is also true. If your species is not vertebrate, switch portals before debugging anything else. Release confusion is the second variant. BioMart always queries one specific Ensembl release, and the release number appears on the site. Results that made sense last month can change after a release update, because annotations are revised and attribute sets are adjusted. If you need data exactly as it was in an earlier release, use the archive site rather than trying to reproduce old results on the current one. ### 7. Expired sessions and browser problems BioMart stores your in-progress query on the server between clicks. If you build a query, leave the tab idle for a long stretch, and then click Results, the session can expire and the export fails or returns nothing even though everything looked correct. Rebuilding the query in one sitting is the reliable fix. Browser-side issues are less common but real. Pop-up blockers can interfere with the results window in some configurations. Extensions that strip cookies break session handling. A cached page from just before a release switch can show stale errors. A private window with extensions disabled is a clean test that takes one minute. ### 8. Scheduled maintenance and release switchover Ensembl runs scheduled maintenance, announced in advance on the home page and the blog, and the period around a release update can be slow while databases are swapped. During these windows, exports that normally work can time out, and the whole site can be briefly unreachable. There is nothing to fix on your side. Check the announcements, wait, and use the archive site if you need an older release in the meantime. ### 9. Network, proxy, and firewall issues A BioMart export is a normal HTTPS request that waits on a long-running server-side job. Anything between you and Ensembl that cuts long-lived connections early will kill large exports while leaving small ones working. Campus proxies, corporate SSL inspection appliances, VPNs, and captive portals all do this. The signature of this cause is location-dependent behavior: the same query works from home and fails on the office network, or works for colleagues and fails for you. Confirm by trying another network or a cloud or server machine you control. If your institution shares one outgoing internet address, heavy use by many people can also lead to throttling. For institutional-scale work, the bulk-download route is the better fit, because it moves the load off the interactive service. ### 10. biomaRt-specific failures biomaRt, the Bioconductor R package, sends your query to the same BioMart web service, so every cause above applies. Three failure modes are specific to the package. First, after an Ensembl release update, dataset names and attribute names can change. A script that worked last release can start reporting that the query returned an invalid result, or that the dataset does not exist. Verify the connection with listMarts() and listDatasets(), find the current dataset with searchDatasets(), and confirm attribute names with searchAttributes(). Keeping biomaRt updated through BiocManager reduces this class of failure. Second, large queries time out exactly as they do in the browser, and the package surfaces it as the invalid-result message. The batching script below is the standard fix. Current biomaRt versions use the httr2 package for HTTP, and the package vignette describes how to adjust request options such as timeouts for your setup. Third, connection to the wrong service. biomaRt provides useEnsembl() for vertebrates and useEnsemblGenomes() for the plants, fungi, bacteria, protists, and metazoa datasets. Ensembl once operated regional mirrors that both the website and biomaRt could target, but the infrastructure has been consolidated in recent years, so check the current Ensembl website and the biomaRt vignette before relying on a mirror setting. ### 11. Rate limiting from repeated automated queries BioMart does not publish a per-user quota, but hammering the service in a tight loop from one address degrades it for everyone and invites throttling. The related Ensembl REST API does publish its limit, 33 requests per second according to the Ensembl REST documentation, and the same etiquette applies to BioMart. Put a delay between requests, run large batch jobs off peak, and cache results to disk so a crashed script does not refetch everything. If you share an institutional network, coordinate with colleagues running similar jobs. ## How to Fix an Ensembl BioMart Export Timeout: Solutions That Scale Once a query is genuinely too big for the web form, the durable solutions are programmatic batching and bulk files. Pick the route that matches your job size. | Job | Best tool | |--|--| | One gene or one region, visual inspection | Ensembl genome browser | | Up to a few thousand rows, one-off | BioMart web interface with Count first | | Thousands to tens of thousands of rows | biomaRt with batching, or the REST API | | Whole genome or whole dataset | Ensembl FTP files, or the public MySQL server | | A recurring pipeline | biomaRt script, or a local copy of the data | ### Batch your IDs with biomaRt This script reads a list of Ensembl gene IDs, splits it into batches of 500, retries each batch with increasing waits, and binds the results into one table. Adjust the attribute list to what you need. ```r library(biomaRt) ensembl <- useEnsembl(biomart = "genes", dataset = "hsapiens_gene_ensembl") fetch_batch <- function(batch, mart) { tryCatch( getBM( attributes = c("ensembl_gene_id", "hgnc_symbol"), filters = "ensembl_gene_id", values = batch, mart = mart ), error = function(e) NULL ) } ids <- unique(trimws(readLines("my_ids.txt"))) chunks <- split(ids, ceiling(seq_along(ids) / 500)) results <- lapply(chunks, function(batch) { out <- NULL for (attempt in 1:5) { out <- fetch_batch(batch, ensembl) if (!is.null(out) && nrow(out) > 0) break Sys.sleep(30 * attempt) } out }) all_results <- do.call(rbind, results) write.table(all_results, "biomart_export.tsv", sep = "\t", row.names = FALSE, quote = FALSE) ``` Two habits make this robust. Save each batch result as you go, so a failure on batch 40 does not cost you the first 39. And log which batches returned NULL, so you can rerun only the failures. A delay between batches, even a few seconds, keeps you a polite citizen of a shared public service. ### Use the Ensembl REST API for moderate ID lists The Ensembl REST API returns JSON over plain HTTPS and is designed for scripted lookups. Ensembl documents a rate limit of 33 requests per second for this service, so add a small delay between calls. The lookup endpoint accepts one ID at a time: ``` curl -H "Content-Type: application/json" "https://rest.ensembl.org/lookup/id/ENSG00000139618?expand=1" ``` It also accepts batches of IDs by POST, which is the efficient way to process a list: ``` curl -X POST -H "Content-Type: application/json" \ -d '{"ids":["ENSG00000139618","ENSG00000141510"]}' \ "https://rest.ensembl.org/lookup/id" ``` The REST API is best for ID mapping and lookups in the thousands. For whole-table exports, use the next route instead. ### Download bulk files from the Ensembl FTP site For whole-genome or whole-dataset needs, do not use BioMart at all. Ensembl publishes the underlying data as files on its FTP site at https://ftp.ensembl.org, organized by release. The annotation files you will use most are GTF, the gene annotation format listing every feature with coordinates and attributes, and FASTA, the sequence files. To fetch the human gene annotation for release 112, replacing the release number with the current one shown on the Ensembl home page: ``` curl -O https://ftp.ensembl.org/pub/release-112/gtf/homo_sapiens/Homo_sapiens.GRCh38.112.gtf.gz ``` To fetch all human cDNA sequences: ``` curl -O https://ftp.ensembl.org/pub/release-112/fasta/homo_sapiens/cdna/Homo_sapiens.GRCh38.cdna.all.fa.gz ``` The FTP site also provides tab-delimited BioMart-format dumps for many species, which reproduce the most common BioMart tables without touching the web service. Test the integrity of any download before using it: ``` gunzip -t Homo_sapiens.GRCh38.112.gtf.gz ``` A short safety note on this step: compressed genome files expand to many times their download size, sometimes by a factor of ten or more. Check free disk space before decompressing a whole-genome file, and work on a machine or server with room for the expanded data. Once the files are local, extract exactly the rows and columns you need with standard tools such as gffread, bedtools, awk, or your language of choice. This route never times out, because the transfer is a plain file download and the processing happens on your hardware. ### Query the public Ensembl MySQL server Ensembl makes its databases readable directly through a public MySQL server at ensembldb.ensembl.org on port 3306 with the username anonymous. Database names follow the pattern species_core_release_assembly, for example homo_sapiens_core_112_38 for human, release 112, GRCh38 assembly. This route suits SQL-comfortable users who need custom joins that BioMart cannot express, and it is the foundation for running BioMart or other tools locally against a copy of the data. ### Run BioMart locally The BioMart software itself is open source and can be installed at your own institution, pointed at locally hosted Ensembl MySQL dumps from the FTP site. This removes the public service time limits entirely and is the standard answer for groups that run large or frequent exports. It requires more setup than any other option here, so treat it as the last rung of the ladder, worth climbing only when batched remote queries are still too small for your needs. ## How to Use Ensembl: Browser, BioMart, and Bulk Files Readers searching for how to use Ensembl often land on this problem because BioMart is one of the first tools they try. A short orientation prevents most export failures before they start. The Ensembl genome browser is the right tool when your question is about one gene or one region. You search for a gene, land on its page, and read transcripts, sequence, variation, homology, and regulation in context. The region view also offers its own export menu for the coordinates or sequences you are looking at, which covers many small export needs without BioMart. BioMart is the right tool when your question is about a list. Give it a filter, such as a chromosome, a gene type, or your own ID list, give it attributes, and get a table. The single habit that prevents most timeouts is running Count before every export and keeping the attribute list minimal. For everything at genome scale, the FTP files are the right tool, as described above. If you are new to the interface, Ensembl publishes free step-by-step browser tutorials and recorded webinars on its own training pages, and the Ensembl blog walks through new features at each release. A one-hour pass through the official tutorial pays for itself the first time you need a nontrivial export. ## Prevention Practices These habits keep Ensembl BioMart exports from failing again: - **Run Count before every export.** It takes seconds and tells you whether the query is web-form sized or pipeline sized. - **Request only the columns you need.** Add ortholog, GO, and variation attributes only when the analysis requires them, and split heavy attribute groups into separate queries you join locally. - **Make TSV your default format.** Convert to other formats on your own machine. - **Batch large ID lists.** A few hundred IDs per query is a reliable working size, automated by the biomaRt script above. - **Save your query.** Copy the URL or the query XML from the Results page so a failed session can be rebuilt exactly, and record the Ensembl release number alongside every exported file. - **Keep biomaRt current.** Update it through BiocManager around each Ensembl release, and re-verify dataset and attribute names in your scripts. - **Cache aggressively.** Save every successful export to disk with the release number in the filename, so crashed scripts and follow-up questions do not refetch data. - **Schedule big jobs off peak and with delays.** Shared public services stay healthy when heavy users spread their load. - **Spot-check output.** Open a few records from any large export in the genome browser and confirm the values match the gene pages before building downstream analysis on the file. ## Common Mistakes and Limitations These are the errors that cost the most time in practice: - **Pasting versioned IDs into a filter that expects unversioned IDs.** Ensembl provides separate filters for IDs with versions. Match the ID form to the filter, or strip the suffix after the dot. - **Pasting gene symbols into the stable ID filter, or mouse IDs into the human dataset.** Both return zero rows and look like a BioMart failure when the query itself is mismatched. - **Selecting every available attribute "just in case."** This is the single fastest route to a timeout. Every attribute is a join, and ortholog and variation attributes are the most expensive. - **Choosing HTML or XLS for a large export.** The formatting overhead alone can push a borderline query over the limit. TSV is the dependable format. - **Looping against the service with no delay and no caching.** A crashed script at batch 60 should not refetch batches 1 through 59. Save as you go, sleep between calls, and log failures. - **Assuming the browser and BioMart can disagree.** They read the same underlying Ensembl databases for the same release, so a discrepancy almost always means the two queries differ in release, dataset, or ID handling rather than in data. - **Using BioMart for whole-genome pulls.** The web service is built for interactive queries. Genome-scale work belongs in FTP files or a local database, full stop. On limitations: attribute and filter names change between Ensembl releases, so scripts that call BioMart or biomaRt need periodic maintenance, and the release number should be recorded with every export. BioMart is a discovery and export tool, not a curated per-gene review, so verify records that matter against the gene page and the primary annotation before committing an analysis to them. Annotation itself differs between sources such as Ensembl and RefSeq, so document which source and release produced each dataset you keep. ## Frequently Asked Questions ### Why does my BioMart export finish but the downloaded file is empty? The most common reason is that the server stopped the query before it finished writing output, which leaves an empty or truncated file. Rebuild the query with fewer attributes, run Count first, and export as TSV. Also check that the export option is set to all results rather than the first few rows. ### How many IDs can I paste into a BioMart filter at one time? Ensembl does not publish a fixed maximum, so treat it as a practical limit that depends on the rest of your query. Lists in the hundreds usually work well. For thousands of IDs, split them into batches or move the job to biomaRt or the REST API. ### Is there a hard row limit on BioMart exports? There is no published row limit. The practical limit comes from the server time and memory a query needs, which is why a query with a huge Count often times out while the same data downloaded from the FTP site works fine. ### Why does biomaRt tell me the query returned an invalid result? This message usually means the web service stopped or returned nothing before your R session got an answer, often because the query was too big or the connection dropped. Retry the query, reduce its size, and make sure your biomaRt version matches the current Ensembl release. ### When should I use BioMart instead of the Ensembl genome browser? Use the genome browser when you want to look at one region or gene in detail. Use BioMart when you want tabular data for many genes at once, such as a list of coordinates, IDs, or orthologs for a gene set. ### How do I get sequences for thousands of genes without a timeout? Download the FASTA and GTF files for your species and release from the Ensembl FTP site and extract the sequences locally with a tool such as gffread or bedtools. This avoids the web service entirely and is the standard approach for large sequence jobs. ### Can I reuse or share a BioMart query so I do not have to rebuild it? Yes. After you build a query, copy the URL from the browser address bar or save the query XML offered on the results page. In biomaRt, record the dataset, filters, and attributes in your script so the query is reproducible. ### Does BioMart behave differently during an Ensembl release update? Yes. Around a release the site can be slow or briefly unavailable while databases are swapped. Check the Ensembl home page or blog for maintenance announcements, and use the archive site if you need data from an older release.