# Ensembl Compara REST API Orthologs: Step-by-Step Guide By the end of this guide you will be able to call the Ensembl Compara REST API, pull every predicted ortholog of a gene with one HTTP request, batch hundreds of genes into a single POST call, request aligned protein sequences for each ortholog pair, and verify that the predictions you downloaded are internally consistent. The commands are copy-paste ready. You need a terminal with `curl`, Python 3 with the `requests` library, and a text editor. No account, no API key, and no database install are required. ## What Ensembl Compara Is and Where Its Ortholog Calls Come From If you are still working out how to use Ensembl as a whole, it helps to know that Ensembl offers three main routes to its data: the genome browser, the BioMart bulk download tool, and the REST API. This guide covers the third route, the Representational State Transfer (REST) application programming interface, which lets a script request [structured data](/blog/data-analysis/what-is-structured-data) over plain HTTP. Ensembl Compara is the Ensembl project that computes cross-species comparisons. For every Ensembl release, Compara builds multiple sequence alignments for homologous protein families, infers gene trees, and records the relationships implied by those trees. A homolog is a gene shared through common ancestry. An ortholog (Ensembl spells it "orthologue" in its data values) is a pair of homologs that separated by a speciation event. A paralog is a homolog that arose by duplication inside a genome. Those distinctions come from standard [evolutionary biology](/blog/guides/evolutionary-biology) and they drive every field you will see in the API response. The quality question matters in practice. A benchmark study compared seven orthology prediction resources, including Ensembl Compara, against curated reference gene trees, and found generally high specificity of orthology assignments for well-conserved protein families [1]. The same study concluded that the differences between databases come mostly from how the databases are built and scoped rather than from the comparison algorithms themselves [1]. Two practical consequences follow. First, Compara ortholog calls are a reasonable default for conserved vertebrate and plant genes. Second, for a divergent or poorly annotated gene, you should cross-check against a second resource rather than treating one database as ground truth. ## How the Homology Endpoints Are Organized The Compara REST endpoints all live under the base URL `https://rest.ensembl.org` and share the prefix `homology`. There are two families of calls you will actually use. | Endpoint pattern | What it returns | |--|--| | `GET /homology/id/:species/:id` | Homology predictions for one gene, identified by Ensembl stable ID | | `GET /homology/symbol/:species/:symbol` | The same, resolved from a gene symbol such as `TP53` | | `POST /homology/id/:species` | Homology predictions for a batch of gene IDs sent in the request body | | `GET /homology/sequence/id/:species/:id` | Homologies plus aligned cDNA or protein sequences | Every GET call needs an `Accept: application/json` header. POST calls additionally need `Content-Type: application/json` because the request body is JSON. Omit the Accept header and the server returns HTML or plain text, which breaks every JSON parser downstream. ## Step-by-Step Walkthrough The worked example uses human BRCA2, stable ID `ENSG00000139618`, because its orthology relationships across mammals are well studied and easy to sanity check. ### Step 1. Confirm the Enstable Gene ID First Symbol-based lookups break when nomenclature changes, so start from a stable ID. If you only have a symbol, resolve it with the cross-reference endpoint. ```bash curl -H "Accept: application/json" \ "https://rest.ensembl.org/xrefs/symbol/human/BRCA2?object_type=gene" ``` Expected output, trimmed to the fields you need: ```json [ { "id": "ENSG00000139618", "type": "gene", "primary_id": "ENSG00000139618", "version": "1", "display_id": "BRCA2", "description": "BRCA2 DNA repair associated" } ] ``` Record the `id` value. That string, not the symbol, is the key the homology endpoints want. ### Step 2. Make Your First Ortholog Call ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/id/human/ENSG00000139618?type=orthologues" ``` The response is a JSON document. Trimmed to one ortholog pair, it looks like this: ```json { "data": [ { "id": "ENSG00000139618", "homologies": [ { "homology_id": "17674901", "type": "orthologue_one2one", "method_link_type": "ENSEMBL_ORTHOLOGUES", "is_high_confidence": 1, "taxonomy_level": "Eutheria", "dn": 0.021, "ds": 0.12, "source_name": "Homo sapiens", "target": { "id": "ENSMUSG00000041147", "gene_stable_id": "ENSMUSG00000041147", "protein_stable_id": "ENSMUSP00000044453", "species": "mus_musculus", "assembly": "GRCm39", "perc_id": 63.0, "perc_pos": 74.0 } } ] } ] } ``` The full response for BRCA2 contains dozens of such objects, one per homologous gene. A single call with `type=orthologues` and no species restriction typically returns orthologs across the vertebrate clade, so expect a response hundreds of kilobytes long for a conserved human gene. ### Step 3. Learn to Read the Response Fields ![Worked calculation dividing dn 0.021 by ds 0.12 to get 0.175 for the human-mouse BRCA2 pair.](/article-images/illustrations/ensembl-compara-rest-api-orthologs-dn-ds-ratio.webp) Each object inside `homologies` describes one relationship between your query gene and one other gene. These fields carry the biology. - `type` gives the relationship class. The common values are `orthologue_one2one`, `orthologue_one2many`, `orthologue_many2many`, `between_species_paralog`, `within_species_paralog`, and `gene_split`. A one2one ortholog means both genes have exactly one counterpart on the other side. A one2many ortholog means the query gene has one counterpart but that counterpart's lineage duplicated, so several related genes exist on one side. `gene_split` marks cases where a single ancestral gene appears split across two loci in one species. - `method_link_type` names the analysis that produced the call. `ENSEMBL_ORTHOLOGUES` and `ENSEMBL_PARALOGUES` are the values you will see most often. - `taxonomy_level` names the most recent common ancestor node in the species tree at which the relationship was inferred, for example `Eutheria` for a human-mouse pair. - `is_high_confidence` is 1 when the orthology call passed Ensembl's internal confidence criteria and 0 otherwise. - `dn` and `ds` are the rates of nonsynonymous and synonymous substitution per site for the pair. They appear only for some ortholog calls. - `perc_id` is the percent identity of the aligned protein sequences. In the sample above, `perc_id` of 63.0 means 63 percent of aligned residues are identical between the human and mouse BRCA2 proteins. A quick worked reading of dN and dS. If a call reports `dn` of 0.021 and `ds` of 0.12, the ratio is 0.021 divided by 0.12, which equals 0.175. A ratio well below 1 is consistent with purifying selection acting on the protein, which is the expected pattern for an ortholog with conserved function. Treat dN and dS as descriptive, not as proof of function. ### Step 4. Filter to Orthologs Only at the Server The `type` parameter controls what the endpoint returns. `type=orthologues` restricts results to orthologs, `type=paralogues` returns paralogy calls, and `type=all` returns both. Requesting only what you need shrinks the response and simplifies parsing. ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/id/human/ENSG00000139618?type=paralogues" ``` For BRCA2 this returns within-species paralogs, human genes related to BRCA2 through duplication in the human or ancestral vertebrate lineage. If you skip the `type` parameter you get a mixed set, which is a frequent source of confusion when a script suddenly contains paralogs. ### Step 5. Narrow to Target Species `target_species` accepts a comma-separated list to keep only the species you care about. ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/id/human/ENSG00000139618?type=orthologues&target_species=mouse,rat" ``` The species values use the Ensembl lowercase naming, `mouse`, `rat`, `chicken`, `zebrafish`. If the API cannot match a species name it returns a 400 status, so check spelling against the Ensembl species list when a call fails unexpectedly. ### Step 6. Narrow by Taxon or by Compara Clade Two related filters exist and they do different jobs. `target_taxon` keeps results from a named branch of the species tree, for example `target_taxon=primates` to keep only primate orthologs. The `compara` parameter selects which Compara database serves the query, which matters when your gene is not from a vertebrate. ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/id/arabidopsis_thaliana/AT1G01010?type=orthologues&compara=plants" ``` The main `compara` values are `vertebrates` (and `mammals` within them) for the core Ensembl species, plus `plants`, `fungi`, and `metazoa` for the Ensembl Genomes clades. For a vertebrate gene the vertebrates compara is the default domain, but passing `compara=vertebrates` explicitly makes your scripts self-documenting and safe if defaults change. ### Step 7. Request Alignments and Sequences When You Need Them Two parameters change how much data comes back. `sequence` takes `none`, `cdna`, or `protein`. `aligned` takes 0 or 1 and only matters when you requested a sequence, asking the server to return the sequences in their aligned, gap-inserted form. ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/id/human/ENSG00000139618?type=orthologues&target_species=mouse&sequence=protein&aligned=1" ``` With `aligned=1`, each homology object carries the aligned [protein sequence](/blog/guides/protein-sequence) and a CIGAR-style line describing the alignment blocks. These responses get large fast, so request alignments only for the gene pairs you actually intend to analyze. For a full alignment-and-sequence pull dedicated to downstream work, the sequence endpoint exists separately. ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/sequence/id/human/ENSG00000139618?type=orthologues&sequence=protein&aligned=1" ``` ### Step 8. Batch Many Genes with POST One HTTP round trip per gene is slow for a gene list of hundreds. The POST form of the endpoint accepts an array of stable IDs. ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ -X POST \ -d '{"ids":["ENSG00000139618","ENSG00000141510"]}' \ "https://rest.ensembl.org/homology/id/human?type=orthologues&format=condensed" ``` The response contains one element in `data` per input ID, each with its own `homologies` array. Note the `format=condensed` flag. The default `format=full` response includes pairwise alignment details and other heavy fields. `condensed` strips those and returns the relationship records only, which is what most ortholog-list use cases want. If your script parses a condensed response expecting alignment fields, it will raise key errors, so pick the format deliberately and document it in your code. Batch limits exist on POST endpoints and they are enforced by the server. Send a list that exceeds the documented maximum for the endpoint and the API rejects the request, so chunk your ID list rather than uploading the whole cohort at once. Chunks of a few hundred IDs with a short pause between requests stay comfortably inside the service guidelines. ### Step 9. Automate with Python and Respect Rate Limits Here is a small, production-shaped client that fetches orthologs for a list of human genes, handles the throttling response, and writes a flat table. ```python import time import csv import requests BASE = "https://rest.ensembl.org" HEADERS = {"Content-Type": "application/json", "Accept": "application/json"} def fetch_orthologs(gene_id, species="human"): url = f"{BASE}/homology/id/{species}/{gene_id}" params = {"type": "orthologues", "format": "condensed"} for attempt in range(5): resp = requests.get(url, params=params, headers=HEADERS, timeout=30) if resp.status_code == 429: wait = float(resp.headers.get("Retry-After", 1)) time.sleep(wait) continue resp.raise_for_status() return resp.json() raise RuntimeError(f"Retries exhausted for {gene_id}") genes = ["ENSG00000139618", "ENSG00000141510", "ENSG00000157764"] rows = [] for gene in genes: payload = fetch_orthologs(gene) entry = payload["data"] for hom in entry["homologies"]: target = hom["target"] rows.append({ "query_gene": entry["id"], "ortholog_gene": target.get("gene_stable_id", target["id"]), "species": target["species"], "homology_type": hom["type"], "taxon_level": hom["taxonomy_level"], "high_conf": hom["is_high_confidence"], "perc_id": target.get("perc_id", ""), }) time.sleep(0.2) with open("orthologs.tsv", "w", newline="") as handle: writer = csv.DictWriter(handle, fieldnames=list(rows.keys()), delimiter="\t") writer.writeheader() writer.writerows(rows) print(f"Wrote {len(rows)} ortholog pairs to orthologs.tsv") ``` The `time.sleep(0.2)` between calls holds you to five requests per second, well inside the published guideline that the public service be used at roughly 15 requests per second or fewer per address. The 429 handling reads the `Retry-After` header and waits as instructed. Response headers named `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `X-RateLimit-Reset` let you monitor how much quota remains. Expected output when you run the script: ``` Wrote 141 ortholog pairs to orthologs.tsv ``` Your count will differ because the ortholog sets track the current Ensembl release. ### Step 10. Validate Before You Trust the Table Validation does not require a second tool. Three cheap checks catch most problems. First, run a reciprocal check. Take one ortholog ID from your output, call the homology endpoint for that gene in its own species, and confirm your original query gene appears in its ortholog list. For the human-mouse BRCA2 example, call the homology endpoint for `ENSMUSG00000041147` in `mouse` and confirm `ENSG00000139618` comes back with `orthologue_one2one`. A pair that fails reciprocity usually reflects a one2many relationship rather than a bug, and the response type field will tell you which. Second, count by type. Sum your `orthologue_one2one`, `one2many`, and `many2many` rows and confirm the total matches the number of objects the API said it returned. A parsing bug that silently drops one2many records shows up here. Third, benchmark a few pairs by hand. Pick three well-known gene pairs from your table, look at their protein percent identities, and confirm the numbers are plausible for the species distance involved. Human-mouse one-to-one orthologs of conserved genes routinely sit at high protein identity, while human-fish pairs sit much lower. The benchmark study behind the reference framework for these databases used curated reference trees as its gold standard, and that same logic, spot-checking predictions against relationships you already trust, is the cheapest validation available to you [1]. ## Key Parameters at a Glance | Parameter | Typical values | What it controls | |--|--|--| | `type` | `orthologues`, `paralogues`, `all` | Which relationship classes are returned | | `format` | `full`, `condensed` | Whether alignment detail and extended fields are included | | `target_species` | Comma-separated Ensembl species names | Keeps orthologs from named species only | | `target_taxon` | Taxon name such as `primates` | Keeps orthologs from one branch of the species tree | | `compara` | `vertebrates`, `mammals`, `plants`, `fungi`, `metazoa` | Which Compara database answers the query | | `sequence` | `none`, `cdna`, `protein` | Whether sequences ride along in the response | | `aligned` | `0`, `1` | Whether sequences come back in aligned, gapped form | | `is_high_confidence` | Filter on response field, value 0 or 1 | Server-side flag you filter client-side, also selectable via parameter | ## Cross-Checking in the Ensembl Genome Browser Before scripting anything, it pays to see the same data visually, and this doubles as a short Ensembl [genome browser tutorial](/blog/guides/genome-browsers-for-researchers-a-guide-to-inspecting-genomic-evidence). Open `www.ensembl.org` in a browser, type `BRCA2` into the search box, and select the human gene. On the gene page, the left menu includes a "Gene tree" link under the Compara section. Clicking it renders the phylogenetic tree for the BRCA2 family, with duplications marked as filled nodes and speciation nodes open. The "Compare locations" view shows the genomic neighborhood of the orthologs in each species side by side. If a relationship you pulled from the API looks odd, find the gene in the tree and check whether it sits below a duplication node, which explains a `one2many` or `many2many` label. The browser and the REST API serve the same underlying Compara release, so disagreement between them signals a parsing mistake on your side, not a data mismatch. ## Rate Limits, Releases, and When to Stop Using the REST API Three operational facts shape real-world projects. The public REST service throttles by address, with a published guideline of roughly 15 requests per second and a cap on concurrent connections. Scripts that ignore this receive HTTP 429 responses. Handle them, add backoff, and keep your request rate deliberately modest. The REST endpoints always reflect the current Ensembl release, and Ensembl updates that release roughly every three months. Ortholog calls, gene trees, and stable ID mappings can all change between releases, so record the release version with every dataset you save. The `X-RateLimit-Reset` header tells you about quota, not release, so pin the release number from the Ensembl homepage at the time of your download into your project metadata. For genuinely bulk workloads, tens of thousands of genes or whole-genome ortholog tables, the REST API is the wrong tool by design. Use the Compara data on the Ensembl FTP site or the Compara MySQL database instead, or drive BioMart for tabular exports. The REST API shines for interactive queries, small-to-medium gene lists, and web applications. A common engineering pattern is REST for exploration and prototyping, then a switch to bulk downloads once the workflow hardens. ## Common Mistakes and Limitations These are the errors that recur across teams adopting the Compara homology endpoints, with fixes. - Sending no Accept header. The server returns a non-JSON representation and `resp.json()` fails with a decode error. Always send `Accept: application/json`. - Mixing up `target_species` and `target_taxon`. The first filters to listed species, the second to a taxonomic branch. Passing a taxon name to `target_species` returns a 400 Bad Request. - Forgetting the `compara` parameter for non-vertebrate species. Querying a plant gene without `compara=plants` fails or returns nothing useful, because the default database is the vertebrate one. - Parsing `format=condensed` output while expecting `full` fields. Alignment data, `dn`, `ds`, and some identifiers are absent in condensed mode. Pick one format per pipeline and code against it. - Assuming one2one orthologs exist everywhere. Duplicated genes produce one2many and many2many records, and collapsing those into a naive one-row-per-gene table silently loses information. Keep the `type` column and decide downstream how to handle multiplicity. - Ignoring `is_high_confidence`. A value of 0 does not mean the call is wrong, it means it did not pass Ensembl's stricter criteria, often because of incomplete annotations or gene-tree uncertainty. For analyses where a wrong ortholog is costly, filter to high-confidence calls and record how many you dropped. - Treating dN and dS as universal. Those fields appear for a subset of ortholog pairs. Code with `.get()` and handle missing values rather than assuming every row has them. - Hammering the API. Bursting past the rate limit triggers 429 responses and, if sustained, longer blocks. Backoff logic plus a fixed inter-request pause is cheaper than debugging a blocked address. - Using species display names with spaces or capitals. The endpoints expect lowercase underscore names such as `arabidopsis_thaliana` and `danio_rerio`. `Danio rerio` fails. - Assuming the API output is absolute truth. No orthology resource is perfect. The cross-database benchmark found that none of the resources it examined provided a fully correct and complete protein classification, and that inter-resource differences track the underlying database concepts as much as the algorithms [1]. For high-stakes gene mappings, check a second resource and reconcile. One scope limitation applies to the endpoints themselves. Compara orthologs are predicted from protein-coding gene trees, and coverage of poorly assembled or poorly annotated genomes is thinner than for reference species. The same benchmark exercise concluded that orthology databases differ in the completeness of predicted relationships, which shows up as missing orthologs for genes in sparse species [1]. A missing call is not evidence of absence of an ortholog. ## Troubleshooting HTTP Errors: A Field Guide When a call fails, the status code plus the response body usually tell you exactly what went wrong, but only if you know how to read them. The table below maps the failures you will actually encounter while working through the Ensembl Compara REST API orthologs step by step, and the fixes that resolve each one. | Status | Typical cause | Fix | |--|--|--| | 400 Bad Request | Malformed species name, invalid parameter value, or a taxon name passed to `target_species` | Check spelling against the Ensembl species list and confirm each parameter value against the parameter table in this guide | | 404 Not Found | The stable ID does not exist in the current release, or you queried the wrong compara database for that organism | Re-resolve the ID with the cross-reference endpoint; if the gene is nonvertebrate, add the correct `compara` value | | 406 Not Acceptable | Missing or wrong `Accept` header | Send `Accept: application/json` on every call | | 429 Too Many Requests | Rate limit exceeded | Sleep for the number of seconds in the `Retry-After` header, then retry | | 500 Internal Server Error | Transient server-side problem, sometimes triggered by an unusual parameter combination | Retry after a short pause; if it persists, simplify the call to isolate the offending parameter | | 503 Service Unavailable | Planned maintenance or heavy load | Check the Ensembl status page, schedule bulk work outside peak hours | Two of these deserve extra attention. The 404 case is the one that most often confuses people, because the same script that worked yesterday can start failing after a release update. Ensembl retires and remaps stable IDs between releases, so an identifier that is absent from the homology endpoint may simply have been merged into a new one. The habit of resolving symbols to fresh stable IDs at the start of every run, described earlier in this guide, protects you from most of these cases. The 400 case usually comes down to one of two typos: capital letters or spaces in a species name, or a clade name such as `primates` supplied where a species name is expected. Both produce the same generic error message, so read your URL string character by character before assuming a server problem. A practical debugging technique is to reduce the failing call to its minimal form. Strip every optional parameter, confirm the bare call returns 200 with JSON, then add parameters back one at a time until the failure reappears. This isolates the culprit in under a minute and beats staring at a long URL trying to spot the problem by eye. Keep a small text file of minimal working calls for each clade you query, and paste new parameters into those known-good templates rather than composing URLs from scratch. ## A Second Worked Example: Moving from Human to a Nonvertebrate Clade The BRCA2 walkthrough in this guide uses the vertebrate compara database, which is the default domain for human queries. The moment you step outside vertebrates, the workflow changes in two ways, and working through a plant example makes those changes concrete. This section complements the earlier arabidopsis one-liner with the full decision sequence. First, you must declare the compara database explicitly. A plant gene queried against the vertebrate domain returns nothing useful or errors outright, because the gene trees that hold your gene live in the plants compara database. Second, species names differ between clades. The plants set includes species such as `arabidopsis_thaliana` that do not exist in the vertebrate set, and conversely. Start from the gene identifier. For arabidopsis, the AGI locus identifier such as `AT1G01010` works directly as the identifier in the homology path, which is convenient because most arabidopsis datasets already use AGI codes. The call pattern is: ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/id/arabidopsis_thaliana/AT1G01010?type=orthologues&compara=plants" ``` A useful intermediate step is to restrict the response to one target species so you can inspect the structure without wading through hundreds of orthologs: ```bash curl -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensmbl.org/homology/id/arabidopsis_thaliana/AT1G01010?type=orthologues&compara=plants&target_species=arabidopsis_lyrata" ``` Note two things about the response you get back. The `taxonomy_level` field now names plant clade nodes rather than mammalian ones, so a value that looked exotic in the vertebrate context, such as a rosid or malvid node label, is normal here. And percent identity values for plant pairs often run lower than the vertebrate numbers you may have calibrated your expectations against, because the divergence times within and between plant families differ from those among mammals. Resist the urge to reuse vertebrate identity thresholds as filters; calibrate per clade by looking at the distribution of `perc_id` values across many pairs before deciding what counts as suspiciously low. If your organism of interest is a fungus, the same pattern applies with `compara=fungi`, and for nonvertebrate animals, `compara=metazoa`. The disciplines that transfer across all clades are these: declare the database, verify the species list for that clade, and recalibrate any numeric filters. Once you have one working call per clade saved as a template, extending a pipeline from vertebrates to plants or fungi is mostly mechanical. ## Parsing Responses with jq Without Writing a Script For quick exploratory work, `jq` turns the JSON response into a tidy table faster than writing a Python script, and it doubles as a way to sanity-check response structure before committing to a parser. Assume you saved a BRCA2 ortholog response to `brca2.json`. Extract one row per ortholog pair with the core fields: ```bash jq -r '.data[] | .id as $q | .homologies[] | [$q, .type, .target.species, .target.id, .target.perc_id] | @tsv' brca2.json ``` The `as $q` binding carries the query gene ID into each homology record, which is essential because the query identifier sits one level above the homologies array and would otherwise be lost when you flatten the structure. Piping through `@tsv` produces tab-separated output you can redirect straight into a file: ```bash jq -r '... same filter ...' brca2.json > brca2_orthologs.tsv ``` Three variations cover most ad hoc needs. To count relationships by type, run: ```bash jq '[.data[].homologies[].type] | group_by(.) | map({(.): length}) | add' brca2.json ``` To keep only high-confidence one-to-one orthologs, filter before flattening: ```bash jq '.data[].homologies[] | select(.type == "orthologue_one2one" and .is_high_confidence == 1)' brca2.json ``` To list the distinct species present in the response, which is a fast way to confirm your `target_species` filter did what you intended: ```bash jq '[.data[].homologies[].target.species] | unique' brca2.json ``` A habit worth building is to run the species-list check after every filtered call. It costs nothing, and it catches the two most common silent failures: a misspelled species that the server dropped rather than rejected, and a filter parameter that landed in the wrong place in the URL. If your downstream script expects mouse orthologs and this check reveals none, you have found the bug before it contaminated any results. The main limitation of jq in this workflow is state. It cannot paginate, throttle, or retry, so treat it as an inspection and one-shot conversion tool, and hand off to Python or R for anything that loops. ## Turning Responses into a Pandas DataFrame and Handling One2Many Multiplicity The Python client in the walkthrough writes a flat table with one row per ortholog pair, which is the correct primitive. Real analyses, though, usually want one row per query gene, and that is where one2many and many2many relationships force a design decision. This section works through the options explicitly, because the choice affects every downstream number you report. Load the pair-level table into pandas and inspect the multiplicity first: ```python import pandas as pd pairs = pd.read_csv("orthologs.tsv", sep="\t") print(pairs["homology_type"].value_counts()) print(pairs.groupby("query_gene").size().describe()) ``` The second line tells you how many orthologs each query gene has. For a gene list dominated by one2one pairs, the distribution clusters near the number of species you queried. Genes that descend from duplications, gene families, and lineage-specific expansions sit in the upper tail. You now have three defensible strategies. Strategy one, restrict to one2one. Filter `pairs` to `homology_type == "orthologue_one2one"` and accept that you lose family members. This is the right choice when you need unambiguous gene-to-gene mapping, for example to transfer gene symbols or build a one-column-per-species matrix. Document the dropped fraction, because a gene panel that loses thirty percent of its members to one2many filtering is telling you something real about its biology. Strategy two, keep all pairs but aggregate with a list column. Group by query gene and collect ortholog identifiers: ```python wide = (pairs .sort_values(["query_gene", "perc_id"], ascending=[True, False]) .groupby("query_gene") .agg(orthologs=("ortholog_gene", list), species=("species", list), n_orthologs=("ortholog_gene", "size"))) ``` Sorting by percent identity before grouping means the highest-identity ortholog sits first in each list, which gives you a principled default representative when you need one. Strategy three, keep all pairs and let the analysis method handle multiplicity. For enrichment analyses and set-based tests, the full pair table is usually the correct input, because collapsing one2many relationships into a single arbitrary representative discards genuine biology. A mistake worth naming explicitly: never de-duplicate by query gene alone with `drop_duplicates()` on the query column. That keeps an arbitrary row, whichever happened to sort first, and silently discards the rest. If you must collapse, collapse deliberately, on a defined key such as the pair with maximum percent identity or the high-confidence call, and record the rule in your code. ## Full Versus Condensed Format: A Field-by-Field Comparison The `format` parameter changes the response in ways that go beyond size, and choosing wrong is one of the most common mistakes in this workflow. The comparison below is organized by what you plan to do next, because that is the only sensible way to choose. With `format=condensed`, each homology record carries the relationship classification, the target gene and protein identifiers, species, and the identity percentages. It omits alignment-related detail, and fields such as `dn` and `ds` that ride along with the full representation may be absent. Response sizes for a well-conserved human gene drop dramatically compared with the full format, which is why the batching example uses condensed mode. If your deliverable is an ortholog list, a cross-species mapping table, or input to a gene-set analysis, condensed is the correct choice, and parsing it is faster and less error-prone precisely because fewer field variants exist. With `format=full`, each record can carry homology-specific extra information, including per-pair alignment metadata where the underlying analysis provides it. This is what you want when you intend to compute your own alignment statistics, filter on dN and dS, or archive a self-contained record of each relationship. The cost is parsing complexity: the code must tolerate fields that appear for some pairs and not others, which in practice means using `.get()` accessors throughout, as the guide's Python client already does for `perc_id`. A third option people overlook is to run the two formats side by side on one gene before writing the production parser. Issue both calls for BRCA2, save both responses, and diff the key sets: ```bash curl -s -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/id/human/ENSG00000139618?type=orthologues&format=condensed" > c.json curl -s -H "Content-Type: application/json" -H "Accept: application/json" \ "https://rest.ensembl.org/homology/id/human/ENSG00000139618?type=orthologues&format=full" > f.json jq '.data.homologies | keys' c.json jq '.data.homologies | keys' f.json ``` The diff between the two key lists is your field inventory, and it tells you exactly which code paths depend on the format choice. Ten minutes of this on one gene prevents an entire class of KeyError bugs later, and the saved responses double as fixtures for the parser tests described in the reproducibility section below. ## Working with Aligned Sequences: Extracting Gaps, Identity, and Coverage When you request `sequence=protein&aligned=1`, each homology record arrives with aligned, gap-inserted sequences and a CIGAR-style description of the alignment blocks. That material supports analyses the pair-level table cannot, but it needs careful handling. This section walks through what to do with it once it lands. The aligned strings carry gap characters that pad one sequence where the other has residues, and the CIGAR-style line records the block structure of the alignment as alternating counts of matched positions and gaps. The critical conceptual point is that position arithmetic differs between the aligned and unaligned representations. An alignment column index and a residue index in the raw protein sequence are not the same number once gaps appear, so if you plan to map an orthologous residue back to a position in the source protein, convert through the alignment blocks rather than counting columns directly. For many purposes you do not need the alignment at all, because the server already computed the summary statistics. `perc_id` and `perc_pos` report identical and similar residues over the aligned region, and those two numbers support a coverage-style sanity check: ```python def alignment_summary(target): pid = target.get("perc_id") ppos = target.get("perc_pos") if pid is None or ppos is None: return None # Similarity above identity implies conservative substitutions exist return {"perc_id": pid, "perc_pos": ppos, "similar_only": ppos - pid} ``` The `similar_only` value, the gap between similarity and identity, is a crude but useful divergence signal. Pairs with high similarity but moderate identity have diverged through conservative substitutions; pairs where the two numbers nearly coincide are either very close orthologs or short, low-information alignments. Two practical cautions apply when you pull sequences through the API. First, responses grow by orders of magnitude once sequences ride along, so the batching strategy changes: fetch relationship records in batches first, then request aligned sequences only for the specific pairs your analysis needs, one call per query gene. Second, the aligned sequences come from the Compara protein alignments behind the gene trees, so they reflect that pipeline's parameter choices rather than a fresh pairwise alignment you would build yourself. That is usually fine, and it is the same alignment the orthology call was based on, which makes the sequences internally consistent with the relationship labels. But if your method demands a custom alignment under specific scoring parameters, download the raw sequences with `aligned=0` and align them in your own toolchain instead. ## Building a Multi-Species Ortholog Matrix from Repeated Calls A common deliverable is a matrix with query genes as rows and species as columns, each cell holding the orthologous gene identifier. The Compara API supports this directly, and the main work is deciding how to fill cells when multiple orthologs exist. Here is a complete, small pipeline. Start from the pair-level table produced by the walkthrough client. Then pivot: ```python def representative(group): # Prefer high-confidence calls, then one2one, then highest identity hc = group[group["high_conf"] == 1] pool = hc if len(hc) else group one2one = pool[pool["homology_type"] == "orthologue_one2one"] pool = one2one if len(one2one) else pool return pool.sort_values("perc_id", ascending=False).iloc["ortholog_gene"] matrix = (pairs .pivot_table(index="query_gene", columns="species", values="ortholog_gene", aggfunc=representative)) ``` The `representative` function encodes a transparent priority: high confidence beats low, one2one beats many, and identity breaks remaining ties. Every matrix built this way should be accompanied by a companion count matrix recording how many candidate orthologs each cell was chosen from, because a cell chosen from ten many2many candidates is a weaker statement than a cell with exactly one candidate: ```python counts = pairs.pivot_table(index="query_gene", columns="species", values="ortholog_gene", aggfunc="size") ``` Missing cells need interpretation, not imputation. A blank cell can mean the species was not queried, the species is absent from the compara database, no ortholog was predicted, or the counterpart gene is unannotated in that assembly. The benchmark study of orthology resources found that databases differ in the completeness of their predicted relationships, and that gaps concentrate in sparsely annotated species [1], so treat a missing call as missing evidence rather than evidence of absence. Recording which of the four explanations applies, at least at the coarse level of species-present versus call-absent, turns an ambiguous matrix into an interpretable one. Finally, decide cell format early. Some pipelines want gene identifiers, some want symbols, and some want both. Fetching symbols later means more API calls, so if your consumers need symbols, add the cross-reference resolution step into the pipeline from the start rather than bolting it on after the matrix is built. ## Applied Pattern: Translating a Gene Panel Across Species A concrete, recurring application pulls all of these pieces together: you have a human gene panel, perhaps a curated set of DNA repair genes or immune genes, and a collaborator works in mouse, rat, or zebrafish. The task is to produce the counterpart gene list with confidence annotations, ready for primer design, knockout planning, or expression analysis. The workflow runs in five passes. Pass one resolves symbols to stable IDs using the cross-reference endpoint, in a batch, and reports any symbol that failed to resolve so nomenclature issues surface immediately rather than mid-pipeline. Pass two fetches condensed ortholog records for the full ID list through the POST endpoint, restricted with `target_species` to the collaborator's species, which keeps each response small enough to inspect. Pass three filters to high-confidence calls and records the drop count per gene. Pass four runs the reciprocal check from the validation section on every mapped pair, which for a panel of tens to hundreds of genes is cheap and catches direction-of-mapping errors. Pass five exports two artifacts: the mapping table itself, and a summary stating the API release, the date, the filter rules, and the number of genes mapped one2one, one2many, or not at all. The summary artifact matters more than beginners expect. When the collaborator reports that gene X mapped to something unexpected, the summary lets you reproduce the call exactly rather than re-deriving it from memory, and because the live endpoints always serve the current release, a reproduction weeks later without recorded metadata may return different results through no fault of anyone. Cross-resource benchmarks of orthology prediction show that even well-conserved families can be classified differently by different databases [1], so the honest framing for a translated panel is a set of predictions with recorded provenance, not a lookup table of certainties. For panels intended for experimental work, one refinement pays for itself: flag pairs whose `taxonomy_level` places the last common ancestor shallower than expected. A human-mouse ortholog whose relationship was inferred at a deeper ancestor node than Eutheria is unusual and worth a manual look in the gene tree view of the genome browser before anyone designs an experiment around it. ## Reproducibility: Caching, Release Pinning, and Testing Your Parser Because the live endpoints always reflect the current release, a pipeline that re-fetches on every run produces results that drift over time. Three inexpensive habits make the workflow reproducible anyway. Cache raw responses. Before parsing, save the response body and headers to disk keyed by the request URL: ```python import hashlib, json, pathlib def cached_get(url, params, headers): key = hashlib.sha256(url.encode()).hexdigest()[:16] path = pathlib.Path("cache") / f"{key}.json" if path.exists(): return json.loads(path.read_text()) resp = requests.get(url, params=params, headers=headers, timeout=30) resp.raise_for_status() path.parent.mkdir(exist_ok=True) path.write_text(resp.text) return resp.json() ``` With a cache in place, parser development, format experimentation, and reruns cost zero API traffic, which also keeps you far from the rate limits. Record the release number from the Ensembl homepage into a metadata file alongside the cache on the day of download, as the operational section of this guide recommends, and copy that number into every derived table. Test the parser against saved fixtures. The failure mode specific to this API is a parser written against one response shape that breaks when a field is absent, either because a different format was used or because a field such as `dn` is optional. Turn the saved responses from your initial exploration into test fixtures and assert the invariants your downstream code depends on: ```python def test_parse_minimal_fixture(): rows = parse_homology_response(open("fixtures/brca2_condensed.json").read()) assert all(r["ortholog_gene"] for r in rows) assert all(r["species"] for r in rows) assert all(r["homology_type"].startswith("orthologue") or r["homology_type"] in ("gene_split",) for r in rows) ``` The last assertion encodes the relationship vocabulary explicitly, so if a future response introduces a value your pipeline has never seen, the test fails loudly instead of the pipeline silently mislabeling rows. Version your outputs. Name derived files with the release and date, for example `orthologs_releaseXX_2026-11-11.tsv`, and never overwrite an older one. Ortholog calls change between releases as gene trees are rebuilt and annotations improve, so two files from different dates are genuinely different datasets, and the benchmark literature on cross-database agreement implies that even within one release, treating predictions as frozen certainties overstates what they are [1]. ## Common Misconceptions About Orthology Predictions Several misunderstandings recur even among experienced bioinformaticians newly working with Compara data, and each one leads to a specific, avoidable error. Misconception one: orthology is a property of genes. Strictly, orthology describes the evolutionary event that separated a pair of genes, and after duplications the classification applies to pairs, not to individual genes. This is why a gene can legitimately appear in several records with different types. A human gene can be in a one2one relationship with its mouse counterpart and simultaneously in one2many relationships with zebrafish duplicates. Scripts that assume each gene has exactly one ortholog per species are imposing a rule the data does not have. Misconception two: one2one means the genes have identical function. Orthology constrains ancestry, not function. Lineage-specific divergence, subfunctionalization after duplication, and gain of novel roles all occur among strict one-to-one orthologs. The dN and dS values provide a hint about selective constraint, and the worked calculation earlier in this guide shows how to read them, but a low ratio indicates constraint on the protein sequence, not proof that a phenotype transfers between species. Misconception three: percent identity is a validity score. `perc_id` describes how similar two aligned proteins are, nothing more. A low identity ortholog can be a correct, high-confidence call for a fast-evolving pair, and a high identity paralog can be the wrong gene to map to. Filtering solely on identity confuses divergence with prediction quality; filter on `is_high_confidence` for quality and read identity as biology. Misconception four: missing orthologs mean the gene has none. As noted above, benchmark comparisons found that resources differ in completeness, particularly for genes in poorly annotated genomes [1]. Absence of a call is a statement about the evidence pipeline, not about evolution. Misconception five: the API and the browser can disagree. They serve the same underlying Compara release, so when your parsed table contradicts the gene tree view in the genome browser, the error is in the extraction, almost always a filter parameter, a format mismatch, or a collapsing rule applied silently. This is why the browser cross-check in this guide is positioned as a debugging tool, not a decorative extra. Misconception six: paralog records are noise to be discarded. Within-species paralogs are the raw material for understanding why one2many relationships exist, and `between_species_paralog` records often mark recent duplications that complicate orthology assignment. Keeping them, clearly labeled, costs one column and preserves options for later analysis. ## Choosing Your Data Channel: REST, BioMart, FTP, or MySQL ![Decision path asking how many genes and how interactive the work is, leading to REST, BioMart, FTP or MySQL.](/article-images/illustrations/ensembl-compara-rest-api-orthologs-choose-channel.webp) Ensembl exposes the same Compara orthology data through several channels, and picking the right one per project stage saves more time than any parsing trick. The decision reduces to three questions: how many genes, how interactive the workflow, and what output shape you need. Use the REST API when the workload is interactive or per-gene: resolving one gene during a discussion, spot-checking a prediction, prototyping a parser, or serving a small web application. The API requires no installation and returns exactly the records you name, at the cost of rate limits and release coupling. Batches of a few hundred identifiers through POST extend the practical range comfortably, as the walkthrough shows. Use BioMart when the deliverable is a tabular export for a defined species pair or a modest set of species, and you want a point-and-click path or a reproducible URL-based query without writing a client. BioMart handles gene-list-sized filters well and returns flat tables that drop straight into spreadsheets and notebooks. Use the FTP site when the workload is genome scale. Compara publishes its homology and gene tree data as bulk files, which load in minutes rather than the hours a request loop would take, and which carry the complete relationship space rather than the subset reachable through per-gene queries. The trade is setup cost: you need storage, a file format reader, and a plan for the much larger schema. Use the Compara MySQL database when you need to join orthology with other Compara tables, filter with real SQL, or run many ad hoc queries against the full dataset interactively. This is the analyst's channel, sitting between the convenience of REST and the bulk of FTP. A pragmatic sequencing for a real project: REST for the first week of exploration, one gene and one answer at a time, while you learn which fields matter; POST batches once the gene list stabilizes; then, if the project grows to whole-genome scope, a one-time FTP or MySQL pull with the REST-era parsing logic repurposed against the bulk format. Teams that skip the exploration phase and jump straight to bulk downloads often discover too late that they needed different fields or different filters, and bulk re-downloads are slower to iterate on than API calls. The reverse error, staying on REST past a few tens of thousands of genes, turns into rate-limit friction and fragile long-running loops. The benchmark study cited throughout this guide compared resources at the level of their databases and scopes [1], and the same principle applies within Ensembl itself: match the channel to the scope. ## Production Tips Checklist A final consolidated checklist distills the practices from this guide into items you can drop into a code review or onboarding document. Before the first call: resolve symbols to stable IDs in a batch and log any failures; confirm the target species names against the Ensembl species list for the clade; decide the compara database explicitly and put it in the URL even when it is the default; choose `condensed` versus `full` deliberately based on the field inventory diff, not habit. During the run: send both `Accept: application/json` and, for POST, `Content-Type: application/json` on every request; keep a fixed inter-request pause and treat 429 responses as routine, reading `Retry-After` rather than guessing; watch `X-RateLimit-Remaining` and slow down before the quota empties rather than after; cache raw responses keyed by URL; chunk gene lists rather than testing the batch ceiling with production data. In parsing: use `.get()` accessors for every optional field, including `dn`, `ds`, `perc_id`, and `perc_pos`; assert the homology type vocabulary explicitly so unknown values fail loudly; keep the type column through every transformation and collapse one2many relationships only by a documented rule; verify species composition of filtered responses with a quick unique-species check. After the run: run the three validation checks from the walkthrough, reciprocity, count-by-type, and manual benchmarking of familiar pairs, before publishing any table; record the release number, download date, and parameter set with every output; retain raw cached responses so any downstream result can be re-derived exactly; and for high-stakes mappings, reconcile against a second orthology resource, since comparative evaluations show that resource-level differences, not algorithmic differences, drive most disagreement between databases [1]. None of these items is expensive individually. Together they convert the API from a convenient endpoint into a dependable component of a reproducible pipeline, which is the difference between a table you trust for a figure and a table you trust for a decision. ## Frequently Asked Questions ### Is the Ensembl Compara REST API free to use? Yes. The public REST endpoints at rest.ensembl.org are free, require no registration or key, and are subject to rate limits that keep the service usable for everyone. ### What is the difference between one2one, one2many, and many2many orthologs? These labels describe how many counterpart genes exist on each side of the speciation event. One2one means a single gene on each side, one2many means duplication on one side produced extra related genes, and many2many means duplications occurred on both sides. ### Can I get paralog predictions from the same endpoints? Yes. Pass `type=paralogues` for paralogy calls or `type=all` for both, using the same `homology` endpoints and species path. ### How do I know which release my ortholog data came from? The REST service always serves the current Ensembl release. Record the release number from the Ensembl homepage on the day you run your download, because there is no release pinning parameter on the live endpoints. ### What does the is_high_confidence flag mean? It marks orthology calls that passed Ensembl's stricter internal criteria. A value of 1 is high confidence, a value of 0 means the call exists but did not meet those criteria, often due to annotation gaps or tree uncertainty. ### What is the fastest way to get orthologs for thousands of genes? Skip the REST API and use bulk channels. The Compara tables on the Ensembl FTP site, the Compara MySQL database, or BioMart exports are designed for genome-scale pulls, while POST batching covers intermediate list sizes. ### Why is my query gene missing orthologs for a species I expected? The species may be absent from the compara database you queried, or the counterpart gene may be missing or poorly annotated in that assembly. Try a broader `compara` clade and check the species list for the release you are using. ### How do Compara orthologs differ from other orthology databases? Every resource builds its predictions from different pipelines, taxonomic scopes, and update cycles. For well-conserved protein families they tend to agree, but for divergent genes the sets can differ noticeably, so cross-checking a second database is standard practice for critical analyses. ## Sources 1. [Conceptual framework and pilot study to benchmark phylogenomic databases based on reference gene trees.](https://pubmed.ncbi.nlm.nih.gov/21737420/)