# Audit Logging for Genomic Data Access: What to Record By the end of this article you will be able to design an audit logging system for genomic data access that captures every event a reviewer, data protection officer, or funder will ask about, deploy a working pipeline on a Linux server with PostgreSQL and Linux audit tooling, make the log tamper-evident with hash chaining or a permissioned blockchain, and run the queries that turn raw records into answers. You will need a Linux host, PostgreSQL (or another relational database), the Linux audit daemon (auditd) if you log at the filesystem layer, Python 3 for the examples, and optionally a MultiChain node if you plan to anchor records to a distributed ledger. Genomic data sits in an unusual position among research assets. A variant file is simultaneously a scientific dataset, a biometric identifier, and a permanent statement about a person and their relatives. Gürsoy and colleagues put it plainly in their report on blockchain logging for genome datasets: genomic variants are sensitive information that can reveal private facts about individuals, so controlling access matters, and a key part of controlled access is the secure storage and efficient query of access logs [1]. This article is about that second half, the part most teams underbuild. ## Why Genomic Data Access Needs Its Own Audit Trail Most bioinformatics teams already have some logging. Web servers log requests, batch schedulers log jobs, databases log transactions. None of those logs, by themselves, answer the question a data access committee actually asks: who touched which participant's genomic data, when, under what authorization, and what did they do with it. The problem with relying on general-purpose system logs is structural. Ma and colleagues, writing about cross-site genomic data sharing, note that a centralized access log audit system is vulnerable to a single point of attack and lacks transparency, because an administrator or an internal adversary can alter the log after the fact [2]. A log that the person under investigation can edit is not an audit trail. It is a diary. This is the core design problem in audit logging for genomic data access: the record must be complete enough to reconstruct every access, and durable enough to survive the person who would prefer it did not exist. There are three practical drivers for getting this right. **Accountability across sites.** Genomic data now moves between institutions under data use agreements rather than sitting in one archive. The 2018 iDASH Secure [Genome Analysis](/blog/guides/genome-analysis) Competition devoted an entire track to cross-site access logging precisely because multi-site sharing breaks the assumption that one trusted party holds the log [3]. When four sites each hold copies of a dataset, each site must record its own access events, and those events must be queryable as a coherent whole. **Regulatory and contractual obligations.** Under the GDPR, European institutions have shifted from centralized genome-phenome archives toward federated models in which distributed nodes handle access requests, conclude agreements with data users, and provide secure access to the data [4]. In a federation, the local access log is often the only authoritative record of what happened to a dataset on your node. A system such as DS-PACK, which supports controlled access workflows at the ELIXIR Luxembourg data hosting platform, centralizes access control management and distributes enforcement, and the audit record at each node is what makes the distributed model defensible [4]. Federated analytics platforms carry the same requirement. The FLAME architecture for privacy-preserving federated learning in healthcare, developed in the German PrivateAIM project and deployed at German university hospitals, includes fine-grained access control and audit logging as core platform features rather than add-ons [5]. **Participant expectations.** Patients treat genomic data as theirs. In a clinical pilot of the PROMISE system, which lets cardiomyopathy patients grant selective access to their own genetic information, 79% of participants rated the safety of their genetic data as very important, and patients generally regarded the data as highly sensitive [6]. Participant-facing sharing platforms raise the same expectation from the other direction. Reviews of personal genomic data sharing initiatives observe that individual control over data is a key theme across platforms where people upload and share genotypes themselves [7]. If a participant asks what happened to their sample's data, your audit log is the answer. Audit logging also belongs in the wider context of bioinformatics [data management](/blog/guides/data-management-basics-principles-processes-and-best-practices). A well-run central data management function treats capturing, integrating, storing, and transferring data as one system, with role-based access control, de-identification, and backup built in, as the operators of a large population health research platform described after managing roughly 5 terabytes of participant data over eighteen months without a critical incident [8]. The audit log is the [connective tissue](/knowledge/veterinary-medicine/veterinary-anatomy-physiology/connective-tissue-types-loose-dense-and-more) that lets you prove the access rules in that system were followed. It works alongside, not instead of, other controls such as encryption, and for pathogen sequences with dual-use potential, complementary techniques such as pooling raw reads across samples before sharing can reduce the value of the data to an attacker in the first place [9]. ## What to Record: The Anatomy of an Audit Event An audit event is a single, self-contained answer to five questions: who, what, when, from where, and with what result. Record all five in every record. A record missing any one of them is folklore, not evidence. The table below lists the fields that belong in a genomic access audit record, why each matters, and a realistic example value. | Field | Why it matters | Example value | |--|--|--| | event_id | Unique identifier for correlation across systems | `a3f9c2e1-77b4-4d0e-9a1f-2c8b5e6d7f01` | | timestamp | Ordering events, bounding breach windows | `2026-11-11T14:03:22.481Z` (UTC, always UTC) | | actor_id | Accountability, the who | `jdoe@institution.edu` or service account `pipeline-fastqc` | | authentication_method | Shows how strongly the identity was verified | `sso-saml`, `api-key`, `certificate` | | action | The verb, read, copy, download, delete, query, list | `download` | | dataset_id | Scope of the authorization | `phs000xxx.v3.p1` or internal `cohort-als-2024` | | object_path | File or record level granularity | `/restricted/alignments/sample_042.bam` | | request_id | Links the event to a specific user request or ticket | `req-2026-118842` | | source_host, source_ip | Provenance, detects credential sharing | `hpc-login-03, 10.4.2.17` | | authorization_ref | The consent or data use agreement that permitted this | `dua-2024-017` | | outcome | allow or deny, both are audit events | `allow` | | bytes_transferred | Exfiltration detection, volume review | `4718592` | | tool_and_version | Reproducibility of the access context | `icloud-aligner 2.1.3` | Several of these fields deserve comment because teams routinely get them wrong. **Record the denial as faithfully as the grant.** A denied access attempt is often the most informative event in the log. Someone tried to read a dataset their agreement does not cover. That is either a misconfiguration worth fixing or an insider risk worth a conversation. Log both outcomes with identical schema. **Record the authorization reference, not just the user.** Tying each access to the data use agreement or consent provision that permitted it is what turns a list of events into a defensible account. This is the step federated platforms take seriously. DS-PACK handles controlled access processes end to end, from data submission through access, with data protection principles built into each component, and a central access control management layer paired with distributed enforcement [4]. Your log should be able to answer: under which agreement did this user receive this file. **Record identity at the level of the human or the named service account, never the shared credential.** If twenty analysts share a `genomics-ro` account, your log records one actor and you have lost accountability entirely. In an academic medical center environment, the Weill Cornell Medicine Data Core structured user access as one of its explicit design dimensions alongside data controls, hardware, software, and audit, supporting more than 200 faculty, staff, and students across nearly 60 projects in its first two and a half years [10]. That level of activity is only auditable when each person and each service has a distinct identity. **Record enough to reconstruct, not enough to leak.** The log records metadata about access. It should never contain sequence content, variant calls, or clinical values. Storing a checksum of the object accessed is a good middle ground: it proves which bytes were touched without copying sensitive content into a second location. Your audit log inherits the sensitivity of whatever you put in it. ## Step-by-Step: Building an Audit Logging Pipeline for Genomic Data Access ![Flow of the audit pipeline steps, from defining the event schema through review and rehearsal.](/article-images/illustrations/audit-logging-genomic-data-access-pipeline-six-steps.webp) The walkthrough below builds a working pipeline in six steps on a Linux host. Each step includes commands, the expected output, and how to verify the step worked. ### Step 1. Define the event schema before you write any code Write the schema down as JSON and treat it as a contract. Every producer of audit events, your web portal, your SFTP gateway, your HPC pipeline jobs, emits the same shape. ```json { "event_id": "a3f9c2e1-77b4-4d0e-9a1f-2c8b5e6d7f01", "timestamp": "2026-11-11T14:03:22.481Z", "actor_id": "jdoe@institution.edu", "auth_method": "sso-saml", "action": "download", "dataset_id": "cohort-als-2024", "object_path": "/restricted/alignments/sample_042.bam", "object_sha256": "6b1f2c9e4a0d5f8c3b7e2a1d9c4f8b3a6e0d2c5b8a7f4e1d3c6b9a2f5e8c4b7d", "request_id": "req-2026-118842", "source_host": "hpc-login-03", "source_ip": "10.4.2.17", "authorization_ref": "dua-2024-017", "outcome": "allow", "bytes_transferred": 4718592, "tool_and_version": "icloud-portal 3.2.0" } ``` Verify by writing a validator before writing any producer. A minimal check in Python: ```python import json, uuid, datetime REQUIRED = ["event_id", "timestamp", "actor_id", "action", "dataset_id", "object_path", "outcome"] def validate(record): missing = [k for k in REQUIRED if k not in record] assert not missing, f"missing fields: {missing}" uuid.UUID(record["event_id"]) datetime.datetime.fromisoformat(record["timestamp"].replace("Z", "+00:00")) assert record["outcome"] in ("allow", "deny") record = json.load(open("example_event.json")) validate(record) print("schema ok") ``` Expected output: ``` schema ok ``` A schema validated at ingestion prevents the most expensive failure mode in audit logging, discovering during an investigation that half your records lack timestamps or actor identities. ### Step 2. Instrument every access path Data leaves your system through more doors than teams expect: the web portal, direct database queries, shared filesystems, SFTP, and interactive compute nodes. Each needs instrumentation. For the database layer, use an audit extension rather than general statement logging. In PostgreSQL, pgAudit produces standardized audit entries: ```ini # postgresql.conf shared_preload_libraries = 'pgaudit' pgaudit.log = 'READ, WRITE' pgaudit.log_catalog = 'off' pgaudit.log_relation = 'on' ``` Then, after a restart: ```sql CREATE EXTENSION pgaudit; ``` A subsequent query against a restricted variant table produces an audit line identifying the session, user, statement, and relations touched. For the filesystem layer, where most raw FASTQ and BAM files actually live in genomics data storage, use the Linux audit daemon. Add a watch rule for read access on the restricted tree: ```bash auditctl -w /srv/genomics/restricted -p r -k genomic-read ``` To review what was captured: ```bash ausearch -k genomic-read -i --format text | tail -n 3 ``` Expected output looks like: ``` type=PROCTITLE msg=audit(11/11/2026 14:03:22.481) : proctitle=samtools type=PATH msg=audit(11/11/2026 14:03:22.481) : item=0 name=/srv/genomics/restricted/alignments/sample_042.bam type=SYSCALL msg=audit(11/11/2026 14:03:22.481) : arch=x86_64 syscall=openat success=yes uid=10042 auid=jdoe ``` Each syscall record gives you the actor (uid and login id), the object, the operation, and the timestamp. Map these into your schema in the shipper. The audit subsystem records what the operating system saw, which is the ground truth when a user bypasses your application and reads a file directly. A practical tip from the iDASH competition designs: batch and index by design rather than as an afterthought. The winning teams instrumented logging at the application boundary where access decisions are made, because that is the only place you reliably know the authorization reference, the request context, and the intended purpose [3]. Filesystem telemetry is your backstop, not your primary source. ### Step 3. Ship records to an append-only store Producers should emit events and forget them. A shipper on each host validates, enriches, and forwards events to a central collector over an encrypted channel. With rsyslog, a forwarding action looks like: ``` action(type="omfwd" target="audit-collector.internal" port="6514" protocol="tcp" StreamDriver="gtls") ``` The collector writes to append-only storage. Verify integrity in transit by comparing record counts at both ends over a fixed window: ```bash # on a producer host, last hour journalctl -u audit-shipper --since "1 hour ago" | grep -c "event forwarded" # on the collector, last hour grep -c '"event_id"' /var/log/genomics-audit/current ``` Both counts must match. If they do not, you have a silent producer failure or a lossy transport, and either one is a gap in your audit trail that you will not discover until you need the log. The central data management literature is clear on why this tier matters. The operators of a large epidemiology research platform attributed their clean incident record to intelligent capture and storage mechanisms, a multi-layered role and rights system for access control, and a well-defined backup process [8]. Your audit store deserves the same treatment as your most protected dataset, restricted write access for the logging service, no delete permission for anyone, and backups. ### Step 4. Make the log tamper evident This is where genomic data access logging diverges from ordinary IT logging. As Ma and colleagues argue, a centralized log that a privileged administrator can edit fails the transparency test, which is why their group and others turned to cryptographic and distributed approaches during the 2018 iDASH competition [2]. The minimal tamper-evident structure you can build yourself is a hash chain. Each record's fingerprint incorporates the fingerprint of the record before it, so altering any historical record breaks every fingerprint that follows it. ```python import hashlib, json def event_body(rec): return json.dumps(rec, sort_keys=True, separators=(",", ":")) def chain(records, prev_hash="0" * 64): for rec in records: body = event_body(rec) digest = hashlib.sha256((prev_hash + body).encode()).hexdigest() rec["prev_hash"] = prev_hash rec["record_hash"] = digest prev_hash = digest return prev_hash ``` Anchoring the running head hash periodically to storage you do not control (or to a ledger you share with the other sites) gives you independent proof that the chain has not been rewritten. Permissioned blockchains take this further and were the subject of the iDASH 2018 blockchain track, which asked participants to build time- and space-efficient ledgering of cross-site genomic access logs on MultiChain [3]. The winning approach used MultiChain data streams, treated as a key-value store embedded in the chain, with a data frame built over the stream keys to support efficient querying [1]. A second team used the same stream API with a two-level index, achieving query complexity independent of the inserted dataset size except for timestamp ranges, which scale logarithmically [11]. A third team applied bucketization, data duplication, and batch loading to make conjunctive equality and range queries efficient over a permissioned network of four nodes representing four sites [3]. Working with a MultiChain stream from the command line looks like this: ```bash # create a stream open for writes by designated nodes multichain-cli genochain create stream audit-stream true # publish an audit event, key = dataset, item = event JSON multichain-cli genochain publish audit-stream cohort-als-2024 \ '{"json": {"event_id":"a3f9c2e1","actor_id":"jdoe@institution.edu", "action":"download","outcome":"allow"}}' # read items back multichain-cli genochain liststreamitems audit-stream false 10 ``` Expected output for the publish command: ``` {"txid": "6f4b8a2e91c3d0e7f5a2b8c1d4e6f3a9b7c2d5e8f1a4b7c0d3e6f9a2c5b8e1d4"} ``` A candid limitation worth knowing before you commit: the winning iDASH solution, while effective in the challenge, required downloading the full chain to nodes, which its authors acknowledged as a drawback for operational deployment [1]. A blockchain is one point in a design space, not a requirement. Decide based on who needs to verify the log and whether they trust your administrators. ### Step 5. Index and query the log A log you cannot query is a log you will never read. Two query patterns matter most for genomic data access audits, and both were explicitly engineered for in the iDASH designs: conjunctive equality queries (user equals X and dataset equals Y) and timestamp range queries [11][3]. On a relational store: ```sql -- every access by one user to one dataset in a time window SELECT timestamp, action, object_path, outcome, bytes_transferred FROM audit_events WHERE actor_id = 'jdoe@institution.edu' AND dataset_id = 'cohort-als-2024' AND timestamp BETWEEN '2026-11-01' AND '2026-11-11' ORDER BY timestamp; -- volume anomaly check, bytes moved per user this week SELECT actor_id, SUM(bytes_transferred) AS total_bytes FROM audit_events WHERE timestamp > now() - interval '7 days' AND action = 'download' GROUP BY actor_id ORDER BY total_bytes DESC; ``` The second query is the single highest-value routine audit in genomic settings. A service account that suddenly downloads hundreds of BAM files is either a misconfigured pipeline or an incident. The iDASH teams solved the indexing problem on append-only ledgers, which do not natively support efficient queries, with techniques you can borrow for any log store: bucketing events by time period, duplicating selected fields into key positions, and building secondary indexes outside the immutable store [2][11][3]. In MultiChain streams this meant encoding query dimensions into the stream keys so that list-stream queries by key become your primary access path [1]. ### Step 6. Review, alert, and rehearse Logging without review produces a false sense of security. Put three processes on a calendar. First, a routine review. Weekly, run the volume anomaly query and a denied-access summary: ```sql SELECT actor_id, dataset_id, COUNT(*) AS denials FROM audit_events WHERE outcome = 'deny' AND timestamp > now() - interval '7 days' GROUP BY actor_id, dataset_id ORDER BY denials DESC; ``` Second, alerting. Alert on denied access bursts, access from unexpected source hosts, off-hours downloads above a threshold you set from your own baseline, and any gap in expected event volume from a producer. Third, rehearsal. Twice a year, pick a sample of events and walk them end to end: from the log record to the request ticket to the data use agreement to the participant cohort. If you cannot close that loop for a randomly chosen event, the log has a schema or provenance gap, and you want to find that out during a rehearsal rather than during an audit. The Weill Cornell Data Core's approach of making audit an explicit design dimension of the environment, documented and shared as tooling, is a useful model here [10]. ## Choosing a Storage Backend ![Four audit storage options compared on tamper evidence, query capability and operational notes.](/article-images/illustrations/audit-logging-genomic-data-access-backend-options.webp) The table below summarizes the main options for the audit store itself, the component that receives and holds your records. | Approach | Tamper evidence | Query capability | Operational notes | |--|--|--|--| | Relational database with audit extension | Low, a privileged user can modify rows | Fast, flexible SQL | Simplest to deploy, pair with restricted roles and backups [8] | | Write-once object storage or WORM filesystem | Moderate, retention policies resist deletion | Moderate, index separately | Good for long retention, verify policy enforcement | | Local hash-chained ledger | Strong, any alteration breaks the chain | Needs a side index | You control it, but a single custodian can still withhold it [2] | | Permissioned blockchain (MultiChain streams) | Strong, replicated across nodes | Requires key design or side index | Cross-site trust without a central authority, plan for chain size and node downloads [11][1][3] | The iDASH designs converged on a consistent pattern worth adopting regardless of backend: keep the immutable record minimal, and maintain the queryable index separately. The team that won the competition stored records in stream keys and built a data frame over the chain for queries [1]. Others built hierarchical timestamp structures to index an immutable ledger [2], or offloaded queryable key-value state alongside the chain [3]. Separation of the evidence layer from the index layer is the durable lesson. ## Common Mistakes and Limitations These are the failures seen repeatedly when teams first build audit logging for genomic data access, with the fix for each. **Instrumenting only the portal.** Teams log the web application and miss direct SSH reads, SFTP transfers, and database clients. The result is a log that shows the polite traffic and none of the bulk exfiltration. Fix it by instrumenting at both the application boundary and the filesystem, as in Step 2, and by reconciling the two streams weekly. **Logging grants but not denials.** Denied attempts are your earliest warning of misuse and misconfiguration. Log them with the full schema, including the attempted object and the rule that caused the denial. **Writing sensitive content into the log.** A log that includes a variant line or a patient identifier has just created a second sensitive dataset that now needs its own protection. Record identifiers of objects and their checksums, never their contents. **Shared credentials.** If analysts and pipelines share accounts, every record maps to a group. Assign each human a personal identity and each automated process a named service account with its own credential, and record the authentication method per event. **Mutable timestamps and unsynchronized clocks.** Records with local-time timestamps from unsynchronized hosts cannot be ordered reliably across sites, which defeats cross-site correlation, the exact problem the multi-site iDASH designs were built to solve [2][3]. Require UTC with millisecond precision and run NTP on every host that emits events. **Treating the log as disposable storage.** Rotating audit logs on a short cycle deletes the history you will need for an investigation that starts after the rotation. Define a retention period deliberately, document it, and back up the store. The central data management experience in large-scale medical research showed that a well-defined backup process is part of the security posture, not an optional extra [8]. **Assuming a blockchain solves governance.** Immutability prevents rewriting history. It does not prevent a bad actor with legitimate credentials from making an unauthorized access in the first place, and it does not decide who is allowed to read the log. As the iDASH reports note, ledger-based systems also carry real operational costs, such as full-chain downloads at nodes [1]. Evaluate against your actual trust model. **Underestimating the access agreement layer.** A log entry that cannot be tied to the consent or data use agreement that authorized the access leaves your review unfinished. Institutions joining data-sharing federations often lack the resources to stand up controlled access processes, which is precisely the gap reusable middleware such as DS-PACK addresses by semi-automating the path from submission through access [4]. If your organization has not formalized that path, the audit log will record events whose authorization is ambiguous, and ambiguity is hard to defend. **A final limitation.** Audit logging is evidence, not prevention. It records what happened so you can detect misuse and account for it. For datasets with dual-use potential, log-adjacent measures, such as the read-pooling approach demonstrated for pathogen sequence data, which prevents full reconstruction of any individual sample while preserving most genomic information, reduce the harm an unauthorized access could cause [9]. Layer the controls rather than choosing among them. ## Frequently Asked Questions ### What fields must every genomic access audit record contain? At minimum: a unique event identifier, a UTC timestamp, the actor identity, the action performed, the dataset and object accessed, the source host, the authorization reference, the outcome, and the bytes transferred if data moved. Records for denied attempts use the same schema. ### Do I need a blockchain for audit logging? No. You need tamper evidence and queryability, and a hash chain plus a separate index can deliver both within one institution. Permissioned blockchains such as MultiChain earn their cost when multiple sites must verify each other's logs without a mutually trusted central administrator, which was the design problem in the 2018 iDASH competition. ### How long should I keep genomic access audit logs? There is no single retention period that fits every agreement and funder. Set retention to cover your longest applicable data use agreement, your institutional policy, and the practical needs of investigations, document the period, and keep the store backed up for its full lifetime. ### Should failed access attempts be logged? Yes, always, with the same fields as successful events. Denied attempts reveal misconfigurations and probe patterns, and their absence from a log is itself a finding during a review. ### How do I link an access event to a consent or data use agreement? Include an authorization reference field in every event and populate it at the access decision point, where the system evaluates permissions. Middleware designed for controlled access federations, such as DS-PACK, centralizes access control decisions so that the reference is available at enforcement time and can flow into the log. ### How can I tell if my audit logs were tampered with? If you use hash chaining, recompute the chain from the earliest record and compare each stored fingerprint, a mismatch localizes the alteration. If you anchor heads to a permissioned blockchain or off-site storage, compare your current chain head against the anchored value, and any divergence indicates rewriting. ### What is the difference between access logs and application logs? Application logs record what software did for debugging, with whatever detail helps engineers. Access logs record who touched which protected dataset under which authorization, with a fixed schema and tamper protection. They answer different questions and should be separate systems. ### Who should be able to read the audit logs? A small, defined group, typically security and compliance staff plus data stewards, with read access logged in turn so the log itself has an audit trail. The logging service needs write-only access at the producer side, and no human should have delete rights on the store. ## Sources 1. [Using blockchain to log genome dataset access: efficient storage and query.](https://pubmed.ncbi.nlm.nih.gov/32693796/) 2. [Efficient logging and querying for blockchain-based cross-site genomic dataset access audit.](https://pubmed.ncbi.nlm.nih.gov/32693835/) 3. [Leveraging blockchain for immutable logging and querying across multiple sites.](https://pubmed.ncbi.nlm.nih.gov/32693807/) 4. [DS-PACK: Tool assembly for the end-to-end support of controlled access human data sharing.](https://pubmed.ncbi.nlm.nih.gov/38750048/) 5. [Enabling Privacy-Preserving Federated Learning in Healthcare: The FLAME Architecture and Policy Framework.](https://pubmed.ncbi.nlm.nih.gov/42175157/) 6. [Controlling my genome with my smartphone: first clinical experiences of the PROMISE system.](https://pubmed.ncbi.nlm.nih.gov/34694434/) 7. [Challenges of web-based personal genomic data sharing.](https://pubmed.ncbi.nlm.nih.gov/26085313/) 8. [Efficient data management in a large-scale epidemiology research project.](https://pubmed.ncbi.nlm.nih.gov/21256617/) 9. [Methods for safely sharing dual-use genetic data.](https://pubmed.ncbi.nlm.nih.gov/41756120/) 10. [Design and Implementation of a Secure Computing Environment for Analysis of Sensitive Data at an Academic Medical Center.](https://pubmed.ncbi.nlm.nih.gov/30815128/) 11. [Decentralized genomics audit logging via permissioned blockchain ledgering.](https://pubmed.ncbi.nlm.nih.gov/32693795/)