{"id":44540889,"url":"https://github.com/ebi-gene-expression-group/atlas-schemas","last_synced_at":"2026-02-13T18:58:54.619Z","repository":{"id":38309032,"uuid":"149509311","full_name":"ebi-gene-expression-group/atlas-schemas","owner":"ebi-gene-expression-group","description":"Schemas for the database and SolrCloud servers used by Expression Atlas and Single Cell Expression Atlas","archived":false,"fork":false,"pushed_at":"2025-09-02T20:11:03.000Z","size":70,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-09-02T22:08:15.144Z","etag":null,"topics":["schemas","solr","solrcloud","sql"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ebi-gene-expression-group.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2018-09-19T20:37:18.000Z","updated_at":"2025-07-03T15:57:12.000Z","dependencies_parsed_at":"2025-09-02T22:06:29.530Z","dependency_job_id":"6528a59f-f4a3-4f96-a03b-25ea9c639137","html_url":"https://github.com/ebi-gene-expression-group/atlas-schemas","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ebi-gene-expression-group/atlas-schemas","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebi-gene-expression-group%2Fatlas-schemas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebi-gene-expression-group%2Fatlas-schemas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebi-gene-expression-group%2Fatlas-schemas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebi-gene-expression-group%2Fatlas-schemas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ebi-gene-expression-group","download_url":"https://codeload.github.com/ebi-gene-expression-group/atlas-schemas/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebi-gene-expression-group%2Fatlas-schemas/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29414315,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-13T06:24:03.484Z","status":"ssl_error","status_checked_at":"2026-02-13T06:23:12.830Z","response_time":78,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["schemas","solr","solrcloud","sql"],"created_at":"2026-02-13T18:58:54.452Z","updated_at":"2026-02-13T18:58:54.610Z","avatar_url":"https://github.com/ebi-gene-expression-group.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Expression Atlas schemas\n\nSchemas for the database and SolrCloud servers used by [Expression Atlas](https://www.ebi.ac.uk/gxa) and [Single Cell\nExpression Atlas](https://www.ebi.ac.uk/gxa/sc).\n\nThis is a central repository used by the web application to bootstrap the embedded database and Solr server in the\ntesting environment, and to initialise a fresh (PostgreSQL) database and SolrCloud cluster to be used by either flavour\nof Expression Atlas.\n\nIn the `db` directory there is a `shared-schema.sql` file that is needed by both Expression Atlas and Single Cell\nExpression Atlas, used to display array designs and design elements of a gene in the [bioentity information\ncard](https://github.com/ebi-gene-expression-group/atlas-bioentity-information).\n\n## Migrations\n\nWe use [Flyway](https://flywaydb.org/) to version-control both single cell and bulk Expression Atlas databases. To\napply new migrations to a database, either from `gxa` or `scxa`, applied to bulk Expression Atlas and Single Cell\nExpression Atlas, respectively, run the following command (fill in the variables as needed):\n```bash\nflyway migrate -url=jdbc:postgresql://${HOST}:5432/${DB} -user=${USER} -password=${PASSWORD} -locations=filesystem:`pwd`\n```\n\nThe [`info`](https://flywaydb.org/documentation/command/info) and\n[`repair`](https://flywaydb.org/documentation/command/repair) commands are helpful to troubleshoot issues (e.g.\nchecksum mismatches).\n\n### How to use it without having to deal with dependencies\n\nYou can use flyway as installed in the `quay.io/ebigxa/atlas-schemas-base` container\n(this is the same container used to run the tests). You will need:\n\n- A checkout of this repo\n- Mount the location of the flyway/[gxa|scxa] directory inside the container\n- Specify the database connection\n\nThis is achieved like this (make sure to replace db_host and db_name):\n\n```\n# go to the root directory of this repo\nexport APP=gxa\nexport DB_CONNECTION_URL=jdbc:postgresql://db_host:5432/db_name\nexport USER=\u003cdb_user\u003e\nexport PASSWORD=\u003cdb_password\u003e\n# Make sure you fill in the above ^^^^^^\nexport LOCATION_IN_CONTAINER=/flyway/$APP\ndocker run -it -v $( pwd )/flyway/$APP:$LOCATION_IN_CONTAINER \\\n  quay.io/ebigxa/atlas-schemas-base \\\n  flyway migrate -url=$DB_CONNECTION_URL -user=${USER} -password=${PASSWORD} -locations=filesystem:$LOCATION_IN_CONTAINER\n```\n\nNote that changes in this repo don't require changes in that container unless that\nwe need a new flyway version.\n\n## Common queries\n\n### Select all experiments for a gene\n\n```\nSELECT\n    experiment_accession\nFROM\n    scxa_cell_group_marker_genes m\n    JOIN scxa_cell_group g ON m.cell_group_id=g.id\n    JOIN experiment e ON g.experiment_accession = e.accession\nWHERE\n    e.private=FALSE AND\n    m.gene_id=':gene_id'\nGROUP BY experiment_accession\n```\n\n### Select clusters for significant genes\n\n```\nSELECT\n    variable as k,\n    value as cluster_id\nFROM\n    scxa_cell_group g\n    JON scxa_cell_group_marker_genes m ON g.id = m.cell_group_id\nWHERE(\n    m.gene_id=':gene_id'\n    AND marker_probability \u003c :threshold AND\n    g.experiment_accession=':experiment_accession' AND (\n      g.variable=':preferred_K' OR\n      m.marker_probability IN (\n        SELECT\n          MIN(marker_probability)\n        FROM\n          scxa_cell_group,\n          scxa_cell_group_marker_genes\n        WHERE\n          experiment_accession = ':experiment_accession' AND\n          gene_id=':gene_id'\n        )\n    )\n)\n```\n\n### Select coordinates with expression of a given gene\n\n```\nSELECT\n    c.cell_id,\n    c.x,\n    c.y,\n    a.expression_level\nFROM\n    scxa_coords c  \n    LEFT JOIN scxa_analytics a ON\n        c.experiment_accession=a.experiment_accession AND\n        c.cell_id=a.cell_id AND\n        a.gene_id=':gene_id'\nWHERE\n    c.experiment_accession=':experiment_accession' AND\n    c.method=':method' AND\n    c.parameterisation-\u003e0-\u003e\u003e'perplexity'=':perplexity'\n```\n\n### Select coordinates with a cell group membership\n\n```\n  SELECT\n    c.cell_id,\n    c.x,\n    c.y,\n    g.value as cluster_id\nFROM\n    scxa_cell_group g\n    JOIN scxa_cell_group_membership m ON\n        g.id = m.cell_group_id AND\n        g.experiment_accession = ':experiment_accession' AND\n        g.variable = ':variable'\n    RIGHT JOIN scxa_coords c ON\n        m.cell_id=c.cell_id AND\n        m.experiment_accession=c.experiment_accession\n    WHERE\n        c.method=':method' AND\n        c.parameterisation-\u003e0-\u003e\u003e'perplexity'=':perplexity' AND\n        c.experiment_accession=':experiment_accession'\n```\n\nStarting from the cell group and right-joining to get the coordinates ensures we get coordinates for all cells, even those without cell group memberships. The experiment accession does appear to be required in both parts of the above query.\n\n### Summary stats for cell group markers\n\n```\nSELECT\n  g.experiment_accession,\n  m.gene_id,\n  g.variable as k_where_marker,\n  h.value as cluster_id_where_marker,\n  g.value as cluster_id,\n  m.marker_probability as marker_p_value,\n  s.mean_expression, s.median_expression\nFROM\n \tscxa_cell_group_marker_gene_stats s\n  JOIN scxa_cell_group g on s.cell_group_id=g.id\n  JOIN scxa_cell_group_marker_genes m ON s.marker_id=m.id\n  JOIN scxa_cell_group h ON m.cell_group_id = h.id\nWHERE\n  g.experiment_accession=':experiment_accession' and\n  m.marker_probability \u003c 0.05 and\n  g.variable = ':k' and\n  expression_type=0\nORDER BY m.marker_probability\n```\n\nNote: The marker stats table references the cell group table twice, once directly, once indirectly via the marker table (which records the precise grouping in which a gene was a marker).\n\n### Find cell groupings with significant markers\n\n```\nSELECT DISTINCT\n  h.variable as k_where_marker\nFROM\n  scxa_cell_group_marker_genes m,\n  JOIN scxa_cell_group h ON m.cell_group_id = h.id\nWHERE\n  h.experiment_accession=':experiment_accession'\n  and m.marker_probability \u003c 0.05\nORDER BY k_where_marker ASC\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febi-gene-expression-group%2Fatlas-schemas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Febi-gene-expression-group%2Fatlas-schemas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febi-gene-expression-group%2Fatlas-schemas/lists"}