{"id":13429371,"url":"https://github.com/SystemsGenetics/gene-oracle","last_synced_at":"2025-03-16T03:31:21.695Z","repository":{"id":69103334,"uuid":"102769475","full_name":"SystemsGenetics/gene-oracle","owner":"SystemsGenetics","description":"Feature extraction algorithm for genomic data","archived":false,"fork":false,"pushed_at":"2021-09-28T19:46:49.000Z","size":17119,"stargazers_count":13,"open_issues_count":5,"forks_count":0,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-10-27T07:32:11.602Z","etag":null,"topics":["bioinformatics","dataset","deep-learning","gene","genetic-algorithm","machine-learning","rna-seq"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SystemsGenetics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-09-07T18:03:02.000Z","updated_at":"2023-09-14T13:53:06.000Z","dependencies_parsed_at":"2024-01-21T03:57:34.139Z","dependency_job_id":null,"html_url":"https://github.com/SystemsGenetics/gene-oracle","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SystemsGenetics%2Fgene-oracle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SystemsGenetics%2Fgene-oracle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SystemsGenetics%2Fgene-oracle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SystemsGenetics%2Fgene-oracle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SystemsGenetics","download_url":"https://codeload.github.com/SystemsGenetics/gene-oracle/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243822312,"owners_count":20353496,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","dataset","deep-learning","gene","genetic-algorithm","machine-learning","rna-seq"],"created_at":"2024-07-31T02:00:36.089Z","updated_at":"2025-03-16T03:31:21.353Z","avatar_url":"https://github.com/SystemsGenetics.png","language":"Python","funding_links":[],"categories":["GitHub Repositories"],"sub_categories":[],"readme":"# Gene Oracle\n\nThis repository contains the code for the Gene Oracle project. Gene Oracle is an ongoing research effort to discover biomarker genes using gene expression data. Gene Oracle identifies gene sets which provide the most predictive power, based on how well they classify samples in a gene expression dataset.\n\nFor more information, refer to the paper: [Uncovering biomarker genes with enriched classification potential from Hallmark gene sets](https://www.nature.com/articles/s41598-019-46059-1)\n\n## Installation\n\nAll of Gene Oracle's dependencies can be installed via Anaconda. On a shared system (such as a university research cluster), it is recommended that you install everything in an Anaconda environment:\n\n```bash\nconda env create -f environment.yml\n```\n\nYou must then \"activate\" your environment in order to use it:\n```bash\nconda activate gene-oracle\n\n# use gene-oracle\n\nconda deactivate\n```\n\nAfter that, simply clone this repository to use Gene Oracle.\n```bash\ngit clone https://github.com/SystemsGenetics/gene-oracle.git\n\n# run the example\ncd gene-oracle\nscripts/run-example.sh\n```\n\n## Usage\n\nGene Oracle consists of two phases, (1) gene set analysis and (2) gene subset analysis. This process encompasses multiple scripts which are run in sequence. The easiest way to learn how to run these scripts, as well as the input / output data involved, is to run the example script as shown above. It demonstrates how to run Gene Oracle on synthetic input data from `make-inputs.py`.\n\n### Model Configuration\n\nGene Oracle can use any classifier provided by scikit-learn, as well as a custom neural network (implemented in TensorFlow), to evaluate gene sets. Several classifiers are defined with sensible default parameters in `models.json`. Consult the [scikit-learn](https://scikit-learn.org/stable/supervised_learning.html#supervised-learning) documention on to see the list of parameters for each classifier. The example run script uses a linear model, which is one of the simplest classifiers available. Other models such as the neural network or random forest may perform better but will take longer to train.\n\n_NOTE: A GPU is required only when using the mlp-tf model with tensorflow-gpu. A GPU might not provide significant speedup a over multicore CPU when training many small neural networks._\n\n### Input Data\n\nGene Oracle takes three primary inputs: (1) a gene expression matrix (GEM), (2) a list of sample labels, and (3) a list of gene sets. These inputs are described below.\n\nThe __gene expression matrix__ should be a plaintext file with rows being samples and columns being genes (features). Values in each row should be separated by tabs.\n```\n\tGene1\tGene2\tGene3\tGene4\nSample1\t0.523\t0.991\t0.421\t0.829\nSample2\t8.891\t7.673\t3.333\t9.103\nSample3\t4.444\t5.551\t6.102\t0.013\n```\n\nFor large GEM files, it is recommended that you convert the GEM to numpy format using `convert.py` from the [GEMprep](https://github.com/SystemsGenetics/GEMprep) repo, as TSPG can load this binary format much more quickly than it does the plaintext format. The `convert.py` script can also transpose your GEM if it is arranged the wrong way:\n```bash\nbin/convert.py GEM.emx.txt GEM.emx.npy --transpose\n```\n\nThis example will create three files: `GEM.emx.npy`, `GEM.emx.rownames.txt`, and `GEM.emx.colnames.txt`. The latter two files contain the row names and column names, respectively. Make sure that the rows are samples and the columns are genes!\n\nThe __label file__ should contain a label for each sample, corresponding to something such as a condition or phenotype state for the sample. This file should contain two columns, the first being the sample names and the second being the labels. Values in each row should be separated by tabs.\n```\nSample1\tLabel1\nSample2\tLabel2\nSample3\tLabel3\nSample4\tLabel4\n```\n\nThe __gene set list__ should contain the name and genes for a gene set on each line, similar to the GMT format. The gene names should be identical to those used in the GEM file. Values on each row should be separated by tabs.\n```\nGeneSet1\tGene1\tGene2\tGene3\nGeneSet2\tGene2\tGene4\tGene5\tGene6\n```\n\n### Phase 1: Gene Set Analysis\n\nThe script `phase1-evaluate.py` takes a list of gene sets and evaluates each gene set by training and evaluating a classifier on the input dataset with only the genes in the set. This script can also evaluate the entire set of genes in the input dataset, as well as random gene sets.\n\nThe script `phase1-select.py` takes evaluation results for gene sets and compares them to results for random sets of equal size. It uses Student's _t_-test to determine the statistical significance of a gene set's score as compared to a null distribution for the given set size. Larger gene sets tend to yield higher classification accuracies, so the _t_-test is used to eliminate this bias when selecting gene sets for subset analysis.\n\n### Phase 2: Gene Subset Analysis\n\nThe script `phase2-evaluate.py` takes a list of gene sets and evaluates subsets of each gene set in order to determine the most salient genes in the gene set. This script can also analyze random gene sets in the same manner.\n\nThe script `phase2-select.py` takes evaluation results for the subsets selected by the previous script, measures the saliency of each gene by how frequently it appeared in all subsets, and separates \"candidate\" genes from \"non-candidate\" genes according to a threshold.\n\n## Nextflow\n\n### Dependencies\n\nThis repository also provides a Nextflow pipeline for running Gene Oracle. All you need is [nextflow](https://nextflow.io/), [Docker](https://docker.com/), and [nvidia-docker](https://github.com/NVIDIA/nvidia-docker). On HPC systems, you can use [Singularity](https://www.sylabs.io/singularity/) in lieu of Docker. If for some reason you can't use either container software, you will have to install Gene Oracle and its dependencies on your local machine.\n\n### Input Data\n\nThe nextflow pipeline assumes you have your input data arranged as follows:\n```\ninput/\n  {dataset1}.emx.txt\n  {dataset1}.labels.txt\n  {dataset2}.emx.txt\n  {dataset2}.labels.txt\n  ...\n  {genesets1}.genesets.txt\n  {genesets2}.genesets.txt\n  ...\n```\n\nThis way, you can place as many gene subsets and datasets and the pipeline will process all of them in a single run.\n\n### Usage\n\nHere is a basic usage:\n```\nnextflow run systemsgenetics/gene-oracle -profile \u003cconda|docker|singularity\u003e\n```\n\nThis example will download this pipeline to your machine and use the default `nextflow.config` in this repo. It will assume that you have Gene Oracle installed natively, and it will process all input files in the `input` directory, saving all output files to the `output` directory, as defined in `nextflow.config`.\n\nYou can also create your own `nextflow.config` file; nextflow will check for a config file in your current directory before defaulting to config file in this repo. You will most likely need to customize this config file as it provides options such as which experiments to run, how many chunks to use where applicable, and various other command-line parameters for Gene Oracle. The config file also allows you to define your own \"profiles\" for running this pipeline in different environments. Consult the Nextflow documentation for more information on what environments are supported.\n\nYou can resume a failed run with the `-resume` flag. Consult the Nextflow documentation for more information on these and other options.\n\n### Kubernetes\n\nYou can run this pipeline, as well as any other Nextflow pipeline, on a [Kubernetes](https://kubernetes.io/) cluster with minimal effort. Consult the [kube-runner](https://github.com/SystemsGenetics/kube-runner) repo for a command-line approach and [Nextflow-API](https://github.com/SciDAS/nextflow-api) for a browser-based approach.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSystemsGenetics%2Fgene-oracle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSystemsGenetics%2Fgene-oracle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSystemsGenetics%2Fgene-oracle/lists"}