{"id":24667622,"url":"https://github.com/torkamanilab/imputation_accuracy_calculator","last_synced_at":"2025-10-08T04:31:18.266Z","repository":{"id":46411367,"uuid":"186462760","full_name":"TorkamaniLab/imputation_accuracy_calculator","owner":"TorkamaniLab","description":null,"archived":false,"fork":false,"pushed_at":"2021-12-15T23:22:03.000Z","size":56589,"stargazers_count":12,"open_issues_count":3,"forks_count":5,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-03-15T23:03:55.967Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TorkamaniLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-13T17:05:15.000Z","updated_at":"2024-02-18T06:31:04.000Z","dependencies_parsed_at":"2022-09-23T00:31:17.952Z","dependency_job_id":null,"html_url":"https://github.com/TorkamaniLab/imputation_accuracy_calculator","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TorkamaniLab%2Fimputation_accuracy_calculator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TorkamaniLab%2Fimputation_accuracy_calculator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TorkamaniLab%2Fimputation_accuracy_calculator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TorkamaniLab%2Fimputation_accuracy_calculator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TorkamaniLab","download_url":"https://codeload.github.com/TorkamaniLab/imputation_accuracy_calculator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235679300,"owners_count":19028344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-26T08:18:04.412Z","updated_at":"2025-10-08T04:31:13.922Z","avatar_url":"https://github.com/TorkamaniLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A tool for calculating imputation accuracy by comparing imputation results to WGS data.\n\nDocumentation and scripts provided for calculating imputation accuracy. Tested using Minimac4, Beagle, Impute5 and our deep learning based imputation tool project (in progress). Should work with any imputed VCF file that has GT or GT+DS format fields.\n\n## Requirements\n\n- samtools/bcftools: used to calculate MAFs\n- python v3: source code was implemented and tested on python 3.6\n  - `pandas`\n  - `numpy`\n  - `cyvcf2`\n\n## Required command line arguments are:\n\nThe following inputs, in vcf.gz format, including its respective tabix tbi file, are required to run.\n\n- ga: genotype array file used as input for imputation\n- imputed: imputation results\n- wgs: ground truth file, containing experimentally determined genotypes (i.e. Whole Genome Sequencing data)\n\n**IMPORTANT PREREQUISITE: All files provided must be in vcf.gz format (compressed, tabixed). Alleles must match, in other words: NO SWAPS, NO FLIPS, SAME BUILD!!! It is not necessary to provide allele frequencies, since the tool will calculate it internally using bcftools.**\n\n## Usage:\n\nThe help shows all the required arguments listed above, plus optional arguments.\n\n```\npython3 Compare_imputation_to_WGS.py -h\nusage: Compare_imputation_to_WGS.py --ga \u003cinput_genotype_array.vcf.gz\u003e --imputed \u003cimputed_file.vcf.gz\u003e --wgs \u003cwhole_genome_file.vcf.gz\u003e\nUse -h or --help to display help.\n\narguments:\n  -h, --help            show this help message and exit\n  --ga GA               optional, path to genotype array file in vcf.gz format, with tbi\n  --wgs WGS             required, path to whole genome file in vcf.gz format, with tbi\n  --imputed IMPUTED     required, path to imputed file in vcf.gz format, with tbi\n  --ref REF             optional, path to reference panel file in vcf.gz\n                        format, with tbi. Used for MAF calculation. WGS file\n                        will be used if no reference file is provided.\n  --max_total_rows MAX_TOTAL_ROWS\n                        optional, maximun number of rows or variants to be loaded\n                        simultaneously, summing all chunks loaded by all cores\n  --max_per_core MAX_PER_CORE\n                        optional, maximun number of variants per chunk per core, lower\n                        it to avoid RAM overload\n  --min_per_core MIN_PER_CORE\n                        optional, minimun number of variants per chunk per core,\n                        increase to avoid interprocess communication overload\n  --sout SOUT           optional, output file path/name per sample, default is\n                        the same as the imputed file with\n                        _per_sample_results.txt suffix\n  --vout VOUT           optional, output file path/name per variant, default is\n                        the same as the imputed file with\n                        _per_variant_results.txt suffix\n```\n\nA detailed report with accuracy ratio, F1 score, Pearson correlation (r2) is generated and wrote to the output file (i.e accuracy_result.txt)\n\n## How to run example:\n```\npython3.6 Compare_imputation_to_WGS.py --ga aric_GT_ancestry-5_cad_190625.vcf.chr1.gz --imputed aric_intersectWGS_rpt-1_ancestry-5_phasing-eagle_seed-E_imputed-HRC_cad_190625.vcf.chr1.gz --wgs aric_WGS_ancestry-5_cad_190625.recode.vcf.chr1.gz\n```\n\n## Results:\n\n```\nProcessing  239 imputed samples\nProcessing chunk: 1 Max rows per chunk: 10000\n1 Read imputed file time:  0.0004201233386993408\n2 Chunking time:  8.239876478910446e-06\n3 Calculation time:  0.16841873712837696\n4 Merging calculations per sample time:  0.0037479093298316\nResults per sample at: aric_intersectWGS_rpt-1_ancestry-5_phasing-eagle_seed-E_imputed-HRC_cad_190625.vcf.chr1_per_sample_results.txt\nResults per variant at: aric_intersectWGS_rpt-1_ancestry-5_phasing-eagle_seed-E_imputed-HRC_cad_190625.vcf.chr1_per_variant_results.txt\nTotal run time (sec): 0.17389075085520744\n\n```\n\nThe results will be displayed as the example bellow (per variant):\n```\nposition        SNP     REF_MAF IMPUTED_MAF     WGS_MAF F-score concordance_P0  IQS     r2      precision       recall  TP      TN      FP      FN TP_ratio TN_ratio        FP_ratio        FN_ratio        RMSE\n22:48742072     22:48742072_G_A 0.0003  0.0013  0.0093  0.992   0.984   0.246   0.143   1.0     0.984   754     738     0       12      1.0     0.984       0.0     0.016   0.126\n22:48742498     22:48742498_G_A 0.0028  0.0     0.002   0.998   0.996   0.0     0.014   1.0     0.996   752     749     0       3       1.0     0.996       0.0     0.004   0.063\n22:48743012     22:48743012_C_A 0.061   0.0578  0.0665  0.947   0.882   0.479   0.3     0.946   0.948   804     610     46      44      0.946   0.933       0.054   0.067   0.307\n22:48742525     22:48742525_G_A 0.0041  0.0013  0.0033  0.998   0.996   0.57    0.36    1.0     0.996   754     747     0       3       1.0     0.996       0.0     0.004   0.066\n22:48741511     22:48741511_C_T 0.0009  0.0007  0.0007  1.0     1.0     1.0     0.999   1.0     1.0     753     751     0       0       1.0     1.00.0      0.0     0.002\n22:48741402     22:48741402_A_G 0.2096  0.0439  0.2646  0.836   0.621   0.233   0.185   0.879   0.796   814     370     112     208     0.879   0.640.121   0.36    0.591\n22:48742178     22:48742178_G_A 0.0089  0.0007  0.0153  0.986   0.971   0.081   0.062   1.0     0.972   753     729     0       22      1.0     0.971       0.0     0.029   0.167\n22:48741385     22:48741385_T_C 0.3718  0.113   0.4149  0.863   0.547   0.192   0.168   0.768   0.984   1078    83      325     18      0.768   0.822       0.232   0.178   0.656\n22:48741702     22:48741702_C_T 0.001   0.0047  0.0293  0.976   0.956   0.287   0.165   0.995   0.958   755     712     4       33      0.995   0.956       0.005   0.044   0.238\n```\n\nResults per sample:\n```\nimputed_ids     WGS_ids F-score concordance_P0  r2      precision       recall  TP      TN      FP      FN      TP_ratio        TN_ratio        FP_ratio    FN_ratio        RMSE\nA00003_A00003   A00003_A00003   0.976   0.95    0.497   0.972   0.981   103.0   94.0    3.0     2.0     0.972   0.979   0.028   0.021   0.152\nA00018_A00018   A00018_A00018   0.932   0.851   0.696   0.972   0.896   103.0   84.0    3.0     12.0    0.972   0.875   0.028   0.125   0.296\nA00056_A00056   A00056_A00056   0.99    0.98    0.613   0.981   1.0     104.0   96.0    2.0     0.0     0.981   1.0     0.019   0.0     0.162\nA00080_A00080   A00080_A00080   0.986   0.97    0.671   0.981   0.991   106.0   93.0    2.0     1.0     0.981   0.989   0.019   0.011   0.141\nA00083_A00083   A00083_A00083   0.99    0.98    0.56    0.981   1.0     104.0   96.0    2.0     0.0     0.981   1.0     0.019   0.0     0.128\nA00099_A00099   A00099_A00099   0.99    0.98    0.139   0.99    0.99    102.0   98.0    1.0     1.0     0.99    0.99    0.01    0.01    0.138\nA00120_A00120   A00120_A00120   0.976   0.96    0.5     0.962   0.99    101.0   96.0    4.0     1.0     0.962   0.99    0.038   0.01    0.21\nA00146_A00146   A00146_A00146   0.972   0.941   0.64    0.963   0.981   103.0   93.0    4.0     2.0     0.963   0.979   0.037   0.021   0.176\nA00152_A00152   A00152_A00152   0.986   0.97    0.403   0.981   0.99    103.0   96.0    2.0     1.0     0.981   0.99    0.019   0.01    0.17\n```\n\nThe results can be interpreted as follows.\n\nMetrics per variant:\n- REF_MAF: Reference Panel MAF (if reference panel is provided)\n- IMPUTED_MAF: Imputed MAF\n- WGS_MAF: Whole Genome MAF\n- F-score: macro F-score (not weighted),\n- concordance_P0: accuracy ratio (concordance, from the article cited bellow [1]),\n- IQS: imputation quality score (from the same article [1])\n- precision: precision\n- recall: recall\n- TP: true positives\n- TN: true negatives\n- FP: false positives\n- FN: false negatives\n- TP_ratio: true positives ratio\n- TN_ratio: true negatives ratio\n- FP_ratio: false positives ratio\n- FN_ratio: false negatives ratio\n- RMSE: root mean squared error\n\nMetrics per sample:\n- F-core: F-score per sample\n- concordance_P0: accuracy ratio\n- r2: r-squared\n- precision: precision\n- recall: recall\n- TP: true positives\n- TN: true negatives\n- FP: false positives\n- FN: false negatives\n- TP_ratio: true positives ratio\n- TN_ratio: true negatives ratio\n- FP_ratio: false positives ratio\n- FN_ratio: false negatives ratio\n- RMSE: root mean squared error\n\nCheck sample_data folder for real sample of inputs/outputs data and test run:\n\n```\ncd sample_data\npython3 ../Compare_imputation_to_WGS.py --ga input_genotype_array_100samples.vcf.gz --wgs whole_genome_file_100samples.vcf.gz --imputed imputed_file_100samples.vcf.gz\n```\n\nThis message will show up if everything worked:\n\n```\nProcessing chunk: 1 Max rows per chunk: 10000\n...\nProcessing chunk: 52 Max rows per chunk: 10000\n1 Read imputed file time:  5.211546601727605\n2 Chunking time:  0.0284710843116045\n3 Calculation time:  90.96355835348368\n4 Merging calculations per sample time:  20.620813813060522\nResults per sample at: imputed_file_100samples.vcf_per_sample_results.txt\nResults per variant at: imputed_file_100samples.vcf_per_variant_results.txt\nTotal run time (sec): 117.97938374243677\n\n```\n\nAfter running this example, you can visualize the results for the test sample data in:\n- imputed_file_100samples.vcf_per_sample_results.txt\n- imputed_file_100samples.vcf_per_variant_results.txt\n\nFor example:\n\n```\nhead imputed_file_100samples.vcf_per_variant_results.txt\n\nposition        SNP     IMPUTED_MAF     WGS_MAF F-score concordance_P0  IQS     r2      precision       recall  TP      TN      FP      FN TP_ratio TN_ratio        FP_ratio        FN_ratio        RMSE\n22:16050783     22:16050783_A_G 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.001\n22:16050922     22:16050922_T_G 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.0\n22:16050984     22:16050984_C_G 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.0\n22:16051269     22:16051269_G_T 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.0\n22:16051477     22:16051477_C_A 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.006\n22:16052240     22:16052240_C_G 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.0\n22:16052271     22:16052271_G_A 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.001\n22:16052428     22:16052428_G_A 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.0\n22:16052639     22:16052639_C_T 0.0     0.0     1.0     1.0     0.0     0.0     1.0     1.0     100     100     0       0       1.0     1.00.0      0.0     0.0\n\n```\n\n\n## References:\n\n[1] Ramnarine S, Zhang J, Chen LS, Culverhouse R, Duan W, Hancock DB, Hartz SM, Johnson EO, Olfson E, Schwantes-An TH, Saccone NL. When does choice of accuracy measure alter imputation accuracy assessments?. PloS one. 2015 Oct 12;10(10):e0137601.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftorkamanilab%2Fimputation_accuracy_calculator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftorkamanilab%2Fimputation_accuracy_calculator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftorkamanilab%2Fimputation_accuracy_calculator/lists"}