{"id":25176045,"url":"https://github.com/quantgen/maize-hub","last_synced_at":"2026-01-30T03:49:26.009Z","repository":{"id":165615428,"uuid":"640999232","full_name":"QuantGen/MAIZE-HUB","owner":"QuantGen","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-20T15:48:59.000Z","size":11465,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-06-08T01:05:59.582Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/QuantGen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-15T15:03:49.000Z","updated_at":"2024-11-17T16:08:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"e166f8b8-ba3e-4820-b288-a03d389a6e2a","html_url":"https://github.com/QuantGen/MAIZE-HUB","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/QuantGen/MAIZE-HUB","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QuantGen%2FMAIZE-HUB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QuantGen%2FMAIZE-HUB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QuantGen%2FMAIZE-HUB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QuantGen%2FMAIZE-HUB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/QuantGen","download_url":"https://codeload.github.com/QuantGen/MAIZE-HUB/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/QuantGen%2FMAIZE-HUB/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28900201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-30T03:36:35.398Z","status":"ssl_error","status_checked_at":"2026-01-30T03:36:34.949Z","response_time":66,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-09T13:15:54.739Z","updated_at":"2026-01-30T03:49:25.980Z","avatar_url":"https://github.com/QuantGen.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# MAIZE-HUB\n\nThis repository has the pipelines used in [Lopez-Cruz *et al*. (2023)](https://doi.org/10.1038/s41467-023-42687-4) for curating the [Genomes to Fields](https://www.genomes2fields.org/) data and to generate Environmental Covariates (EC) for each of the year-locations in the data set. We also provide pipelines we used to analyze this data using Bayesian models.\n\n**Funding**: [NSF PGRP-Tech Grant 2035472](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2035472\u0026HistoricalAwards=false).\n\nIf you have questions about the content of this repository, please contact Marco Lopez-Cruz (lopezcru@msu.edu) and Gustavo de los Campos (gustavoc@msu.edu).\n\n## Menu\n 1. [Requeriments and software](#soft_req)\n 2. [Data source](#data_source)\n 3. [Data curation and environmental covariates workflow](#data_curation)\n 4. [Data analysis and benchmarks modules](#data_analysis)\n 5. [Historical environmental covariates](#hist_ecov)\n   \n## Final curated data set\n\nThe final curated data set can be downloaded form this Figshare repository https://doi.org/10.6084/m9.figshare.22776806.\n\nOnce the files are downloaded and uncompressed, it will generate the following files\n\n```\ndata\n├── ECOV.csv\n├── ECOV_KL.csv\n├── ECOV_layered.csv\n├── ECOV_period.csv\n├── GENO.csv\n├── MAP.csv\n└── PHENO.csv\n```\n\nYou can read them into an R environment using the following code.\n\n```r\n PHENO=read.csv('data/PHENO.csv') \n ECOV=read.csv('data/ECOV.csv', row.names=1)\n GENO=data.table::fread('data/GENO.csv',sep=',',data.table=FALSE) \n rownames(GENO)=GENO[,1]\n GENO=as.matrix(GENO[,-1])\n```\n\n\u003ca name=\"soft_req\"\u003e\u003c/a\u003e\n### 1) Software requeriments\n\nEnvironmental Covariates derivation:\n  - [APSIM Next Generation](https://apsimnextgeneration.netlify.app/) crop modeling tool (\u003e=2021.11.3.6921)\n  - [apsimx](https://cran.r-project.org/web/packages/apsimx/vignettes/apsimx.html) R-package (\u003e=2.3.1)\n  - [tidygeocoder](https://jessecambon.github.io/tidygeocoder/) R-package (\u003e=1.0.5)\n  - [SFSI](https://github.com/MarcooLopez/SFSI) R-package (\u003e=1.3.1)\n\nData analysis:\n  - [BGLR](https://github.com/gdlc/BGLR-R) R-package (\u003e=1.1.0)\n  - [tensorEVD](https://github.com/MarcooLopez/tensorEVD) R-package (\u003e=0.1.1)\n\n\u003ca name=\"data_source\"\u003e\u003c/a\u003e\n### 2) Data source\n\nThe phenotypic and genotypic data can be downloaded from the G2F repository\n  - Phenotypic data: https://www.genomes2fields.org/resources\n  - Genotypic data: https://doi.org/10.25739/tq5e-ak26\n\nThese data must be downloaded and saved in the `source` folder of the following pipeline.\n\n\u003ca name=\"data_curation\"\u003e\u003c/a\u003e\n###  3) Data curation and environmental covariates workflow\n\nThe pipeline used to generate the curated data set can be downloaded from this [link](https://github.com/QuantGen/MAIZE-HUB/blob/main/data_curation_and_ecov.zip).\n\nOnce the file is uncompressed, it will generate a collection of folders, each containing a module of the workflow (see folder tree below). The number that preceeds the folder name indicate the order in which the modules need to be run. Because some modules use outputs from other modules as inputs, as a rule of thumb we recommend running the modules sequentially.\n\nBefore running any module you need to populate the `source` folder with the data downloaded from G2F (see [Data source](#data_source)). \n\nThe overall structure of the workflow is as follows\n\n```\n├── 1_phenotypes\n│   ├── code\n│   │   └── 1_get_phenotype.R\n│   └── output\n├── 2_genotypes\n│   ├── code\n│   │   ├── 1_SNP_filter_codify.sub\n│   │   └── 2_LD_prune_genotypes.R\n│   └── output\n├── 3_year_loc_summary\n│   ├── code\n│   │   └── 1_year_location_summary.R\n│   └── output\n├── 4_weather\n│   ├── code\n│   │   └── 1_get_met_apsim.R\n│   └── output\n├── 5_APSIM\n│   ├── code\n│   │   ├── 1_apsim_sim.R\n│   │   └── 1_apsim_sim.sub\n│   └── output\n├── 6_ecov\n│   ├── code\n│   │   └── 1_get_env_cov.R\n│   └── output\n├── 7_GDD\n│   └── code\n│       └── 1_add_GDD_to_pheno.R\n├── README\n├── source\n│   ├── Agronomic_information\n│   │   | # save here the agronomic information files downloaded from G2F (see Source above), one file per year.\n│   ├── Genotype\n│   │    # save here the compressed vcf file with genotypes, downloaded from G2F\n│   ├── Metadata\n│   │   | # save here the metadata downloaded from G2F (see Source above), one file per year.\n│   └── Phenotype\n│       | # save here the phenotype files downloaded from G2F (see Source above), one file per year.\n└── tools\n    ├── APSIM_functions.R\n    ├── ecov_utils.R\n    ├── Functions.R\n    ├── LD_prune.R\n    ├── read_metadata.R\n    └── read_phenotype.R\n```\n\n\u003ca name=\"data_analysis\"\u003e\u003c/a\u003e\n### 4) Data analysis and benchmarks modules\n\nWe used the curated data set to run various analyses including, principal components analysis, environmental-covariate-phenotype association analysis, variance components estimation, and assessment of prediction accuracy using Bayesian models.\n\nThe workflow used for analysis can be downloaded from the following [link](https://github.com/QuantGen/MAIZE-HUB/blob/main/analysis.zip).\n\nThe uncompressed file will generate the following folders. Unlike the data curation, the analysis modules can be run in any order, since there are no dependencies between modules of the analysis workflow.\n\n```\npipeline_analysis\n├── 1_PC_genotypes\n│   ├── code\n│   │   ├── 1_get_G_matrix_PC.R\n│   │   └── 2_make_Figure_1.R\n│   └── output\n├── 2_PC_ecov\n│   ├── code\n│   │   ├── 1_get_WWt_matrix_PC.R\n│   │   └── 2_make_Figure_2.R\n│   └── output\n├── 3_ecov_varcomps\n│   ├── code\n│   │   ├── 1_ecov_variance_components.R\n│   │   └── 2_make_Figure_3.R\n│   └── output\n├── 4_ecov_WAS\n│   ├── code\n│   │   ├── 1_ecov_WAS_SMR.R\n│   │   └── 2_make_Figures_4_\u0026_S6.R\n│   └── output\n├── 5_drought_heat_stress\n│   ├── code\n│   │   ├── 1_make_Figure_5.R\n│   │   ├── 2_make_Figure_6.R\n│   │   ├── 3_HI30_SDR_mixed_model.R\n│   │   └── 4_make_Figures_7_\u0026_S7.R\n│   └── output\n├── 6_genomic_prediction\n│   ├── code\n│   │   ├── 1_get_model_components.R\n│   │   ├── 2_ANOVA_mixed_model.R\n│   │   ├── 3_make_Tables_3_\u0026_S4.R\n│   │   ├── 4_10F_CV_model.R\n│   │   ├── 5_LYO_CV_model.R\n│   │   └── 6_make_Figures_8_\u0026_S8.R\n│   └── output\n├── README\n└── tools\n    ├── c_utils.c\n    ├── c_utils.o\n    ├── c_utils.so\n    ├── ecov_utils.R\n    ├── Functions.R\n    ├── get_folds.R\n    ├── get_variance.R\n    └── TENSOR_EVD.R\n```\n\n\u003ca name=\"hist_ecov\"\u003e\u003c/a\u003e\n### 5) Historical environmental covariates\n\nHistorical ECs for 23 years (2000-2022) were derived for each of the 28 geographical locations. The final historical EC file contains 207 covariates for all the 644 year-city combinations. This data set can be downloaded from this [link](https://github.com/QuantGen/MAIZE-HUB/blob/main/historical_ecov.zip).\n\nThe uncompressed file will generate the following files.\n\n```\n[historical_ecov]\n├── ECOV.csv\n├── ECOV_KL.csv\n├── ECOV_info.csv\n├── ECOV_layered.csv\n├── ECOV_period.csv\n└── README\n```\n\nThe 'year-city' (e.g., *2000-College Station*) names (i.e., the row names) of the historical EC data can be used to match it to the file of observed phenotypes (`PHENO.csv`). This column does not exist in the `PHENO.csv` file and has to be created. Some locations have 'early' and 'late' planting date and, therefore, have an extra suffix in the name (e.g., *2000-College Station-Early* and *2000-College Station-Late*). The following R-code can be used to create these 'year-city' names in the `PHENO.csv` file.\n\n```r\n PHENO=read.csv('data/PHENO.csv')\n PHENO$year_city=paste0(PHENO$year,\"-\",PHENO$city)\n\n ECOV2=read.csv('historical_ecov/ECOV.csv', row.names=1)\n ECOV_info=read.csv('historical_ecov/ECOV_info.csv')\n\n for(yl in unique(PHENO$year_loc)){\n  index=which(PHENO$year_loc==yl)\n  tmp=ECOV_info[grep(PHENO[index,\"year_city\"][1],ECOV_info$year_city),]\n  dd=as.Date(tmp$date_plant_opt)-mean(as.Date(PHENO[index,\"date_plant\"]))\n  PHENO[index,\"year_city\"]=tmp[which.min(abs(dd)),\"year_city\"]\n }\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquantgen%2Fmaize-hub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquantgen%2Fmaize-hub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquantgen%2Fmaize-hub/lists"}