{"id":16122127,"url":"https://github.com/foxcapades/eupath-build-plan","last_synced_at":"2026-02-14T13:03:44.874Z","repository":{"id":90411024,"uuid":"206343668","full_name":"Foxcapades/EuPath-Build-Plan","owner":"Foxcapades","description":"WIP EuPathDB Build rework proposal.","archived":false,"fork":false,"pushed_at":"2019-09-10T14:26:50.000Z","size":186,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-06T14:59:32.681Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Foxcapades.png","metadata":{"files":{"readme":"readme.adoc","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-04T14:50:00.000Z","updated_at":"2019-09-10T14:26:52.000Z","dependencies_parsed_at":"2023-03-13T17:58:36.894Z","dependency_job_id":null,"html_url":"https://github.com/Foxcapades/EuPath-Build-Plan","commit_stats":{"total_commits":36,"total_committers":1,"mean_commits":36.0,"dds":0.0,"last_synced_commit":"5154c6de0069f02192701720ddd259b7b259326d"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Foxcapades/EuPath-Build-Plan","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Foxcapades%2FEuPath-Build-Plan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Foxcapades%2FEuPath-Build-Plan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Foxcapades%2FEuPath-Build-Plan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Foxcapades%2FEuPath-Build-Plan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Foxcapades","download_url":"https://codeload.github.com/Foxcapades/EuPath-Build-Plan/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Foxcapades%2FEuPath-Build-Plan/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29444072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-14T12:43:28.304Z","status":"ssl_error","status_checked_at":"2026-02-14T12:43:14.160Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-09T21:09:37.635Z","updated_at":"2026-02-14T13:03:44.847Z","avatar_url":"https://github.com/Foxcapades.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"= Build System\n:icons: font\nifdef::env-github[]\n:warning-caption: ⚠\nendif::[]\n:toc: left\nElizabeth Harper \u003cepharper@upenn.edu\u003e\n\n// External URLS\n:gradleDocRoot: https://docs.gradle.org/current/userguide\n:condSubstBlog: https://medium.com/codequest/Gradle-multi-project-build-substituting-jar-dependencies-with-local-projects-4a5323f8680b\n:condSubstDocs: {gradleDocRoot}/customizing_dependency_resolution_behavior.html\n:docSrc: https://github.com/Foxcapades/EuPath-Build-Plan\n:docPage: https://Foxcapades.github.io/EuPath-Build-Plan\n:javaPlatDocs: https://docs.gradle.org/current/userguide/java_platform_plugin.html\n:cpanGitUse: https://metacpan.org/pod/distribution/App-cpanminus/bin/cpanm\n:searchRes: {docSrc}/blob/master/search-results\n:searchAnt: {searchRes}/ant.txt\n:searchMaven: {searchRes}/maven.txt\n:gradleDocs: {gradleDocRoot}/userguide.html\n\n\nDocument Render: {docPage} +\nDocument Source: {docSrc}\n\n\n== Goals\n\n1. Modernize our build process\n2. Break apart the monolith build\n3. Simplify/Unify our build scripts\n4. Separate build from deployment\n5. Separate build from version control\n\n\n== Increments\n\n\n=== Part 1: Maven to Gradle\n\nExchange the maven pom files for \u003c\u003cgradle,Gradle\u003e\u003e builds.  Parts of\nthe Ant build that call to maven will now call to Gradle.\n\nGradle projects will declare their project dependencies and\nthe build should function similarly to the previous build\nsystem.\n\nThe base-pom file will be converted to a Gradle \u003c\u003cjavaPlat,\n`java-platform`\u003e\u003e Bill of Materials.\n\nBoilerplate details, such as developer information or custom\n\u003c\u003cartRep,Artifact Repositories\u003e\u003e will be extracted to a\nGradle plugin which can be published to a Maven repository.\n\nThis conversion is almost 1 to 1 and wil be tedious, but\nshould not be not difficult.  This particular change should\nonly affect users in that the log output will be different.\n\nAt this point in time the Gradle distribution will follow\nthe standard practice of including a base version in the\nsource repository.\n\n{searchMaven}[All references to Maven] will need to be\nremoved or updated at this time.\n\n\n==== Impact on Servers\n\nNone as \u003c\u003cgradle,Gradle\u003e\u003e will be included with the source code.\n\nFor developer convenience (potentially faster builds) it may\nbe nice to have Gradle installed on the development servers,\nbut this is not required.\n\nWARNING: TODO: get a list of all non-project scripts that\n  call Maven directly\n\n\n==== Impact on Jenkins\n\nNone at this point, the Jenkins `rebuilder` script will call\nAnt as normal.\n\n==== Impact on Developers\n\nDifferent looking log output during build.\n\nPotentially faster project build times.\n\n\n=== Part 2: Ant to Gradle\n\nConvert the build XML declarations into \u003c\u003cgradle,Gradle\u003e\u003e actions.\n\nAt this point, custom scripts will be left alone, calls to\nthem will be maintained, and converted to calls from Gradle.\n\nMost Ant actions or behaviors are available easily in Gradle\nor can be easily recreated as a function in Gradle.  Any Ant\nfunctions we rely on which have no built-in mirror in Gradle\nwill be made into a plugin which will be published to our\ninternal Maven repository to be made available to all\nprojects.\n\n{searchAnt}[All references to Ant] will need to be replaced\nor updated at this point.\n\n\n==== Impact on Servers\n\nThe `rebuilder` script will be updated to call Gradle rather\nthan Ant.\n\nWARNING: TODO: get a list of all non-project scripts that\n  call ant directly.\n\n\n==== Impact on Jenkins\n\nThe Jenkins `rebuilder` script will be updated to call\nGradle rather than Ant.\n\n==== Impact on Developers\n\nAny Ant calls will now be made through Gradle.\n\n\n=== Part 3: Scripts to Gradle\n\nConvert the custom wrapper and helper scripts into \u003c\u003cgradle,\nGradle\u003e\u003e actions.\n\nThis will mean custom (Java) code will need to be written.\nThis code will be created as a Gradle plugin and published\nto our internal Maven server so as to be made available to\nall project builds.\n\nWARNING: TODO: Get a list of all project wrapper scripts\n\nWARNING: TODO: Get a list of all project calls to those\n  wrapper scripts\n\n\n==== Impact on Servers\n\nWARNING: TODO: Get a list of all non-project wrapper scripts\n\nWARNING: TODO: Get a list of all non-project calls to both\n  project and non-project wrapper scripts\n\n\n==== Impact on Jenkins\n\nWARNING: TODO: figure this out!\n\n\n==== Impact on Developers\n\n\nWARNING: Find out what scripts users actually call anymore\n  many may be stripped out entirely.\n\n\nScripts that users call will be converted to gradle actions.\n\n\n=== Part 4: Split the monolith\n\nTake each project's \u003c\u003cgradle,Gradle\u003e\u003e build and convert its\ndependency on sibling projects into artifact dependencies\nwith \u003c\u003c#condSubst,Conditional Substitution\u003e\u003e.\n\n.Early phase Gradle dependency\n[source, groovy]\n----\n// Require local projects\ndependencies {\n  compile project(':FgpUtil.Util')\n  compile project(':WDK.Model')\n  compile project(':WDK.Service')\n  ...\n}\n----\n\n.Late phase Gradle dependency def\n[source, groovy]\n----\n// Use local project if available\nconfigurations.all {\n  dependencies.compile { -\u003e\n    if it.location.exists {\n      substitute module(it.name) with project(\"${it.group}.${it.name}\")\n    }\n  }\n}\n\n// Otherwise default to compiled jar\ndependencies {\n  compile 'org.gusdb.FgpUtil:Util'\n  compile 'org.gusdb.WDK:Model'\n  compile 'org.gusdb.WDK:Service'\n  ...\n}\n----\n\n\n=== Part 5: Deployments\n\nUsing a Gradle task on the root projects themselves, a\nseparate 'deploy' project, or some other tool if we are okay\nwith duplicating our dependency list, we can build a tar or\ntars that bundle the required components for a site. Jenkins\ncan then copy the tar(s) to the desired location and run any\nexisting post build steps (such as WDKCache).\n\n[WARNING]\nTODO: +\nHow will conifer fit into this? +\ncan it be moved to a post step? +\nhow much work would that be?\n\n\n\n== Issues / Concerns\n\n\n=== Bundling of non-java project code\n\nWith \u003c\u003cgradle,Gradle\u003e\u003e's customizable nature, additional\ncompile tasks can be added for non-Java source code.\n\nIn CI additional steps can be performed to publish those\ncomponents as bundles or packages as per the standard\npractices for the specific language.\n\nIn early phases of the build update, non-Java dependencies\nwill be treated as they are in the current build process.\n\nWhen we migrate towards a separate build/deploy strategy\nwe will need to have a system in place to host, version, and\nupdate these dependencies.\n\nDependencies that do not have a package manager may be\nbundled as a OS package for yum or apt, or alternatively\nthese may just be pulled at deploy time by the deployment\nscript.\n\n==== Possible approaches by language\n\n\n===== Perl\n\nWith Perl we have a couple of options available to us.  We\ncan use a private CPAN repository, or alternatively, we can\nsplit the Perl code into separate git repositories and\ndepend on it {cpanGitUse}[via git itself].\n\n\n====== Versioning\n\nDepending on the package hosting approach above we may use\ndifferent strategies, using git we can use git tags to\nmaintain releases or different versions.  Using a private\nCPAN repository, we would need to come up with a concrete\nversioning strategy.\n\n\n===== JavaScript/TypeScript\n\nThis is already handled for us via NPM or Yarn.  Similarly\nto Perl, we may desire to host our JavaScript \u0026 TypeScript\ncode in a private NPM repository, or split that code out and\njust use git.\n\n\n==== R\n\nWARNING: TODO\n\n\n==== C\n\nWARNING: TODO\n\n\n==== Python\n\nWARNING: TODO\n\n\n==== Others?\n\nWARNING: TODO\n\n\n=== Artifact Versioning\n\nWARNING: Everything about this needs to be discussed.  The\n  below sections outline a rough idea that may work for our\n  team.\n\n\n==== Releases\n\nFor releases, artifacts will be built using the version\nformat `\u003csiteRelease\u003e.\u003cartifactBuild\u003e` or, for example\n`46.0`.  The java platform project will be updated to\nreflect the concrete versions of the artifacts that will be\nused for a site release.\n\nThe `\u003cartifactBuild\u003e` will be the number of builds of the\nrelease artifact for a site version, so as live patches are\nadded, this number will increment.\n\n.Example Release History\n[source]\n----\nWDK:\n  48.0 - Site Release\n  48.1 - Memory leak fix\n  49.0 - Site Release\n  49.1 - Corrected typo\n  49.2 - Fix for user-comment search\n----\n\n\n==== Development\n\nWhile in development artifacts will be versioned using the\nformat `\u003cnextRelease\u003e.SNAPSHOT-\u003cartifactBuildNumber\u003e`.\n\nThe `\u003cartifactBuildNumber\u003e` (and potentially `\u003cnextRelease\u003e`)\nparams will be managed by CI to avoid conflicts.\n\nThe development \u003c\u003cjavaPlat,Java Platform\u003e\u003e will be given a\nwildcard version identifier that simply specifies that\nbuilds should use the latest artifact.\n\n.Example Version History\n[source]\n----\nWDK:\n  46.SNAPSHOT-105: Added public user datasets to user dataset list\n  46.SNAPSHOT-106: Corrected issue with deleted public user datasets\n--- Site Build 47 released---\n  47.SNAPSHOT-1: Added user id field to service endpoint result\n  47.SNAPSHOT-2: Reworked question service\n----\n\n\n=== Multi-project development workflow\n\nInitially the projects will declare each other as local\ndependencies which will mean that components will be rebuilt\nas needed in a manner similar to the previous builds.\n\nEventually dependencies will be handled with conditional\nsubstitutions, a built in Gradle feature, which will use a\nlocal project when available instead of the Maven repository\nversion of that dependency.\n\n\n== Questions\n\n* Depending on different language dependencies?\n** These are runtime dependencies and the bundling of the\n   deployable package will be managed by the deployment\n   script.\n+\nPerhaps this means we should split repositories in git along\nlanguage lines?\n\n* How will conifer fit into this world of separate projects?\n\n* Factoring builds?\n\n* Division of deployments?\n\n* Triggers for builds?  (build cascade when a dependency)\n** Does conditional substitution trigger child project builds?\n\n* Should conifer be split out of FgpUtil into it's own project?\n** Should all non-java stuff in FgpUtil be split into it's own project?\n\n\n== Definitions\n\n\n[#artRep]\n=== Artifact Repository\n\nAn Artifact Repository is a server or system designed to\nprovide hosting and versioning for bundled packages of code\nor resources for distribution or linking as complete\napplications or libraries.\n\nMost widely used languages have one or more commonly used\nPackage Managers which will generally use an Artifact\nRepository, or Source Repository when retrieving or\ninstalling applications or dependencies.\n\n\n[#condSubst]\n=== Conditional Substitution\n\nA Gradle feature that enables the substitution of one\ndependency with another if some condition is met.\n\nIn our case this would likely mean using a sibling project\ninstead of an artifact if that project exists in your\nworkspace.\n\nSee {condSubstBlog}[this blog post], or {condSubstDocs}[the\nGradle docs] for a more detailed explanation.\n\n\n[#gradle]\n=== Gradle\n\nGradle is a build and dependency management and scripting\ntool specifically designed around working with JVM\nlanguages.\n\nGradle's functionality covers the use cases for both Ant and\nMaven, and is easily extensible to include our custom build\ntasks.\n\nSee {gradleDocs}[the Gradle user manual] for more\ninformation.\n\n\n[#javaPlat]\n=== Java Platform\n\nThe \u003c\u003cgradle,Gradle\u003e\u003e method of creating a Bill of Materials\nfor a project or group of projects.\n\nThe Java Platform can be used to define and/or constrain\nprojects to a known working set of dependencies.  Similar to\nour base-pom files, our dependency versions can be declared\nhere and omitted from individual project dependency\ndeclarations.\n\nSee {javaPlatDocs}[the Gradle docs] for more information.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoxcapades%2Feupath-build-plan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffoxcapades%2Feupath-build-plan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoxcapades%2Feupath-build-plan/lists"}