{"id":13989347,"url":"https://github.com/project-ncl/pnc","last_synced_at":"2025-04-09T10:10:32.847Z","repository":{"id":23681206,"uuid":"27052762","full_name":"project-ncl/pnc","owner":"project-ncl","description":"System for managing, executing, and tracking builds","archived":false,"fork":false,"pushed_at":"2024-04-22T12:42:20.000Z","size":29730,"stargazers_count":50,"open_issues_count":10,"forks_count":58,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-04-22T13:54:01.570Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/project-ncl.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2014-11-23T23:43:43.000Z","updated_at":"2024-04-22T13:54:05.126Z","dependencies_parsed_at":"2023-09-23T06:58:49.572Z","dependency_job_id":"45f6a3a3-fe7b-4f20-baf0-662726e85dd4","html_url":"https://github.com/project-ncl/pnc","commit_stats":null,"previous_names":[],"tags_count":166,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-ncl%2Fpnc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-ncl%2Fpnc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-ncl%2Fpnc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/project-ncl%2Fpnc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/project-ncl","download_url":"https://codeload.github.com/project-ncl/pnc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248018061,"owners_count":21034048,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-09T13:01:36.957Z","updated_at":"2025-04-09T10:10:32.819Z","avatar_url":"https://github.com/project-ncl.png","language":"Java","funding_links":[],"categories":["Java","项目管理"],"sub_categories":[],"readme":"PNC\n===========\nA system for managing, executing, and tracking cross-platform builds.\n\n\n## Running the PNC\n\nPNC is composed of multiple services, to run the system you need:\n\nMinimal:\n- PNC Orchestrator (this repo)\n- [Indy repository manager](https://github.com/Commonjava/indy)\n- [OpenShift](https://www.openshift.org/)\n- [RHSSO](https://access.redhat.com/products/red-hat-single-sign-on) / [KeyCloak](http://keycloak.jboss.org/)\n- [PostgreSQL database](https://www.postgresql.org/)\n\nFull feature setup:\n- [Dependency analysis](https://github.com/project-ncl/dependency-analysis)\n- [jBPM server](http://www.jbpm.org/)\n\nPNC is delivered as JEE EAR package that can be deployed to [EAP](http://developers.redhat.com/products/eap/) / [WildFly](http://wildfly.org/) or other JEE application server. \n\nJEE Server requirements:\n- [JBoss EAP/Wildfly Adapter](https://keycloak.gitbooks.io/securing-client-applications-guide/content/topics/oidc/java/jboss-adapter.html) \n- Hibernate as JPA provider\n- [PostgreSQL JDBC driver](https://jdbc.postgresql.org/)\n\n\n## Developing\n\n**Requirements**\n- JDK 8\n- Maven 3.2\n- Java IDE (IntelliJ IDEA community recommended)\n    - code style template: https://github.com/project-ncl/ide-config\n\n**REST API documentation and testing**\n\nOpenAPI api doc is available via any OpenAPI viewer using this spec http://hostname/pnc-rest/v2/openapi.json\n\n*Requires running instance of PNC \n\n\n## Building\n\nThe default build is executed by running `mvn clean install`.\n\nThe default build does not run \"integration tests\" annotated with @ContainerTest.\nTests annotated with @DebugTest are also skipped by default as they are usually written to run against a remote running server.\n\n\n### Building with all tests\n\nTo run container tests use profile `-Pcontainer-tests`.\nExtra parameter `-Deap6.zip.url` is required to provide the location of server distribution archive.\nTests requiring JEE application server (tested with JBoss EAP 7.0).\n\nExample:\n\n\tmvn clean install -Pcontainer-tests -Deap.zip.url=file:///home/development/jboss-eap.zip\n\nTo run debug tests use `-Pdebug-tests`.\n\nBy default it is required to specify `install` phase to run container tests.\n\nTo use `verify` phase append property `-DuseTargetBuilds` (no value required) to use artifacts generated during package phase in target folder.\n\n#### Installing application server for integration tests manually\n\nApplication server is installed by default to folder target in project top level folder.\nDuring installation additional modules (HSQL, Postgresql JDBC drivers, datasources, RHSSO) required to run integration tests are installed into the server.\n\nTo run only installation of application server to specific folder use `-Dtest.server.unpack.dir=`. \n\nExample to install server to /tmp folder:\n\n\tmvn clean verify -Pcontainer-tests -Deap6.zip.url=SERVER_ZIP_URL -pl :test-common,:test-arquillian-container -Dtest.server.unpack.dir=/tmp\n\n#### Running integration tests using pre-installed and running application server\n\n1. Install application server (see Installing application server for integration tests manually)\n2. start the server\n\n\tsh /tmp/jboss-eap/bin/standalone.sh\n\n3. run integration tests with additional system properties and excluded module test-arquillian-container\n\n\t-Darq.container.wf.configuration.jbossHome=/tmp/jboss-eap\n\t-Darq.container.wf.configuration.allowConnectingToRunningServer=true\n\t-pl \\!:test-arquillian-container\n\n\tmvn clean verify -Pcontainer-tests -pl \\!:test-arquillian-container -Deap6.zip.url= -DuseTargetBuilds -Darq.container.wf.configuration.jbossHome=/tmp/jboss-eap -Darq.container.wf.configuration.allowConnectingToRunningServer=true\n\n\n### Running integration tests in Intellij IDEA\n\n1. Install application server (see Installing application server for integration tests manually)\n2. start the server\n\n\tsh /tmp/jboss-eap/bin/standalone.sh\n\n3. Create new Run/Debug configuration of type Arquillian JUnit\n\n - setup arquillian container - Click button Configure in Arquillian Container tab, add 'Manual container configuration', add 'Maven dependency' in dependencies, type in text field `org.jboss.as:jboss-as-arquillian-container-managed:7.2.0.Final`\n - select test class or package or module in Configuration tab\n - specify path to application server and allow to connect to running server in VM options:\n\n    -Darq.container.wf.configuration.jbossHome=/tmp/jboss-eap\n    -Darq.container.wf.configuration.allowConnectingToRunningServer=true\n\n  - optionally append following properties (no value required) to VM options:\n    -DcreateArchiveCopy - to write ear deployed by arquillian to project folder (eg. for content inspection)\n    -DuseTargetBuilds - to use artifact generated by package phase in target folder \n    \nProperty useTargetBuilds can be used together with configuration 'Run maven goal' `mvn package -DskipTests=true` in 'before lunch' tab to deploy code changes without actual need to run mvn clean install\n\n\n### Building for Production (Postgresql DB)\n\nA Maven profile called `production` is provided to configure the appropriate settings to build a deployment file which is compatible with the postgresql database.\n\n    mvn install -Pproduction\n\nTODO: this is not working currently, integration tests are always run against HSQLDB\nThe container tests can also be run against postgresql by activating the `container-tests` profile, and the `production` profile.\n\n    mvn install -Pproduction,container-tests\n\n\n## Configuration\n\nAll configurations are centralized in the configuration file `moduleconfig/src/main/resources/pnc-config.json`.\n\nYou can copy the file to external location and specify the location via `-Dpnc-config-file=/path/to/pnc-config.json` parameter.\n\nOr you can configure the app using the environment variables listred in the file.\n\nFor the configuration descriptions see api doc of classes in `moduleconfig/src/main/java/org/jboss/pnc/common/json/moduleconfig`\n\n### PollingMonitor configuration\n\nIf you want to specify the ThreadPool size for `PollingMonitor`, you can do so by using the system property `polling_monitor_threadpool`.\n\nFor example, you can specify via `-Dpolling-monitor-threadpool=10` parameter\n\nYou can also specify the environment variable `polling_monitor_threadpool`.\n\nNote that the system property has precedence over the environment variable if both are defined.\n\n## Database set-up\n\n### Manually Configuring the Datasource for HSQL DB\n\nYou will need to download the [hsqldb jar file](http://repo1.maven.org/maven2/org/hsqldb/hsqldb/2.3.3/hsqldb-2.3.3.jar) and copy the jar file into the standalone/deployments directory of your JBoss server.  Check the server log to see that the driver deployed successfully.\n\nFrom the EAP/Wildfly admin console, select Configuration--\u003eConnector--\u003eDatasources.\nClick the \"Add\" button to add a new datasource and set the required fields.\n\n    Name: NewcastleTestDS\n    JNDI Name: java:jboss/datasources/NewcastleTestDS\n    JDBC Driver: hsqldb-2.3.3.jar\n    Connection URL: jdbc:hsqldb:mem:newcastletestmemdb\n\nYou can test the connection before saving the datasource settings.  Finally, enable the datasource, and it is ready to be used by the newcastle application.\n\n\n### Installing Postgres\n\nBasic installation instructions for Postgres on recent versions of Fedora can be found on the Fedora wiki: https://fedoraproject.org/wiki/PostgreSQL\n\n(Optional) In addition to the postgresql server, it's useful to install the gui admin tool pgadmin3 (dnf install pgadmin3).\n\nOnce postgresql is installed and started, you can create and modify the databases using the psql command.\n\nConnect to postgres\n\n    $ psql -h localhost newcastle -U \u003cROOT_USERNAME\u003e\n\nCreate the user \"newcastle\nCreate user\n\n    postgres=# CREATE USER newcastle WITH PASSWORD newcastle;\n\nCreate the newcastle database\nCreate newcastle database\n\n    postgres=# CREATE DATABASE newcastle OWNER newcastle;\n\nOnce the database is created, the schema can be built using the SQL files included with the newcastle source code, or it can be created automatically if it doesn't exist.\n\n\n### Configuring the Datasource\n\nYou will need to download and install the PostgreSQL JDBC driver into Wildfly (https://jdbc.postgresql.org/download.html).  Copy the postgresql jdbc driver jar into the standalone/deployments directory of your JBoss server.  Check the server log to see whether the driver deployed successfully.\n\nFrom the EAP/Wildfly admin console, select Configuration--\u003eConnector--\u003eDatasources.\nClick the \"Add\" button to add a new datasource and set the required fields.\n\n    Name: NewcastleDS\n    JNDI Name: java:jboss/datasources/NewcastleDS\n    JDBC Driver: postgresql-9.3.jar\n    Connection URL: jdbc:postgresql://localhost:5432/newcastle\n    Username: newcastle\n    Password: newcastle\n\nYou can test the connection before saving the datasource settings.\n\n\n## Authentication\n\nThis project comes with possibility to be secured. Security is delivered via Keycloak project http://keycloak.jboss.org/.\nTo be able to turn on whole project on secure side you need 2 parts to fulfill.\n1. **Have running and configured Keycloak server instance.** \u003cbr/\u003e\n2. **Build \u0026 configure security for your PNC installation.**\n\n\n### Authentication - prepare Keycloak server\u003cbr/\u003e\n**PRE-REQUIREMENTS:** \u003cbr/\u003e\n\nInstall your Keycloak server standalone or in Openshift according to \n* http://keycloak.github.io/docs/userguide/html/server-installation.html.\n* http://keycloak.github.io/docs/userguide/html/openshift.html\n\u003cbr/\u003e\n\n**Configure Keycloak server**\u003cbr/\u003e\n\n1. Create your realm \nBy default the Keycloak server comes with \"master realm\", which is for demo purpose, so please create your own realm.\n\u003cbr/\u003e\n2. Put your Direct Grant API on at https://`\u003cyour-server\u003e`/auth/admin/master/console/#/realms/`\u003cyour-realm\u003e`/login-settings\n\u003cbr/\u003e\n3. Add/Create your users via https://`\u003cyour-server\u003e`/auth/admin/master/console/#/realms/`\u003cyour-realm\u003e`/users\n\u003cbr/\u003e\n4. Define roles and assign users to it via https://`\u003cyour-server\u003e`/auth/admin/master/console/#/realms/`\u003cyour-realm\u003e`/roles\n\u003cbr/\u003e\n5. Create 3 client app's at https://`\u003cyour-server\u003e`/auth/admin/master/console/#/realms/`\u003cyour-realm\u003e`/clients \n\u003cbr/\u003e\n6. First client app for pncweb UI with:\n  * Client Protocol = openid-connect\n  * Access Type = confidental\n  * Valid Redirect URIs = http://localhost:8080/pnc-web/*  (Please add URI's for different host's as you need for your installed pnc Web UI's)\n  * Go to \"Installation\" tab and select \"Keycloak JSON\" format and copy\u0026paste or download the installation.\n7. Second client app for pncrest with:\n  * Client Protocol = openid-connect\n  * Access Type = confidental\n  * Valid Redirect URIs = http://localhost:8080/pnc-rest/*  (Please add as much URI's as you need for your installed pnc rest's)\n  * Go to \"Installation\" tab and select \"Keycloak JSON\" format and copy\u0026paste or download the installation.\n8. Third client app for pncdirect with:\n  * Client Protocol = openid-connect\n  * Direct Grants Only = ON\n  * Access Type = public\n  * Go to \"Installation\" tab and select \"Keycloak JSON\" format and copy\u0026paste or download the installation.\n\u003cbr/\u003e\n\n**HINTS** \u003cbr/\u003e\n\nConfigure keycloak subsystem secure-deployment in EAP/wildfly stanbdalone.xml\n\n1. pncrest additional props -\u003e use these below for skipping ssl \u0026 defining rest for accepting only access_token for authentication/authorization.\n  * \"ssl-required\": \"none\",\n  * \"bearer-only\" : true,\n2. pncweb additional props -\u003e look at http://keycloak.github.io/docs/userguide/html/ch08.html#adapter-config to find more about adapter's config.\n  * \"ssl-required\" : \"none\",\n  * \"use-resource-role-mappings\" : false,\n  * \"enable-basic-auth\" : false,\n  * \"enable-cors\" : true,\n  * \"cors-max-age\" : 1000,\n  * \"cors-allowed-methods\" : \"POST,PUT,DELETE,GET\",\n  * \"bearer-only\" : false,\n\n\n### Authentication - build \u0026 configure your PNC\u003cbr/\u003e\nBy default PNC project comes with no security at all, it is up to you to turn it on.\nIn case you want to enable authentication use `-Pauth` together with your build command.\nEnabling authentication means following\u003cbr/\u003e\n\n1. Your backend REST endpoints will become secured\n  * rest.war will use the web.xml with security-constraints \u0026 security-roles (rest/src/main/webconfig/auth/web.xml)\n2. Your pnc web UI gain the SSO ability and authentication via Keycloak login page.\n  * with your first unauthenticated session you will be redirected from pnc web UI into Keycloak login page.\n  * you will be asked to provide your credentials.\n  * after successful log-in you will be redirected back to pnc web UI.\n3. Configure your JEE server (EAP) for keycloak\n  * Use -Dauth.eap.home=``\u003cpath to your EAP installation\u003e`` with you build command, if you want EAP configure for Keycloak.\n  According the http://docs.jboss.org/keycloak/docs/1.1.0.Final/userguide/html/ch08.html#jboss-adapter-installation installation will be performed on server for the given path.\n\nBefore running the PNC you have to update the pnc-config.json and set \"authenticationProviderId\" to \"Keycloak\" instead of the default \"JAAS\" if you want to use Keycloank authentication.\n\n## Configuring the Openshift pod definitions\n\nPNC is using OpenShift to provide clean build environmtn for each build.\nYou can override the pod definitions used to configure the build-agent pod via pnc configuration:\n\n```json\n{\n  \"@module-group\": \"pnc\",\n  \"configs\": [\n      {\n        \"@module-config\": \"openshift-build-agent\",\n          \"pncBuilderPod\": {...},\n          \"pncBuilderService\": {...},\n          \"pncBuilderRoute\": {...},\n          \"pncBuilderSshRoute\": {...}\n      },\n      ...\n  ]\n```\nYou only need to define the key for which you want to modify its definition. There is no need to define all the keys.\n\n\n## Metrics support\n\nPNC tracks metrics of JVM and its internals via Dropwizard Metrics. The metrics can currently be reported to a Graphite server by specifying as system property or environment variables those properties:\n- metrics\\_graphite\\_server (mandatory)\n- metrics\\_graphite\\_port (mandatory)\n- metrics\\_graphite\\_prefix (mandatory)\n- metrics\\_graphite\\_interval (optional)\n\nIf the `metrics_graphite_interval` variable (interval specified in seconds) is not specified, we'll use the default value of 60 seconds to report data to Graphite.\n\nThe graphite reporter is configured to report rates per second and durations in terms of milliseconds.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproject-ncl%2Fpnc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fproject-ncl%2Fpnc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fproject-ncl%2Fpnc/lists"}