{"id":13748664,"url":"https://github.com/hortonworks/cloudbreak","last_synced_at":"2025-06-25T22:06:40.065Z","repository":{"id":16877953,"uuid":"19638422","full_name":"hortonworks/cloudbreak","owner":"hortonworks","description":"CDP Public Cloud is an integrated analytics and data management platform deployed on cloud services. It offers broad data analytics and artificial intelligence functionality along with secure user access and data governance features.","archived":false,"fork":false,"pushed_at":"2025-06-20T08:52:20.000Z","size":231308,"stargazers_count":358,"open_issues_count":43,"forks_count":238,"subscribers_count":351,"default_branch":"master","last_synced_at":"2025-06-20T09:24:13.847Z","etag":null,"topics":["big-data","cloud","cloudera","deployment","hacktoberfest","hadoop","java"],"latest_commit_sha":null,"homepage":"https://docs.cloudera.com/cdp-public-cloud/cloud/index.html","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"dokterbob/wake-up-light","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hortonworks.png","metadata":{"files":{"readme":"README-devnotes.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"audit-connector/build.gradle","citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2014-05-10T10:03:07.000Z","updated_at":"2025-06-20T08:52:25.000Z","dependencies_parsed_at":"2023-09-26T16:24:19.320Z","dependency_job_id":"20c62b7a-f658-4bdb-b623-edf7bf8f36da","html_url":"https://github.com/hortonworks/cloudbreak","commit_stats":null,"previous_names":[],"tags_count":12723,"template":false,"template_full_name":null,"purl":"pkg:github/hortonworks/cloudbreak","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hortonworks%2Fcloudbreak","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hortonworks%2Fcloudbreak/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hortonworks%2Fcloudbreak/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hortonworks%2Fcloudbreak/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hortonworks","download_url":"https://codeload.github.com/hortonworks/cloudbreak/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hortonworks%2Fcloudbreak/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261960527,"owners_count":23236574,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","cloud","cloudera","deployment","hacktoberfest","hadoop","java"],"created_at":"2024-08-03T07:00:46.966Z","updated_at":"2025-06-25T22:06:40.057Z","avatar_url":"https://github.com/hortonworks.png","language":"Java","readme":"# Dev Environment Setup\nThis document briefly provides the instructions to setup local development environment. is based upon the primary [README.md](https://github.infra.cloudera.com/cloudbreak/cloudbreak/blob/master/README.md) and is more practical set of instructions that will complete the setup. \n## Install CBD\nFollow these steps to install first time\n`git pull` or `git clone` the latest cloudbreak-deployer from [github](https://github.infra.cloudera.com/cloudbreak/cloudbreak-deployer.git)\n\nDo the following to install the CBD on local\n\n```\nmkdir cbd-local\ncd cbd-local\ncurl -s https://github.infra.cloudera.com/raw/cloudbreak/cloudbreak-deployer/master/install-dev | sh \u0026\u0026 cbd --version\n```\n\nThen create a file Profile and copy the following content to it:\n\n```\nexport CB_LOCAL_DEV_LIST=\nexport UAA_DEFAULT_SECRET= \u003c default secret value\u003e\n\nexport AWS_ACCESS_KEY_ID= \u003caws key id\u003e\nexport AWS_SECRET_ACCESS_KEY= \u003caws secret access key\u003e\nexport CB_AWS_ACCOUNT_ID= \u003caws account id\u003e\n\nexport CLOUDBREAK_SRC= \u003cYour path to CloudBreak source code e.g./Users/ujjwal/cloudera/src/cloudbreak\u003e\n\nexport CB_SCHEMA_SCRIPTS_LOCATION=$CLOUDBREAK_SRC/core/src/main/resources/schema\nexport DATALAKE_SCHEMA_SCRIPTS_LOCATION=$CLOUDBREAK_SRC/datalake/src/main/resources/schema\nexport ENVIRONMENT_SCHEMA_SCRIPTS_LOCATION=$CLOUDBREAK_SRC/environment/src/main/resources/schema\nexport FREEIPA_SCHEMA_SCRIPTS_LOCATION=$CLOUDBREAK_SRC/freeipa/src/main/resources/schema\nexport PERISCOPE_SCHEMA_SCRIPTS_LOCATION=$CLOUDBREAK_SRC/autoscale/src/main/resources/schema\nexport REDBEAMS_SCHEMA_SCRIPTS_LOCATION=$CLOUDBREAK_SRC/redbeams/src/main/resources/schema\nexport EXTERNALIZEDCOMPUTE_SCHEMA_SCRIPTS_LOCATION=$CLOUDBREAK_SRC/externalized-compute/src/main/resources/schema\nexport REMOTE_ENVIRONMENT_SCHEMA_SCRIPTS_LOCATION=$CLOUDBREAK_SRC/environment-remote/src/main/resources/schema\n\nexport ULU_SUBSCRIBE_TO_NOTIFICATIONS=true\nexport CB_INSTANCE_UUID=$(uuidgen | tr '[:upper:]' '[:lower:]')\nexport CB_INSTANCE_NODE_ID=5743e6ed-3409-420b-b08b-f688f2fc5db1\nexport PUBLIC_IP=localhost\nexport VAULT_AUTO_UNSEAL=true\nexport DPS_VERSION=2.0.0.0-142\n```\n\nExecute on command prompt: `cbd restart`\n\nThis will do the following:\n\nInstall the CBD using installer. Containers for Cloudbreak environment. Install the vault and configure the vault password. It will also install certs in the CBD-LOCAL folder\n\nCreate a file cbd-local/etc/license.txt  and copy the content from [here](https://github.infra.cloudera.com/raw/Starship/cmf/cdpd-master/web/dev_configs/test-license.txt.asc \"https://github.infra.cloudera.com/raw/Starship/cmf/cdpd-master/web/dev_configs/test-license.txt.asc\") in the file.\n\n## Update Latest CBD\nIn most cases `cbd update-to-dev` command should get the latest published CBD binary from build repository.\nHowever another way to install from Source may be required.\nUse these steps to build CBD from the source.\nChange directory using `cd cloudbreak-deployer` and execute the release-docker target using make. \nThis is the easiest way to build CBD as it will use the build container with all necessary build dependencies. \n```\nmake release-docker\nwhich cbd\ncp build/Darwin/cbd /usr/local/bin/\n```\nAnd then start the environment in local docker container as per the Profile configurations.\n```\ncbd restart\n```\nOnce all the containers have come up you can test.\nTest the environment by opening the control panel console app from [Locally Cloudera Management Console http://localhost/](http://localhost)\n\n## Configure IntelliJ\n\nCreate a file idea.env in cbd-local with following values:\n\n```\nAWS_ACCESS_KEY_ID= \u003caws key id\u003e\nAWS_SECRET_ACCESS_KEY=\u003caws secret access key\u003e\nCB_AWS_ACCOUNT_ID= \u003caws account id\u003e\nAWS_STS_REGIONAL_ENDPOINTS=regional\nVAULT_ROOT_TOKEN=\nAWS_USE_FIPS_ENDPOINT=false\nCM_PRIVATE_REPO_USER=cloudbreak-deployer\nCM_PRIVATE_REPO_PASSWORD=\u003crepo password\u003e\nCLUSTERPROXY_ENABLED=false\nALTUS_UMS_HOST=localhost\nGATEWAY_CERT_GENERATION_ENABLED=false\nCB_DB_PORT_5432_TCP_ADDR=localhost\nCB_DB_PORT_5432_TCP_PORT=5432\nDATALAKE_DB_PORT_5432_TCP_ADDR=localhost\nDATALAKE_DB_PORT_5432_TCP_PORT=5432\nREDBEAMS_PORT_5432_TCP_ADDR=localhost\nREDBEAMS_DB_PORT_5432_TCP_PORT=5432\nCB_CERT_DIR=/Users/ujjwal/cloudera/cbd-local/certs\nAUTH_CONFIG_DIR=/Users/ujjwal/cloudera/cbd-local\nCB_IDENTITY_SERVER_URL=http://localhost:8089\nCB_ENABLEDPLATFORMS=AWS,AZURE,GCP,MOCK,YARN\n\n\n```\n\nCreate another file datalake.env with following value\n\n```\nSERVER_PORT=8086\n```\n\nConfigure intelliJ Plugin = envfile plugin to IntelliJ open *settings\u003eplugin* select *marketplace* and search for `EnvFile` plugin\n\nSelect Services from the side bar. For each servince in CloudBreak, DatalakeApplication, EnvironmentApplication, FreeIPAApplication, MockInfrastructureApplication, MockThunderheadApplication.\n\nAnd select each service's *edit configuration* select Enable EnvFile and add [+] .env file `cbd-local/idea.env`\n\nDatalakeApplication also needs the env files `cbd-local/datalake.env`\n\n## Setup Local Profile\n\n### configure CDP config \u0026 cred\n\nThe mock access key for a local CB can be generated by visiting this in the browser:\n\nhttps://localhost/thunderhead/auth/mockkey/TENANT/EMAIL\n\n[https://localhost/thunderhead/auth/mockkey/cloudera/upriyadarshan@cloudera.com](https://localhost/thunderhead/auth/mockkey/cloudera/upriyadarshan@cloudera.com)\n\nWhatever values you previously picked when you logged into the UI (https://localhost/) the first time, the same ones shall be also included as TENANT and EMAIL in the above URL. The response will contain the access key for CDP CLI when targeting local CB.\n\nTechnically, TENANT and EMAIL are used to construct a mock user CRN:\n\ncrn:cdp:iam:us-west-1:TENANT:user:EMAIL\n\nthe user crn is encoded with base64, and that becomes the apikeyid (MOCK ACCESS KEY). The privatekey returned in the above mock endpoint is a fixed string \"nHk...d2g=\" that is otherwise completely ignored in CB for authentication.\n\nIntroduce a new Profile section named local in ~/.cdp/config:\n\n```\n[profile local]\nform_factor = public\nendpoint_url = http://localhost\ncdp_endpoint_url = http://localhost\n```\n\nand provide credentials for this profile in ~/.cdp/credentials:\n\n```\n[local]\ncdp_access_key_id = YOUR_MOCK_ACCESS_KEY\ncdp_private_key = YOUR_MOCK_PRIVATE_KEY\n```\n\nwhere YOUR_MOCK_ACCESS_KEY is the value generated with the above mock endpoint URL.\n\nOnce configured, `local` can be used as profile for all CDP commands and the command will target the Configured Endpoint (on localhost) along with credentials.\n\n### Create AWS Access Keys\n\nTo create an AWS access key and secret, follow these steps:\n\n1. Sign in to the AWS Management Console : Use your credentials to log in.\n2. Navigate to the IAM Console : Go to the IAM Management Console and select \"Users\" from the sidebar.\n3. Select a User : Click on the user for whom you want to create the access key.\n4. Create Access Key : Under the \"Security credentials\" tab, click \"Create access key.\"\n\n* Follow the prompts to generate the key.\n\n1. Save the Keys : Once created, you'll see the access key ID and secret access key. Save these securely, as the secret key will not be shown again.\n\n### Configure CBD Profile\n\nUpdate `cbd-local\\Profile` file with following environment variables with the AWS Access key and secret obtained from AWS:\n\n```\nexport CB_AWS_ACCOUNT_ID=\"YOUR_AWS_ACCOUNT_ID\"\nexport AWS_ACCESS_KEY_ID=\"YOUR_ACCESS_KEY\"\nexport AWS_SECRET_ACCESS_KEY=\"YOUR_SECRET_KEY\"\n```\n\n### Create External ID credentials in AWS\n\n1. Create a policy suitable for cross account access: Since cloudera management control panel uses Customer's cloud providers account it is important to establish this cross account access between and this policy is pre-requisite for things to work.\n   The policy is minimal, it can be found in the source cloudbreak/cloud-aws-common/src/main/resources/definitions/\n\n   [aws-environment-minimal-policy.json](https://github.infra.cloudera.com/cloudbreak/cloudbreak/blob/master/cloud-aws-common/src/main/resources/definitions/aws-environment-minimal-policy.json) \u003c-- for AWS commerial Cloud (Preferred for Dev)\n\n   [aws-gov-environment-minimal-policy.json](https://github.infra.cloudera.com/cloudbreak/cloudbreak/blob/master/cloud-aws-common/src/main/resources/definitions/aws-gov-environment-minimal-policy.json) \u003c-- for AWS GOV Cloud\n\n2. Create an AWS IAM External Access Role And Credential in Cloudera Manageement Console:\n\n   Open Cloudera managment console on http://localhost : select Shared Resources -\u003e Credentials.\n   Provide the credential name and copy the External ID\n\n   Open AWS console open IAM -\u003e roles: create new Role\n   Choose AWS Account ...\n   in Options: Require External ID copy the External ID copied from Cloudera Credentials\n   Add the following policies to this role\n\n   ```\n   Cross Account Access Policy (defined in the previous step)\n   ```\n\nCopy the ARN of this new Role and copy it in the Cloudera Console's Credentials and save it.\n\nNOTE: this credential's name is used as `--credential-name` in CDP commands.\n\n\n# Test the environment terminal\n\n```\ncdp environments list-environments --profile local\n```\n\n# Reference\n\n[https://cloudera.atlassian.net/wiki/spaces/ENG/pages/10177871890/SDX-Infra+Developer+Onboarding#Setup-Cloudbreak-deployer](https://cloudera.atlassian.net/wiki/spaces/ENG/pages/10177871890/SDX-Infra+Developer+Onboarding#Setup-Cloudbreak-deployer)\n\n[https://github.infra.cloudera.com/cloudbreak/cloudbreak#cloudbreak-deployer](https://github.infra.cloudera.com/cloudbreak/cloudbreak#cloudbreak-deployer)\n\n[https://github.infra.cloudera.com/cloudbreak/cloudbreak/blob/master/README.md](https://github.infra.cloudera.com/cloudbreak/cloudbreak/blob/master/README.md)\n","funding_links":[],"categories":["Java","大数据"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhortonworks%2Fcloudbreak","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhortonworks%2Fcloudbreak","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhortonworks%2Fcloudbreak/lists"}