{"id":13450890,"url":"https://github.com/bazeltools/bazel-deps","last_synced_at":"2025-03-23T16:32:37.140Z","repository":{"id":9343948,"uuid":"61169687","full_name":"bazeltools/bazel-deps","owner":"bazeltools","description":"Generate bazel dependencies for maven artifacts ","archived":false,"fork":false,"pushed_at":"2024-06-04T17:46:27.000Z","size":979,"stargazers_count":249,"open_issues_count":101,"forks_count":121,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-07-31T07:15:53.409Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bazeltools.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-06-15T02:02:06.000Z","updated_at":"2024-07-31T07:16:04.730Z","dependencies_parsed_at":"2024-01-16T03:46:28.432Z","dependency_job_id":"3f9d5f42-ad69-4c8c-829b-909c1d8593cb","html_url":"https://github.com/bazeltools/bazel-deps","commit_stats":null,"previous_names":["johnynek/bazel-deps"],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazeltools%2Fbazel-deps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazeltools%2Fbazel-deps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazeltools%2Fbazel-deps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazeltools%2Fbazel-deps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bazeltools","download_url":"https://codeload.github.com/bazeltools/bazel-deps/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221856498,"owners_count":16892454,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T07:00:39.888Z","updated_at":"2024-10-28T16:32:07.540Z","avatar_url":"https://github.com/bazeltools.png","language":"Scala","readme":"# bazel-deps\n\nGenerate [bazel](https://bazel.build/) dependencies transitively for maven artifacts, with scala\nsupport.\n\n## Quickstart\n\nThis repo can be cloned and built locally, or you can download pre-build binaries for MacOS and Linux in the releases page. Automatic releases are generated for every commit against master.\nWe also include a bash script in the releases which will let you easily download/run on mac/linux a default configuration for running bazel-deps.\n\nA flow like:\n1) Download the bash script paired with the release, it has the expected per platform sha256's embedded in it\n2) Place in your repo and `chmod +x update_dependencies.sh`, maybe in a scripts folder if you wish.\n3) Copy the `dependencies.yaml` from this repo, or write your own\n4) Run the script, it should produce some files in `3rdparty`\n5) Add to your workspace:\n```python\nload(\"//3rdparty:workspace.bzl\", \"maven_dependencies\")\nmaven_dependencies()\nload(\"//3rdparty:target_file.bzl\", \"build_external_workspace\")\nbuild_external_workspace(name = \"third_party\")\n```\n6) You can now add dependencies to your `BUILD` files as described [below](#alternate-outputs-external-repo).\neg `deps = [\"@third_party//3rdparty/jvm/org/scalaj:scala_http\"]`.\n\n## Usage\n\nFirst, list all of your maven artifact dependencies in a [Dependencies](#dependencies) file.\n\nNext, run parseproject on your project yaml file. This will create a tree of BUILD files that\nmatch the maven group id, and the artifact id will be a label in a BUILD file. You should not\nedit these by hand, and instead have a separate directory for any exceptions that you manage\nalong with [Replacements](#replacements). For instance, this project is setup with:\n\n```bash\n./bazel run //:parse -- generate -r `pwd` -s 3rdparty/workspace.bzl -d dependencies.yaml\n```\n\nWe give three arguments: the path to the file we will include in our workspace. The path to the root\nof our bazel repo. The path to the dependencies file. You can also run with `--help`.\n\nThen you should add\n```\nload(\"//3rdparty:workspace.bzl\", \"maven_dependencies\")\n\nmaven_dependencies()\n```\nto your workspace to load the maven dependencies.\n\nFor example, if your project is located at `MY_PROJ_DIR`, your dependencies file is at\n`MY_PROJ_DIR/dependencies.yaml`, and your checkout of bazel-deps is at `BAZEL_DEPS`, to generate the\ndependencies you need to do the following:\n\n```bash\ncd $BAZEL_DEPS\n./bazel run //:parse generate -- --repo-root \"$MY_PROJ_DIR\" --sha-file 3rdparty/workspace.bzl --deps dependencies.yaml\n```\n\nThe final result in `MY_PROJ_DIR` will look like this\n\n```\nMY_PROJ_DIR\n├── 3rdparty             \u003c-- everything under here is generated by running gen_maven_deps.sh\n│   ├── workspace.bzl        \u003c-- load() this from main WORKSPACE\n│   └── jvm/                 \u003c-- generated BUILD files in this directory.\n├── BUILD\n├── WORKSPACE\n└── dependencies.yaml    \u003c-- your project's dependencies are declared in here.\n```\n\nWhenever you update the dependencies declared in dependencies.yaml you will need to regenerate the\ncontents of the `3rdparty` directory by re-running `$BAZEL_DEPS/gen_maven_deps.sh generate`.\n\n### CI integration\nIn a CI, you will often want to make sure there is alignment between the configuration file for bazel-deps\nand the resulting generated files or directories, you can run `generate` with `--check-only` and it will\ncheck that each file matches bit-for-bit, but does not generate. If something does not match what would have\nbeen generated, you get a non-zero return value and a list of the mismatches logged to error.\n\n### Alternate outputs, external repo\nBazel-deps can also prepare the outputs, not as a file tree but an external repo. With this one would refer to targets as\n`@third_party//foo:bar` rather than `//3rdparty/jvm/foo/bar`. This is useful if you do not want to check in generated code to your\nrepo. Also if multiple repos are depending upon one another and using bazel deps this can avoid broken transitive dependencies.\nThat is if there are two repos `A` and `B` where `B` depends on `A`:\nwhere A has `Foo 1.0 dependson Jackson27`\nand B has `Foo 2.0 depends on circe`\nwith the checked in version both will compile from source against the local copy of Foo, but transitively on the classpath\nin the repo `B` `Jackson27` will be on the classpath rather than `circe`.\n\nTo use this option you would execute bazel-deps like:\n ```bash\n cd $BAZEL_DEPS\n ./bazel run //:parse generate -- --repo-root \"$MY_PROJ_DIR\" --sha-file 3rdparty/workspace.bzl --deps dependencies.yaml --target-file 3rdparty/target_file.bzl --disable-3rdparty-in-repo\n```\n\nIn your `dependencies.yaml` file you will likely want:\n`thirdPartyDirectory: \"\"`\nto avoid prefixing the remote repo path with `3rdparty/jvm`.\n\nAnd finally to load it from your `WORKSPACE` you would use:\n```\nload(\"//3rdparty:target_file.bzl\", \"build_external_workspace\")\n\nbuild_external_workspace(name = \"third_party\")\n```\n\n### Customized integration\nIf you want to fully control how you create your third party dependencies, you can use bazel deps simply\nto normalize all the jars into a single canonical version for each artifact and present a json lock file\nwhich has the hashes and dependencies of each artifact. To do this you would do:\n```\n./bazel run //:parse -- generate -r `pwd` -d dependencies.yaml --resolved-output lock.json\n```\nThe schema of the lock file should be rather obvious and it has all the information you would need.\n\n## Assumptions and usage\nThis tool will generate one canonical version for every jar in the transitive dependencies of\nthe root dependencies declared. You have three conflict resolution modes currently (which currently\napply globally):\n\n- fail: if more than one version is found transitively, fail.\n- fixed: for all artifacts explicitly added, use that version, otherwise fail if any other artifact has multiple versions.\n- highest: for all artifacts explicitly added, use that version, otherwise take the highest version.\n\nIn any case, we add a comment for any duplicates found in the workspace loading file.\n\nTo declare dependencies, add items to the `dependencies` key in your yaml file. The format\nshould be yaml or json. It should have [`dependencies`](#dependencies) and it may have [`replacements`](#replacements)\nand [`options`](#options). Important: only dependencies explicitly named have public visibility,\ntransitive dependencies not listed in the dependencies file have visibility limited to the third\nparty directory.\n\n### \u003ca name=\"dependencies\"\u003eDependencies\u003c/a\u003e\n\nDependencies are a map from maven group id to artifact id, with some metadata, such as:\n```yaml\ndependencies:\n  com.google.guava:\n    guava:\n      version: \"18.0\"\n      lang: java\n```\nLanguage is always required and may be one of `java, scala, scala/unmangled`. To control the scala\nversion, see the [Options section](#options). A common case are projects with many modules. For instance in\nthe [scalding project](https://github.com/twitter/scalding) there are many modules: `-core, -date,\n-args, -db, -avro` to name a few. To reduce duplication you can do:\n\n```yaml\ndependencies:\n  com.twitter:\n    scalding:\n      version: 0.16.0\n      lang: scala\n      modules: [core, date, args, db, arvo]\n```\nThe `version` field is optional. If it is absent, it means this jar is expected to be found by\ntransitive dependencies, and it is available to be used outside of the thirdparty directory, but the\nexact version used can be selected according to the version resolution rules. It is an error to have\nan unversioned dependency that is not a transitive dependency of another versioned dependency.\n\nA target may optionally add `exports` and `exclude` lists to a dependency. `exports` should be just the group and\nartifact (such as: `com.twitter:scalding-core` in the above), and they should be listed in the dependencies. `exclude`\nlist should also be only the group and artifact.\n\nIt's possible to add `generateNeverlink` option to a dependency, which will make the generator to generate this dependency twice:\n1. With the normalized name as usual.\n2. With the name `${normalized}_neverlink` and neverlink is set as true.\nThis option should be used only for java dependencies, it will be ignored in any other lang.\n\nEach group id can only appear once, so you should collocate dependencies by group. WARNING the parsing library\nwe are using does not fail on duplicate keys, it just takes the last one, so watch out. It would be good\nto fix that, but writing a new yaml parser is out of scope.\n\n#### \u003ca name=\"packaging-classifiers\"\u003ePackaging and Classifiers\u003c/a\u003e\n\nDepending on artifacts with classifiers is straightforward: just add the packaging and classifier as part of the\nartifact id:\n\n```yaml\ndependencies:\n  net.sf.json-lib:\n    json-lib:jar:jdk15: # artifact:packaging:classifier\n      lang: java\n      version: \"2.4\"\n```\n\n**Note**: Currently, only `jar` packaging is supported for dependencies. More work is needed on the `bazel-deps` backend\nto ensure that non-jar dependencies are written as `data` attributes, instead of regular jar dependencies.\n\nExcluding artifacts with packaging or classifiers is similar to including dependencies. Non-jar packaging _is_ supported\nfor `exclude`.\n\n```yaml\n  com.amazonaws:\n    DynamoDBLocal:\n      lang: java\n      version: \"1.11.86\"\n      exclude:\n        - \"com.almworks.sqlite4java:sqlite4java-win32-x86:dll\"\n        - \"com.almworks.sqlite4java:sqlite4java-win32-x64:dll\"\n        - \"com.almworks.sqlite4java:libsqlite4java-osx:dylib\"\n        - \"com.almworks.sqlite4java:libsqlite4java-linux-i386:so\"\n        - \"com.almworks.sqlite4java:libsqlite4java-linux-amd64:so\"\n```\n\n#### \u003ca name=\"annotation-processors\"\u003eAnnotation Processors (`processorClasses`)\u003c/a\u003e\n\nA target may also optionally add `processorClasses` to a dependency. This is for [annotation processors](https://docs.oracle.com/javase/8/docs/api/javax/annotation/processing/Processor.html).\n`bazel-deps` will generate a `java_library` and a `java_plugin` for each annotation processor defined. For example, we can define Google's auto-value annotation processor via:\n```yaml\ndependencies:\n  com.google.auto.value:\n    auto-value:\n      version: \"1.5\"\n      lang: java\n      processorClasses: [\"com.google.auto.value.processor.AutoValueProcessor\"]\n```\nThis will yield the following:\n```python\njava_library(\n    name = \"auto_value\",\n    exported_plugins = [\n        \":auto_value_plugin\",\n    ],\n    visibility = [\n        \"//visibility:public\",\n    ],\n    exports = [\n        \"//external:jar/com/google/auto/value/auto_value\",\n    ],\n)\n\njava_plugin(\n    name = \"auto_value_plugin\",\n    processor_class = \"com.google.auto.value.processor.AutoValueProcessor\",\n    deps = [\n        \"//external:jar/com/google/auto/value/auto_value\",\n    ],\n)\n```\nIf there is only a single `processorClasses` defined, the `java_plugin` rule is named `\u003cjava_library_name\u003e_plugin`. If there are multiple\n`processorClasses` defined, each one is named `\u003cjava_library_name\u003e_plugin_\u003cprocessor_class_to_snake_case\u003e`.\n\n### \u003ca name=\"options\"\u003eOptions\u003c/a\u003e\nThere are a number of ways to customize the generated build files. These are controlled\nby the `options` dictionary at the root-level of the dependencies file. This is a list of\nall of the supported options.\n\n* buildHeader: usually you will want to configure your scala support here:\n```yaml\n  buildHeader:\n    - load(\"@io_bazel_rules_scala//scala:scala_import.bzl\", \"scala_import\")\n```\n* languages: an array of languages to be supported either Java or a specific version of Scala, e.g. `[ \"java\", \"scala:2.12.8\" ]`.\n* thirdPartyDirectory: path to where we write the BUILD files for thirdparty. The default is `3rdparty/jvm`. If you choose the Google default of `third_party` you will need to configure the `licenses` option as well.\n* versionConflictPolicy: `fixed`, `fail` or `highest`\n* transitivity: `runtime_deps` or `exports`\n* resolvers: the maven servers to use. Each resolver is defined by three keys, an \"id\", a \"type\", and a \"url\".\n```yaml\n  resolvers:\n    - id: \"mavencentral\"\n      type: \"default\"\n      url: https://repo.maven.apache.org/maven2/\n    - id: \"myserver\"\n      type: \"default\"\n      url: https://my.private.maven.server.com/mvn/\n```\n* resolverCache: (with `resolverType: aether`) where bazel-deps should cache resolved packages.  `local` (`target/local-repo` in the repository root)\n  or `bazel_output_base` (`bazel-deps/local-repo` inside the repository's Bazel output base -- from `bazel info\n  output_base`).  Coursier ignores this option and uses `~/.cache/coursier`.\n* namePrefix: a string added to the generated workspace names, to avoid conflicts.  The external repository names and\n  binding targets of each dependency are prefixed.\n* strictVisibility: this is enabled by default, when enabled a target must be explicitly declared in the\n  `dependencies.yaml` file or it will not be visible to the rest of the workspace. If it is set to `false` all targets\n  will be generated with `public` visibility.\n* licenses: a set of strings added a licenses rule to each generated bazel target.  Required by\n  bazel if your build targets are under `third_party/`. See the [licenses](https://docs.bazel.build/versions/master/be/functions.html#licenses) function in Bazel.\n* resolverType: `aether` or `coursier`. Note that `aether` is slower and seems to silently miss some dependencies for\n  reasons we don't yet understand.\n* buildFileName: filename of the generated build files\n\nIn the default case, with no options given, we use:\n- allow java and scala `2.11`\n- `3rdparty/jvm` as the `thirdPartyDirectory`.\n- `highest` versionConflictPolicy\n- `exports` transitivity\n- use maven central as the resolver\n- `local` resolverCache\n- empty namePrefix (`\"\"`)\n- `coursier` resolverType\n- `BUILD` as build file name\n\n### \u003ca name=\"replacements\"\u003eReplacements\u003c/a\u003e\nSome maven jars should not be used and instead are replaced by internal targets. Here are\nsome examples of this:\n\n1. A subproject in the repo is published as a maven artifact (`A`). Others (`B`) depend on this artifact (`B -\u003e A`) and in turn we depend on those (we have added `B` to our dependencies file). We don't want to pull `A` from a maven repo, since we build it internally, so we replace that artifact with an internal target.\n2. We get some scala artifacts directly from the sdk. So, if a jar says it needs `org.scala-lang:scala-library` we already have that (and a few other jars) declared, and we don't want to risk having two potentially incompatible versions.\n3. A small external project has both a bazel build and a maven publishing. We prefer to use the bazel build so we can easily pull more recent versions by bumping up a gitsha rather than waiting for jar to be published.\n\nThe replacements work on the level of artifacts. An artifact is replaced one-for-one with a local\nbazel target. For instance:\n```yaml\nreplacements:\n  org.scala-lang:\n    scala-library:\n      lang: scala/unmangled # scala-library is not mangled like sbt does with other jars\n      target: \"@io_bazel_rules_scala_scala_library\"\n    scala-reflect:\n      lang: scala/unmangled\n      target: \"@io_bazel_rules_scala_scala_reflect\"\n```\n\nIn this way, we redirect maven deps to those providers.\n\nNote, we stop walking the graph when we see a replaced node, so the replacement target is now\nresponsible for building correctly, and correctly exporting any dependencies that need to be\non the compile classpath.\n\n## Code\nThis code was originally forked from [pgr0ss/bazel-deps](https://github.com/pgr0ss/bazel-deps)\n\nThis code was inspired by the [aether examples](https://github.com/eclipse/aether-demo/blob/322fa556494335faaf3ad3b7dbe8f89aaaf6222d/aether-demo-snippets/src/main/java/org/eclipse/aether/examples/GetDependencyTree.java) for walking maven dependencies.\n","funding_links":[],"categories":["Scala","Tooling"],"sub_categories":["Migration"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbazeltools%2Fbazel-deps","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbazeltools%2Fbazel-deps","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbazeltools%2Fbazel-deps/lists"}