{"id":15445264,"url":"https://github.com/jhermann/artifactory-debian","last_synced_at":"2025-04-19T20:41:51.735Z","repository":{"id":10271125,"uuid":"12384455","full_name":"jhermann/artifactory-debian","owner":"jhermann","description":":wrench: :package: Tools to use Artifactory as a Debian (APT) repository, and dput DEB packages into it, or upload them to Bintray.","archived":false,"fork":false,"pushed_at":"2017-07-07T10:13:48.000Z","size":395,"stargazers_count":45,"open_issues_count":1,"forks_count":5,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-11-28T23:52:00.298Z","etag":null,"topics":["artifactory","bintray","cli-utilities","debian","debian-packages","debian-repositories","dput","plugin","python","release-automation","webdav"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jhermann.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-08-26T16:50:37.000Z","updated_at":"2024-07-17T02:49:55.000Z","dependencies_parsed_at":"2022-08-31T11:22:40.759Z","dependency_job_id":null,"html_url":"https://github.com/jhermann/artifactory-debian","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhermann%2Fartifactory-debian","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhermann%2Fartifactory-debian/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhermann%2Fartifactory-debian/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jhermann%2Fartifactory-debian/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jhermann","download_url":"https://codeload.github.com/jhermann/artifactory-debian/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229097389,"owners_count":18019735,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artifactory","bintray","cli-utilities","debian","debian-packages","debian-repositories","dput","plugin","python","release-automation","webdav"],"created_at":"2024-10-01T19:44:29.759Z","updated_at":"2024-12-10T17:03:25.557Z","avatar_url":"https://github.com/jhermann.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ![Logo](https://raw.github.com/jhermann/artifactory-debian/master/doc/_static/artifactory-debian-logo.png) Debian Repositories in Artifactory\n\n`artifactory-debian` offers tools to use [Artifactory](http://www.jfrog.com/) as a Debian (APT) repository,\nand conveniently [upload DEB packages](#package-uploading) to *Artifactory* or *Bintray*. Also see [the wiki](https://github.com/jhermann/artifactory-debian/wiki).\n\n| **dput-webdav** | **Open HUB** |\n|:-------------:|:-------------:|\n| [![Travis Status](https://travis-ci.org/jhermann/artifactory-debian.png?branch=master)](https://travis-ci.org/jhermann/artifactory-debian)  [![Download](https://api.bintray.com/packages/jhermann/deb/dput-webdav/images/download.svg) ](https://bintray.com/jhermann/deb/dput-webdav/_latestVersion) | [![Ohloh stats](https://www.ohloh.net/p/artifactory-debian/widgets/project_thin_badge.gif)](https://www.ohloh.net/p/artifactory-debian) |\n\n**Table of Contents**\n- [Motivation and Overview](#motivation-and-overview)\n- [Repository Setup](#repository-setup)\n\t- [Artifactory Configuration](#artifactory-configuration)\n\t- [Indexing Host Configuration](#indexing-host-configuration)\n- [Installing Packages from Artifactory Repositories](#installing-packages-from-artifactory-repositories)\n- [Package Uploading](#package-uploading)\n\t- ['webdav' Upload Method for 'dput'](#webdav-upload-method-for-dput)\n\t- [Installing the 'webdav' Plugin](#installing-the-webdav-plugin)\n\t- ['webdav' Plugin Configuration](#webdav-plugin-configuration)\n\t- [Uploading to Bintray](#uploading-to-bintray)\n- [Acknowledgements](#acknowledgements)\n\n\n## Motivation and Overview\n\nPrinciples of *Continuous Delivery* mandate that you propagate a binary artifact,\nonce it's built, unchanged through the different quality gates and deployment stages of\nyour delivery pipeline. The need for keeping them in ideally a single place becomes obvious\npretty fast, together with similar build artifacts like Java JARs.\n\nArtifactory is a repository server for binaries that can provide such a place, and offers the\nadditional advantage of attribute management on top of storing the contained files.\nWith that you can for example add cryptographic signatures of passed quality gates and the like,\nwhen a candidate release progresses through the pipeline.\n\n:mega: | Starting with version 3.3, Artifactory can handle Debian repositories natively (see [RTFACT-4613](https://www.jfrog.com/jira/browse/RTFACT-4613)). This project enabled you to manage your Debian packages within Artifactory before that, and still provides the `dput` plugin for easy uploading to a repository using the standard Debian tool chain.\n----: | :----\n\nThe following diagram shows a typical setup and how the components interact.\nWhen a package maintainer uploads to Artifactory using `dput`,\na `post_upload_command` remotely triggers a Jenkins job that pulls\nthe repository configuration (from a local SCM) and the indexing code (from GitHub).\nThat job then scans the available repositories using a read-only `davfs2` mount,\ncreates new index files, and finally uploads those back into Artifactory.\nUsers can then download the index files via `apt-get update` and install available packages as usual,\nwithout realizing they're accessing an Artifactory server, except for the specific `apt` source definition syntax\n(for details see [Installing Packages from Artifactory Repositories](#installing-packages-from-artifactory-repositories)).\n\n![Configuration \u0026 Data Flow](https://raw.github.com/jhermann/artifactory-debian/master/doc/_static/data-flow.png)\n\nIn the Artifactory web interface, the final result after following the setup instructions below will look like this…\n\n![Sample screenshot of a working repository](https://raw.github.com/jhermann/artifactory-debian/master/doc/_static/artifactory-repo-browser.png)\n\n\n## Repository Setup\n\nDetailed information about the initial repository setup can be found at\n[Configuration of Artifactory and Repository Indexing](https://github.com/jhermann/artifactory-debian/wiki/Configuration-of-Artifactory-and-Repository-Indexing).\nWhat follows is a shorter, no-nonsense version.\n\n\n### Artifactory Configuration\n\nTo create your Debian repositories in Artifactory, follow these steps:\n\n* Login with an administrator account.\n* Add a *Repository Layout* named `debian-default` with this *Artifact Path Pattern*:\n\n```sh\n[orgPath]/[module]/[baseRev]/[module]-[baseRev](-[patchLevel\u003c[.~0-9a-zA-Z]+\u003e])(_[archTag\u003c\\w+\u003e]).[ext]\n```\n\n* Set the layout's *Folder/File Integration Revision RegExp* fields to `.*`, and save it.\n* Create a new Artifactory repository named `debian-local` using the new layout.\n\nNow you can instantly start to upload packages into `debian-local`.\n\n\n### Indexing Host Configuration\n\nIn order for `apt-get` to find packages, index data needs to be generated (what `apt-get update` downloads).\nThis is what the script `deb-index.sh` does; the script and a configuration example can be found in the\n[indexing](https://github.com/jhermann/artifactory-debian/tree/master/indexing) directory.\n\nYou can use any host that has network access to your Artifactory server for indexing, and\nrun the index task via either a crontab entry, or as a job on a continuous integration server.\nThis describes a Jenkins setup, for using `cron` just adapt the Jenkins configuration steps accordingly.\n\n**On your workstation**\n\n* Describe your repositories in the `apt-ftparchive.conf` and `repo-«reponame».conf` configuration files;\nsee the provided examples and `man apt-ftparchive` for details. Commit them to your local version control.\n\n\n**On a Jenkins slave**\n\n* Open a shell session on the indexing host, and copy or clone this git repository.\n* Call the script with the `setup` argument like this:\n\n```sh\nsudo ./deb-index.sh setup \"http://repo.example.com/artifactory/\"\n```\n\n* When your configured editor pops up with `/etc/davfs2/secrets`, fill in Artifactory credentials for read-only access.\n\n\n**In the Jenkins web interface**\n\n* Install the `EnvInject` Jenkins plugin, if you don't already have it.\n* Create a Jenkins job bound to the slave where you called `deb-index.sh setup`:\n  * Set the workspace checkout location to the configuration files you just committed.\n  * Add the `ARTIFACTORY_CREDENTIALS` environment variable with an account (`user:pwd`) having uploading permission, at the *Inject passwords to the build as environment variables* setting of *Build Environment*.\n  * Add a *Shell Build Step* like follows:\n\n```sh\nexport ARTIFACTORY_URL=\"http://repo.example.com/artifactory/\"\n\ntest -d artifactory-debian \\\n    \u0026\u0026 ( cd artifactory-debian \u0026\u0026 git pull --ff-only ) \\\n    || git clone \"https://github.com/jhermann/artifactory-debian.git\"\n\nartifactory-debian/indexing/deb-index.sh refresh\n```\n\n* Start the job via *Build Now*.\n* Check in the Jenkins log and Artifactory web interface that the index files were generated and uploaded (cf. above screenshot).\n\nAnd you're now ready to use your shiny new toy…\n\n\n## Installing Packages from Artifactory Repositories\n\nThe resulting repositories can be added to a machine like this:\n\n```sh\necho \"deb http://repo.example.com/artifactory/debian-local noplat/\" \\\n    \u003e/etc/apt/sources.list.d/artifactory-noplat.list\napt-get update\n```\n\nThen to give it a spin, try to list some packages only found in your new repository, using `apt-cache search`.\nOr simply install a package via `apt-get`.\n\n\n## Package Uploading\n\n### 'webdav' Upload Method for 'dput'\n\n`dput` conveniently provides a plugin directory for uploading methods,\nso we just use that to add Artifactory support.\nThe `http` plugin of `dput` could be used to PUT packages into Artifactory,\nbut is thoroughly broken (at least the one I get on Ubuntu 12.04).\n\nBesides that, it lacks some features I consider essential, so I decided to write a new `webdav` plugin\nspecifically with Artifactory in mind. It tries to be compatible to the `http` plugin,\nand offers the following new features:\n* `login` credentials can include the password.\n* `incoming` can be a full URL, also containing dynamic (package) metadata.\n* HTTPS support is provided by simply using a `https://…` URL.\n* Some extended features like mapping distributions to repository names.\n\nSupport for [dput-ng](http://people.debian.org/~paultag/dput-ng/) might be a good idea, when that one gets more traction\n(didn't even know about it before I searched for existing `dput` plugins).\n\n\n### Installing the 'webdav' Plugin\n\n**Package Installation**\n\nTo install a **release version** via adding [Bintray](https://bintray.com/jhermann/deb/dput-webdav) as a package source, run these commands as `root`:\n\n```sh\necho \"deb http://dl.bintray.com/jhermann/deb /\" \\\n    \u003e/etc/apt/sources.list.d/bintray-jhermann.list\napt-get update\napt-get install -o \"APT::Get::AllowUnauthenticated=yes\" dput-webdav\n```\n\nIf you need to build from source, change into the ``dput-webdav`` directory\nof this project, and call ``dpkg-buildpackage -uc -us``.\nThe package files will be created in the project root, use ``dpkg -i …`` to install.\n\n\n**Manual Installation**\n\nIf for some reason you can't use a packaged installation,\nor have a version of `dput` other than `0.9.6` installed,\ncopy the plugin from GitHub using this command:\n\n```sh\nsudo bash -c \"umask 0133; curl -skSL -o /usr/share/dput/webdav.py \\\n    https://raw.github.com/jhermann/artifactory-debian/master/dput-webdav/webdav.py\"\n```\n\n**Other Installation Options**\n\nFor an in-depth discussion of options, see the\n[Detailed Install Instructions](https://github.com/jhermann/artifactory-debian/wiki/Detailed-Install-Instructions)\nwiki page.\n\n\n### 'webdav' Plugin Configuration\n\nYour `~/.dput.cf` needs a section describing your Artifactory service,\nand the upload credentials are placed in an extra file, for easier permission management.\nSo all you need is to create two files:\n\n* Edit `~/.dput.cf` to include the following snippet (with the appropriate `fqdn`):\n\n```ini\n[DEFAULT]\ndefault_host_main = artifactory-debian\nprogress_indicator = 2\n\n[artifactory-debian]\nmethod = webdav\nfqdn = repo.example.com:80\nlogin = file:~/.artifactory.credentials\nincoming = http://{fqdn}/artifactory/debian-local/{repo}/{source}/{upstream}/#mindepth=3\u0026overwrite=0\nallow_unsigned_uploads = 1\n#run_lintian = 1\n#check_version = 1\n# repo_mappings = unstable=snapshots *-experimental=snapshots *=incoming\n\n# trigger Jenkins reindex job after successful upload\n#post_upload_command = curl -k \"\u003cJENKINS_URL\u003e/job/artifactory-debian-reindex/build?token=DOIT\u0026cause=dput+upload\"\n```\n\n* Call `⍽ echo -n \"«username»:«password»\" \u003e~/.artifactory.credentials; chmod 600 ~/.artifactory.credentials` with your credentials filled in (put a space in front to exclude the command from shell history).\n\n  It's recommended to use your *“Encrypted Password”* instead of the cleartext password. That password starts with ``AP`` and is revealed after you **unlock** the *“User Profile”* page in the web UI.\n\n  Of course, you can also use the ``~/.netrc`` file, instead of a specific ``~/.artifactory.credentials`` one, as outlined in the next section.\n\nThe above `~/.dput.cf` works with the indexing solution contained in this project.\nIf by now you use the *built-in* Debian repository type of Artifactory,\nremove or comment the `post_upload_command` (it's not longer needed, indexing is automatic),\nand change the `incoming` value as follows:\n\n```ini\nincoming = http://{fqdn}/artifactory/debian-local/pool/{source}/{upstream};deb.architecture={deb_architecture};deb.component=local;deb.distribution={repo}#mindepth=3\u0026overwrite=0\n```\n\nReplace the `debian-local` path component if you named your repository differently.\n\nTo fully understand the `dput` WebDAV plugin configuration and be able to customize it,\nread [WebDAV Plugin Configuration](https://github.com/jhermann/artifactory-debian/wiki/WebDAV-Plugin-Configuration).\nAlso refer to `man dput.cf` for the common configuration options shared by all upload methods.\n\n\n### Uploading to Bintray\n\nTo use the `webdav` plugin for uploads to [Bintray](https://bintray.com/),\nadd this configuration to your workstation's user account:\n\n* Extend your `~/.dput.cf` with this snippet:\n\n```ini\n[bintray]\nmethod = webdav\nfqdn = api.bintray.com\nlogin = netrc:\nincoming = https://{fqdn}/content/{loginuser}/deb/{source}/{upstream}/#mindepth=0\u0026overwrite=1\nallow_unsigned_uploads = 1\n```\n\n* Put your login name and API key into `~/.netrc` (and don't forget to `chmod 600` that file); the API key you'll find in [your profile](https://bintray.com/profile/edit) when you click on `API Key` in the sidebar menu:\n\n```\nmachine api.bintray.com\n    login YOURUSERNAME\n    password 00...YOURAPIKEY...ff\n```\n\n\nAs an example, the following is the log of the first release, where `dput-webdav` uploaded itself:\n\n```console\n$ dput bintray dput-webdav*changes\nUploading to bintray (via webdav to api.bintray.com):\n  Uploading dput-webdav_1.0.dsc:  done.\n  Uploading dput-webdav_1.0.tar.gz: / done.\n  Uploading dput-webdav_1.0_all.deb: / done.\n  Uploading dput-webdav_1.0_amd64.changes: / done.\nSuccessfully uploaded packages.\n```\n\n\n## Acknowledgements\n\nThanks to…\n* the authors of `dput`, `apt-ftparchive`, `davfs2`, `curl`, and `bash`.\n* [Inkscape](http://inkscape.org/) and [Shutter](https://en.wikipedia.org/wiki/Shutter_%28software%29) for eye candy.\n* [DocToc](http://doctoc.herokuapp.com/) and Thorsten Lorenz for easy TOC maintenance.\n* [1\u00261](https://github.com/1and1) for free ☕ ☕ ☕ and ⌛.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhermann%2Fartifactory-debian","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjhermann%2Fartifactory-debian","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjhermann%2Fartifactory-debian/lists"}