{"id":21452273,"url":"https://github.com/packit/dist-git-to-source-git","last_synced_at":"2025-07-14T22:31:00.304Z","repository":{"id":40431785,"uuid":"248803381","full_name":"packit/dist-git-to-source-git","owner":"packit","description":"Converting dist-git to source-git","archived":false,"fork":false,"pushed_at":"2024-10-21T23:07:27.000Z","size":545,"stargazers_count":4,"open_issues_count":40,"forks_count":9,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-10-22T19:43:36.801Z","etag":null,"topics":["hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/packit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-20T16:33:28.000Z","updated_at":"2023-10-31T09:44:46.000Z","dependencies_parsed_at":"2023-02-16T15:35:32.449Z","dependency_job_id":"76921c8d-35b2-4029-a434-77eebfe1254a","html_url":"https://github.com/packit/dist-git-to-source-git","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/packit%2Fdist-git-to-source-git","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/packit%2Fdist-git-to-source-git/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/packit%2Fdist-git-to-source-git/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/packit%2Fdist-git-to-source-git/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/packit","download_url":"https://codeload.github.com/packit/dist-git-to-source-git/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226001596,"owners_count":17558007,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest"],"created_at":"2024-11-23T04:28:36.878Z","updated_at":"2024-11-23T04:28:37.528Z","avatar_url":"https://github.com/packit.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://zuul-ci.org/gated.svg)](https://softwarefactory-project.io/zuul/t/local/builds?project=packit-service/dist2src)\n[![black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)](https://github.com/pre-commit/pre-commit)\n[![Docker Repository on Quay](https://quay.io/repository/packit/dist2src/status \"Docker Repository on Quay\")](https://quay.io/repository/packit/dist2src)\n\nThis project is no longer being developed.\n\n[Please read details about the decommissioning.](https://github.com/packit/source-git-onboarding#decommissioning)\n\n# Converting dist-git to source-git\n\nThis should be similar to [How to source-git?], except that it isn't :)\n\nThe main difference is that [How to source-git?] starts with the upstream\nhistory and applies the patches from dist-git.\n\nIn this case though, the starting point is dist-git itself, so a few things\nneed to be done differently.\n\n_Note:_ things below were done on git.centos.org/rpms/rpm.\n\n## Intermezzo: what should happen to all the patches?\n\nHere is what we could do with all the patches:\n\n- Patches which are used in the spec file and are applied to the sources will\n  be deleted, a.k.a they wont show up in the source-git tree.\n\n- Patches which are used in the spec file and cannot be applied - this is\n  where tooling should err out, as something bad happened in dist-git.\n\n- Patches which are not used in the spec file will end up being in the\n  source-git tree under `centos-packaging/SOURCES/`. Maintainers might want to\n  clean this up in the future. Or these might be other kinds of files stored\n  for yet unknown reasons.\n\n### For the future\n\nAre there any expectations for how the history of a source-git repo will\nevolve?\n\nWhat should happen when sources in dist-git are updated? Currently we will\nrecreate the source-git branch.\n\nWhat should happen when the spec file or the patches in dist-git are updated?\n\n## Usage\n\nYou should always run this tool in the provided container (CentOS 8) to get the\ncorrect environment - RPM macros. See down below how to do it.\n\n`dist2src get-archive` calls [`get_sources.sh`] or the script specified in\n`DIST2SRC_GET_SOURCES`, so you either need to get and place this script in a\ndirectory in your PATH or use the environment variable to specify the tools to\ndownload the sources from the lookaside cache of the dist-git of your choice.\n\n## The Process\n\nWhen creating a source-git commit from dist-git, the process will be the\nfollowing:\n\n1. Take the content of the lookaside cache from a dist-git commit.\n\n2. Run `rpmbuild -bp` to unpack the archive and apply patches.\n\n3. Copy spec file, other sources and rebase patches on top.\n\nSimply put:\n\n    $ cd git.centos.org\n    $ dist2src convert rpms/rpm:c8s src/rpm:c8s\n\n## Using `rpmbuild -bp` to generate source-git repositories\n\nThe core way of the conversion process is running `rpmbuild -bp` to execute the\n`%prep` stage from the spec file which results in a directory containing the\nunpacked sources (under `\u003cdist-git\u003e/BUILD/*`).\n\nWith the following RPM-macro tweaks this directory can be turned into a Git\nrepository from where the script can pull the history resulting from the\nconversion:\n\n- We override all `scm_setup` and `scm_apply` macros so they create a git repo\n  from the archive and commit every patch applied.\n- SPEC-files which use `%setup` are modified before the conversion to use\n  `%autosetup -N` instead. This means the tool initializes a Git repository and\n  applies the patches [as Git commits](packitpatch). Currently it will also\n  create a \"Changes after running %prep\" commit to capture any modification of\n  the exploded sources which happens additionally to applying the patches.\n\n## Converting in a CentOS environment\n\nIn order to correctly evaluate the macros up to the `%prep` section, the right\ntarget environment has to be used. This is achieved by running the conversion\nscript in a container, built to match the target environment. By default this\nis CentOS 8.\n\nTo build the image, run:\n\n```\n$ podman build \\\n    [--build-arg base_image=centos:8] \\\n    [--build-arg package_manager=\"yum -y\"]  \\\n    -t dist2src .\n```\n\nThe build arguments are optional, the defaults being `centos:8` and `yum -y`.\n\nTo run the conversion:\n\n```\npodman run --rm -v $PWD:/workdir:z \\\n    dist2src dist2src convert rpms/\u003cpackage\u003e:\u003cbranch\u003e source-git/\u003cpackage\u003e:\u003cbranch\u003e\n```\n\nWhere the current working directory has the package cloned in an `rpms`\nsub-directory and the resulting source-git repo is going to be stored in\n`source-git`.\n\n[how to source-git?]: https://packit.dev/docs/source-git/how-to-source-git/\n[`get_sources.sh`]: https://wiki.centos.org/Sources#get_sources.sh_script\n[rebase-helper's `get_applied_patches()`]: https://github.com/rebase-helper/rebase-helper/blob/e98f4f6b14e2ca2e8cbb8a8fbeb6935e5d0cf289/rebasehelper/specfile.py#L351\n\n## Tests\n\nIn the `tests` test suite, you can find functional tests which convert\nreal dist-git packages.\n\nThey can be run in a container (`make build-test-image` and\n`make check-in-container`) or on localhost (`make check`).\n\n- You can override the container engine via the env var `CONTAINER_ENGINE`.\n- When running locally, have mock installed and set up -- the last step of\n  the testing is to build the generated SRPM from a source-git repo\n  using `mock --rebuild -r centos-stream-x86_64`).\n\nIt's also possible to invoke a single case (package):\n\n```\n$ TEST_TARGET=\"tests/test_convert.py::test_conversions[rpm-c8s]\" make check-in-container\n/usr/bin/podman run \\\n\t-ti --rm \\\n\t-v /??...??/dist-git-to-source-git/dist2src:/usr/local/lib/python3.6/site-packages/dist2src:Z \\\n\t-v /??...??/dist-git-to-source-git/packitpatch:/usr/bin/packitpatch:Z \\\n\t-v /??...??/dist-git-to-source-git/macros.packit:/usr/lib/rpm/macros.d/macros.packit:Z \\\n\t-v /??...??/dist-git-to-source-git/tests:/tests:Z \\\n\t--entrypoint= \\\n\t-u 1000 \\\n\t \\\n\tdist2src pytest --color=yes --showlocals -vv /tests/test_convert.py::test_conversions[rpm-c8s]\n============================ test session starts =============================\nplatform linux -- Python 3.6.8, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3.6\ncachedir: .pytest_cache\nrootdir: /tests\ncollected 1 item\n\n../tests/test_convert.py::test_conversions[rpm-c8s] PASSED             [100%]\n\n======================= 1 passed in 15.02s =======================\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpackit%2Fdist-git-to-source-git","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpackit%2Fdist-git-to-source-git","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpackit%2Fdist-git-to-source-git/lists"}