{"id":16847440,"url":"https://github.com/indygreg/apple-opensource-downloader","last_synced_at":"2025-04-11T06:32:26.122Z","repository":{"id":66133291,"uuid":"430003577","full_name":"indygreg/apple-opensource-downloader","owner":"indygreg","description":"Download Apple's open source code from opensource.apple.com","archived":false,"fork":false,"pushed_at":"2022-07-24T05:06:33.000Z","size":17,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-25T04:26:51.440Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/indygreg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-20T04:02:36.000Z","updated_at":"2023-01-13T10:33:28.000Z","dependencies_parsed_at":"2023-02-24T08:00:29.557Z","dependency_job_id":null,"html_url":"https://github.com/indygreg/apple-opensource-downloader","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indygreg%2Fapple-opensource-downloader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indygreg%2Fapple-opensource-downloader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indygreg%2Fapple-opensource-downloader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indygreg%2Fapple-opensource-downloader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/indygreg","download_url":"https://codeload.github.com/indygreg/apple-opensource-downloader/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248355620,"owners_count":21090059,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T13:07:57.547Z","updated_at":"2025-04-11T06:32:26.085Z","avatar_url":"https://github.com/indygreg.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Apple Open Source Downloader\n\nThis repository defines a Rust crate and CLI program to automate the downloading\nof Apple's open source code from https://opensource.apple.com/.\n\nThe primary goal of this project is to enable more intuitive usage and\ninspection of Apple's open source code. Using this tool you can:\n\n* Convert the history of an Apple component (like the `xnu` core OS\n  primitives) to a Git repository and easily view differences between releases.\n* Convert the history of a set of open source components (such as everything\n  comprising macOS) to a Git repository and easily view differences between\n  releases.\n* Query for all available open source components and releases.\n\nThe canonical home for this project is\nhttps://github.com/indygreg/apple-opensource-downloader. Please report issues or\nsubmit enhancements there.\n\n# Installing\n\n```\n# From crates.io\n$ cargo install apple-opensource-downloader\n\n# From Git\n$ cargo install --git https://github.com/indygreg/apple-opensource-downloader.git --branch main\n```\n\n# Using\n\nThe `apple-opensource-downloader` CLI is provided. It defines sub-commands to\nperform various actions. Run `apple-opensource-downloader help` to see the help.\n\n## Download a Single Component to a Git Repository\n\nThe `component-to-git` sub-command will download all versions of a named\nsoftware component (see the component list at\nhttps://opensource.apple.com/tarballs/) and write their contents as Git commits\nto a Git repository. This allows you to see differences between the versions of\na component.\n\n```\n$ apple-opensource-downloader component-to-git --no-bare xnu aos/xnu\n\u003cwait for this to finish\u003e\n\n$ cd aos/xnu\n$ git log\ncommit c2011455c3d75195791bd20d189abae4917c8c81 (HEAD -\u003e main, tag: 7195.141.2)\nAuthor: Apple Open Source \u003copensource@apple.com\u003e\nDate:   Fri Jan 1 00:00:00 2021 +0000\n\n    xnu 7195.141.2\n\n    Downloaded from https://opensource.apple.com/tarballs/xnu/xnu-7195.141.2.tar.gz\n\ncommit e76ea20b5519ae2eaf2b74698bd2331141e028fa (tag: 7195.121.3)\nAuthor: Apple Open Source \u003copensource@apple.com\u003e\nDate:   Fri Jan 1 00:00:00 2021 +0000\n\n    xnu 7195.121.3\n\n    Downloaded from https://opensource.apple.com/tarballs/xnu/xnu-7195.121.3.tar.gz\n\n...\n```\n\nThe Git trees and commit objects should be deterministic provided that the\nversion of this software is identical and the Apple-hosted source archives don't\nchange. i.e. different machines should produce Git commits with the same\ncommit IDs.\n\n## Download all Components to Git Repositories\n\nThe `components-to-gits` sub-command will download each available component and\nwrite each to separate Git repositories. It is equivalent to running\n`component-to-git` for every named component.\n\n## Download An Apple Software Release to a Git Repository\n\nThe `release-to-git` command can be used to download all components in a logical\nApple software release (such as macOS or iOS) to a Git repository. This enables\nyou to view the history and changes of open source components between releases.\n\n```\n$ apple-opensource-downloader release-to-git --no-bare macos aos/macOS\n$ cd aos/macOS\n$ git log\n\n\n```\n\n# Known Issues\n\nThe HTML parsing isn't the most robust and may not scrape all available software.\n\nIf Apple changes the HTML on opensource.apple.com, it will break this tool.\n\nIf Apple imposes throttling on their servers, it will likely break this tool.\n\nCreated Git repositories don't use packfiles and their performance may be\nsub-optimal. Run `git gc` after Git repo creation to optimize the Git repositories.\n\nWe don't support incrementally updating Git repositories. Git repositories have\ntheir history recreated from scratch on every invocation. This is obviously\ninefficient.\n\nVarious advertised URLs on opensource.apple.com result in an HTTP 404. These\nare sometimes ignored by this tool.\n\nWhen importing software releases (such as macOS), the components from one release\nto the next may vary. e.g. SQLite could be there in release A, gone in release B,\nand reappear in release C. This may make `git diff` output non-representative.\n\nGit commits have a hard-coded date that has no basis in reality.\n\nThe naming and layout of Apple's components can at times be confusing and\ninconsistent. We don't yet make an effort to reconcile this.\n\n# Legal Compliance\n\nThe content downloaded by this tool may be governed by license and usage\nrestrictions defined outside this tool. Check for usage restrictions\nposted at https://opensource.apple.com/ and within the downloaded content.\n\ni.e. if you redistribute the downloaded content, Apple may take an issue\nwith that.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findygreg%2Fapple-opensource-downloader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Findygreg%2Fapple-opensource-downloader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findygreg%2Fapple-opensource-downloader/lists"}