{"id":18727696,"url":"https://github.com/mobilenativefoundation/index-import","last_synced_at":"2025-05-16T07:05:32.665Z","repository":{"id":40495040,"uuid":"193001166","full_name":"MobileNativeFoundation/index-import","owner":"MobileNativeFoundation","description":"Tool to import swiftc and clang index-store files into Xcode","archived":false,"fork":false,"pushed_at":"2025-05-01T03:06:40.000Z","size":1518,"stargazers_count":327,"open_issues_count":12,"forks_count":25,"subscribers_count":276,"default_branch":"main","last_synced_at":"2025-05-01T03:34:31.888Z","etag":null,"topics":["bazel","xcode"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MobileNativeFoundation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-06-20T23:54:32.000Z","updated_at":"2025-04-16T04:42:04.000Z","dependencies_parsed_at":"2023-10-03T08:16:15.784Z","dependency_job_id":"676e57cd-41eb-448f-92e3-5d962e7b254b","html_url":"https://github.com/MobileNativeFoundation/index-import","commit_stats":{"total_commits":87,"total_committers":14,"mean_commits":6.214285714285714,"dds":0.7011494252873562,"last_synced_commit":"0457a33cccb6e5c9b077f276bf35f111037ba9f4"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobileNativeFoundation%2Findex-import","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobileNativeFoundation%2Findex-import/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobileNativeFoundation%2Findex-import/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MobileNativeFoundation%2Findex-import/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MobileNativeFoundation","download_url":"https://codeload.github.com/MobileNativeFoundation/index-import/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254485060,"owners_count":22078767,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bazel","xcode"],"created_at":"2024-11-07T14:18:30.362Z","updated_at":"2025-05-16T07:05:27.657Z","avatar_url":"https://github.com/MobileNativeFoundation.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# index-import\n\nA tool to import `swiftc` and `clang` generated indexes into Xcode.\n\n## Overview\n\nThe `index-import` tool makes indexes portable. The ability to copy indexes into Xcode has a few possible uses:\n\n1. Using a separate build system (Bazel, Buck, CMake, SwiftPM, etc)\n2. Distributing a CI built index to developer machines\n3. Sharing an index between two or more local checkouts\n\nThe common goal is to reduce or eliminate the time Xcode spends indexing.\n\nFor Xcode to be able to use an index, the file paths contained in the index must match the paths Xcode uses for lookup. This is the main feature of `index-import`, rewriting the paths inside the index files. This path remapping requires knowing input and output paths.\n\nPath remapping is done with regex substitution. `index-import` accepts one or more `-remap` flags which are formatted as `\u003cregex\u003e=\u003csubstitution\u003e`. See the [examples](#examples) below. Path remapping is conceptually similar to `sed s/regex/substitution/`. In all cases, the substitution will either be a path within the project, or a path within `DerivedData`.\n\n## Examples\n\nThe simplest example is to consider the case of two checkouts of the same project on the same machine. If one project has a built index, it can be imported into the other. To do this, two paths need to be remapped: the project directory and the build directory (`DerivedData`).\n\n```sh\n#!/bin/bash\n\nbuild_dir1=\"/Users/me/Library/Developer/Xcode/DerivedData/MyApp-abc123\"\nbuild_dir2=\"/Users/me/Library/Developer/Xcode/DerivedData/MyApp-xyz789\"\n\nindex-import \\\n    -remap \"/Users/me/MyApp=/Users/me/MyApp2\" \\\n    -remap \"$build_dir1=$build_dir2\" \\\n    \"$build_dir1/Index/DataStore\" \\\n    \"$build_dir2/Index/DataStore\"\n```\n\nA more complex example is importing an index from a [Bazel](https://bazel.build) built project. This example would be run as an Xcode \"Run Script\" build phase, which provides many environment variables, including: `SRCROOT`, `CONFIGURATION_TEMP_DIR`, `ARCHS`.\n\n```sh\n#!/bin/bash\n\nset -euo pipefail\n\n# Input: /Users/me/Library/Developer/Xcode/DerivedData/PROJECT-abc/Build/Products\n# Output: /Users/me/Library/Developer/Xcode/DerivedData/PROJECT-abc\nderived_data_root=$(dirname \"$(dirname \"$BUILD_DIR\")\")\nreadonly xcode_index_root=\"$derived_data_root/Index.noindex/DataStore\"\n\n# Captures: 1) module name\nreadonly bazel_swiftmodules=\"^/__build_bazel_rules_swift/swiftmodules/(.+).swiftmodule\"\nreadonly xcode_swiftmodules=\"$BUILT_PRODUCTS_DIR/\\$1.swiftmodule/$ARCHS.swiftmodule\"\n\n# Captures: 1) target name, 2) object name\nreadonly bazel_objects=\"^\\./bazel-out/.+?/bin/.*?(?:[^/]+)/([^/]+?)_objs(?:/.*)*/(.+?)\\.swift\\.o$\"\nreadonly xcode_objects=\"$CONFIGURATION_TEMP_DIR/\\$1.build/Objects-normal/$ARCHS/\\$2.o\"\n\nindex-import\n    -remap \"$bazel_swiftmodules=$xcode_swiftmodules\" \\\n    -remap \"$bazel_objects=$xcode_objects\" \\\n    -remap \"^\\.=$SRCROOT\" \\\n    -remap \"DEVELOPER_DIR=$DEVELOPER_DIR\" \\\n    -incremental \\\n    @\"$index_stores_file\" \\\n    \"$xcode_index_root\"\n```\n\nSince Xcode 14 / Swift 5.7, `clang` and `swiftc` support remapping paths\nin index data using `-ffile-prefix-map=foo=bar` and `-file-prefix-map\nfoo=bar` respectively. Using this makes it easy to generate a\nreproducible index that can be transferred between machines, and then\nremapped to local only paths using one of the examples above.\n\n## Build Instructions\n\nThe build uses [CMake](https://cmake.org) because [Apple's LLVM fork](https://github.com/apple/llvm-project) uses CMake. The `index-import` build script is small, but depends on the libraries from LLVM. To build `index-import`, first [install the tools required by Swift](https://github.com/apple/swift/blob/main/docs/HowToGuides/GettingStarted.md#system-requirements), then build swift by following the [Swift build instructions](https://github.com/apple/swift/blob/main/docs/HowToGuides/GettingStarted.md#building-the-project-for-the-first-time).\n\nWhen building Swift, keep the following in mind:\n\n1. Checkout the desired release branch of Swift using something like `./swift/utils/update-checkout --clone --scheme release/5.7`.\n2. Build Swift using `--release`/`-R` for performance\n\nBuilding all of Swift can take a long time, and most of that isn't needed by `index-import`. A faster way to build `index-import`, is to build only `libIndexStore.dylib`. Here are the commands to do just that:\n\n```sh\n./swift/utils/build-script --release --skip-build --llvm-targets-to-build X86\nninja -C build/Ninja-ReleaseAssert/llvm-macosx-x86_64 libIndexStore.dylib\n```\n\nOnce swift (or `libIndexStore.dylib`) has been built, `index-import` can be built as follows. The _key_ step is to update your `PATH` variable to include the llvm `bin/` directory (from the swift-source build directory). This ensures CMake can find all necessary build dependencies.\n\n```sh\n# From the index-import directory\nmkdir build\ncd build\nPATH=\"path/to/swift-source/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/bin:$PATH\"\ncmake -G Ninja -DCMAKE_BUILD_TYPE=Release ..\nninja\n```\n\nIf you need to cross compile checkout [RELEASING.md](RELEASING.md)\n\nOr, if you prefer Xcode for building and debugging, you can replace the last 2 lines with the following:\n\n```\ncmake -G Xcode -DCMAKE_BUILD_TYPE=Release ..\nopen index-import.xcodeproj\n```\n\n## Index File Format\n\nThe index consists of two types of files, Unit files and Record files. Both are [LLVM Bitstream](https://www.llvm.org/docs/BitCodeFormat.html#bitstream-format), a common binary format used by LLVM/Clang/Swift. Record files contain no paths and can be simply copied. Only Unit files contain paths, so only unit files need to be rewritten. A read/write API is available in the `clangIndex` library. `index-import` uses [`IndexUnitReader`](https://github.com/apple/llvm-project/blob/swift/release/5.7/clang/include/clang/Index/IndexUnitReader.h) and [`IndexUnitWriter`](https://github.com/apple/llvm-project/blob/swift/release/5.7/clang/include/clang/Index/IndexUnitWriter.h).\n\n## Resources\n\nThe best information on the `swiftc` and `clang` index store comes from these two resources:\n\n* [Adding Index‐While‐Building and Refactoring to Clang](https://www.youtube.com/watch?v=jGJhnIT-D2M), 2017 LLVM Developers Meeting, by Alex Lorenz and Nathan Hawes\n* [Indexing While Building whitepaper](https://docs.google.com/document/d/1cH2sTpgSnJZCkZtJl1aY-rzy4uGPcrI-6RrUpdATO2Q/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmobilenativefoundation%2Findex-import","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmobilenativefoundation%2Findex-import","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmobilenativefoundation%2Findex-import/lists"}