{"id":13515042,"url":"https://github.com/microsoft/hyperspace","last_synced_at":"2025-09-27T13:30:32.819Z","repository":{"id":41495887,"uuid":"262123165","full_name":"microsoft/hyperspace","owner":"microsoft","description":"An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.","archived":true,"fork":false,"pushed_at":"2022-01-14T04:54:30.000Z","size":5104,"stargazers_count":424,"open_issues_count":135,"forks_count":114,"subscribers_count":34,"default_branch":"master","last_synced_at":"2025-01-15T05:27:14.696Z","etag":null,"topics":["acceleration","analytics","big-data","databases","indexing","spark"],"latest_commit_sha":null,"homepage":"https://aka.ms/hyperspace","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null}},"created_at":"2020-05-07T18:03:59.000Z","updated_at":"2024-10-29T00:16:46.000Z","dependencies_parsed_at":"2022-08-10T02:35:03.139Z","dependency_job_id":null,"html_url":"https://github.com/microsoft/hyperspace","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2Fhyperspace","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2Fhyperspace/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2Fhyperspace/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2Fhyperspace/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microsoft","download_url":"https://codeload.github.com/microsoft/hyperspace/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234438033,"owners_count":18832609,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["acceleration","analytics","big-data","databases","indexing","spark"],"created_at":"2024-08-01T05:01:05.626Z","updated_at":"2025-09-27T13:30:32.173Z","avatar_url":"https://github.com/microsoft.png","language":"Scala","readme":"![Icon](https://github.com/rapoth/hyperspace/blob/master/docs/assets/images/hyperspace-small-banner.png?raw=true)\n\n# Hyperspace\n\nAn open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.\n\n[aka.ms/hyperspace](https://aka.ms/hyperspace)\n\n[![Build Status](https://dev.azure.com/ossspark/public/_apis/build/status/microsoft.hyperspace?branchName=master)](https://dev.azure.com/ossspark/public/_build/latest?definitionId=3\u0026branchName=master)\n[![javadoc](https://javadoc.io/badge2/com.microsoft.hyperspace/hyperspace-core_2.12/javadoc.svg)](https://javadoc.io/doc/com.microsoft.hyperspace/hyperspace-core_2.12/latest/com/microsoft/hyperspace/index.html)\n\n## Contributing\n\nThis project welcomes contributions and suggestions.  Most contributions require you to agree to a\nContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us\nthe rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.\n\nWhen you submit a pull request, a CLA bot will automatically determine whether you need to provide\na CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions\nprovided by the bot. You will only need to do this once across all repos using our CLA.\n\nPlease review our [contribution guide](CONTRIBUTING.md).\n\n### Development on Windows\n\nThis repository contains symbolic links which don't work properly on Windows. To build this project on Windows, you can use our provided Git aliases to replace symbolic links with junctions.\n\n```sh\n$ git config --local include.path ../dev/.gitconfig\n$ git replace-symlinks # replace symlinks with junctions\n$ #git restore-symlinks # use this to restore symlinks if you need\n```\n\n### Using IntelliJ\n\nYou can use the built-in sbt shell in IntelliJ without any problems. However, the built-in \"Build Project\" command may not work. To fix the issue, go to Project Structure -\u003e Project Settings -\u003e Modules and follow these steps:\n\n- Mark `src/main/scala` and `src/main/scala-spark2` as \"Sources\" and `src/test/scala` and `src/test/scala-spark2` as \"Tests\" for the `spark2_4` module.\n- Mark `src/main/scala` and `src/main/scala-spark3` as \"Sources\" and `src/test/scala` and `src/test/scala-spark3` as \"Tests\" for the `spark3_0` module.\n- Remove the `root` and `hyperspace-sources` modules.\n- An example of [Project Structure](./docs/assets/images/project_structure.png)\n\nAdditionally, you might have to run `sbt buildInfo` if you encounter an error like `object BuildInfo is not a member of package com.microsoft.hyperspace` for the first build.\n\n## Inspiration and Special Thanks\n\nThis project would not have been possible without the outstanding work from the following communities:\n\n- [Apache Spark](https://spark.apache.org/): Unified Analytics Engine for Big Data, the engine that\n  Hyperspace builds on top of.\n- [Delta Lake](https://delta.io): Delta Lake is an open-source storage layer that brings ACID\n  transactions to Apache Spark™ and big data workloads. Hyperspace derives quite a bit of inspiration\n  from the way the Delta Lake community operates and pioneering of some surrounding ideas in the\n  context of data lakes (e.g., their novel use of optimistic concurrency).\n- [Databricks](https://databricks.com/): Unified analytics platform. Many thanks to all the inspiration\n  they have provided us.\n- [.NET for Apache Spark™](https://github.com/dotnet/spark): Hyperspace offers .NET bindings for\n  developers, thanks to the efforts of this team in collaborating and releasing the bindings just-in-time.\n- [Minimal Mistakes](https://github.com/mmistakes/minimal-mistakes): The awesome theme behind\n  Hyperspace documentation.\n\n## Code of Conduct\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).\nFor more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or\ncontact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n## License\n\nApache License 2.0, see [LICENSE](https://github.com/microsoft/hyperspace/blob/master/LICENSE).\n","funding_links":[],"categories":["Repositories"],"sub_categories":["Tools \u0026 Samples"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fhyperspace","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrosoft%2Fhyperspace","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fhyperspace/lists"}