{"id":14982318,"url":"https://github.com/gchq/gaffer","last_synced_at":"2025-05-12T13:34:44.357Z","repository":{"id":2952587,"uuid":"47973088","full_name":"gchq/Gaffer","owner":"gchq","description":"A large-scale entity and relation database supporting aggregation of properties","archived":false,"fork":false,"pushed_at":"2025-02-06T10:57:03.000Z","size":228754,"stargazers_count":1782,"open_issues_count":138,"forks_count":360,"subscribers_count":137,"default_branch":"develop","last_synced_at":"2025-05-04T20:41:26.160Z","etag":null,"topics":["accumulo","aggregation","big-data","graph","graph-database","hadoop","hbase","parquet","spark"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gchq.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2015-12-14T12:12:39.000Z","updated_at":"2025-04-25T20:21:39.000Z","dependencies_parsed_at":"2024-02-26T12:26:03.840Z","dependency_job_id":"473c7dfe-1ff1-4df9-a695-caf83d3e8a47","html_url":"https://github.com/gchq/Gaffer","commit_stats":{"total_commits":4316,"total_committers":78,"mean_commits":"55.333333333333336","dds":0.6631139944392956,"last_synced_commit":"9bf6f1b19615f97b68a41705e28d5fa4773b6158"},"previous_names":[],"tags_count":150,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gchq%2FGaffer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gchq%2FGaffer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gchq%2FGaffer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gchq%2FGaffer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gchq","download_url":"https://codeload.github.com/gchq/Gaffer/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253748549,"owners_count":21957941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accumulo","aggregation","big-data","graph","graph-database","hadoop","hbase","parquet","spark"],"created_at":"2024-09-24T14:05:11.118Z","updated_at":"2025-05-12T13:34:44.329Z","avatar_url":"https://github.com/gchq.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg align=\"right\" width=\"300\" height=\"auto\" src=\"logos/logoWithText.png\"\u003e\n\n# Gaffer\n\n![ci](https://github.com/gchq/Gaffer/actions/workflows/continuous-integration.yaml/badge.svg?branch=develop)\n[![codecov](https://codecov.io/gh/gchq/Gaffer/branch/develop/graph/badge.svg?token=D7FRqMeurU)](https://codecov.io/gh/gchq/Gaffer)\n[\u003cimg src=\"https://img.shields.io/badge/docs-passing-success.svg?logo=readthedocs\"\u003e](https://gchq.github.io/gaffer-doc/latest/)\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/uk.gov.gchq.gaffer/gaffer2/badge.svg)](https://maven-badges.herokuapp.com/maven-central/uk.gov.gchq.gaffer/gaffer2)\n\nGaffer is a graph database framework. It allows the storage of very large graphs containing rich properties on the nodes and edges. Several storage options are available, including Accumulo and an in-memory Java Map Store.\n\nIt is designed to be as flexible, scalable and extensible as possible, allowing for rapid prototyping and transition to production systems.\n\nGaffer offers:\n\n - Rapid query across very large numbers of nodes and edges\n - Continual ingest of data at very high data rates, and batch bulk ingest of data via MapReduce or Spark\n - Storage of arbitrary Java objects on the nodes and edges\n - Automatic, user-configurable in-database aggregation of rich statistical properties (e.g. counts, histograms, sketches) on the nodes and edges\n - Versatile query-time summarisation, filtering and transformation of data\n - Fine grained data access controls\n - Hooks to apply policy and compliance rules to queries\n - Automated, rule-based removal of data (typically used to age-off old data)\n - Retrieval of graph data into Apache Spark for fast and flexible analysis\n - A fully-featured REST API\n\nTo get going with Gaffer, visit our getting started pages ([1.x](https://gchq.github.io/gaffer-doc/v1docs/summaries/getting-started.html), [2.x](https://gchq.github.io/gaffer-doc/latest/administration-guide/gaffer-deployment/quickstart)).\nWe also have a demo available to try that is based around a small uk road use dataset. See the example/road-traffic [README](https://github.com/gchq/Gaffer/blob/master/example/road-traffic/README.md) to try it out.\n\nGaffer is under active development. Version 1.0 of Gaffer was released in October 2017, version 2.0 was released in May 2023.\n\n## Contributing\n\nWe welcome contributions to the project.\n\n### Quickstart\n\n[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/gchq/Gaffer?quickstart=1)\n\nTo quickly and easily get access to an environment with everything installed and setup correctly you can use GitHub Codespaces, or alternatively GitLab GitPod.\nThese provide remote coding environments using VS Code with the required plugins, Java version and Maven preinstalled.\n\nOur Javadoc can be found [here](http://gchq.github.io/Gaffer/). Gaffer's documentation is kept in the [gaffer-doc](https://github.com/gchq/gaffer-doc) repository and [published on GitHub pages (gchq.github.io)](https://gchq.github.io/gaffer-doc/latest/).\n\n### Local Requirements\n\nFor building Gaffer locally you need Java 8 or 11 and Maven installed locally in a *nix environment. MS Windows will work for most purposes, but is not recommended because tests utilising Hadoop fail due to limited Hadoop support on Windows.\nGaffer will compile with newer versions of Java, but some tests will fail because of a lack of support for newer Java in certain external dependencies.\n\nTo build Gaffer run `mvn clean install -Pquick` in the top-level directory. This will build all of Gaffer's core libraries and some examples of how to load and query data.\n\n### Contribution Process\n\nDetailed information on our ways of working can be found [in our developer docs](https://gchq.github.io/gaffer-doc/latest/development-guide/ways-of-working). In brief:\n\n- Sign the [GCHQ Contributor Licence Agreement](https://cla-assistant.io/gchq/Gaffer)\n- Push your changes to a fork\n- Submit a pull request\n\n### Inclusion in other projects\n\nGaffer is hosted on [Maven Central](https://mvnrepository.com/search?q=uk.gov.gchq.gaffer) and can easily be incorporated into your own maven projects.\n\nTo use Gaffer from the Java API the only required dependencies are the Gaffer graph module and a store module for the specific database technology used to store the data, e.g. for the Accumulo store:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003euk.gov.gchq.gaffer\u003c/groupId\u003e\n    \u003cartifactId\u003egraph\u003c/artifactId\u003e\n    \u003cversion\u003e${gaffer.version}\u003c/version\u003e\n\u003c/dependency\u003e\n\u003cdependency\u003e\n    \u003cgroupId\u003euk.gov.gchq.gaffer\u003c/groupId\u003e\n    \u003cartifactId\u003eaccumulo-store\u003c/artifactId\u003e\n    \u003cversion\u003e${gaffer.version}\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nThis will include all other mandatory dependencies. Other (optional) components can be added to your project as required.\n\n## Related repositories\n\nThe [gafferpy](https://github.com/gchq/gafferpy) repository contains a python shell that can execute operations.\n\nThe [gaffer-docker](https://github.com/gchq/gaffer-docker) repository contains the code needed to run Gaffer using Docker or Kubernetes.\n\nThe [koryphe](https://github.com/gchq/koryphe) repository contains an extensible functions library for filtering, aggregating and transforming data\nbased on the Java Function API. It is a dependency of Gaffer.\n\n## License\n\nGaffer is licensed under the Apache 2 license and is covered by [Crown Copyright](https://www.nationalarchives.gov.uk/information-management/re-using-public-sector-information/uk-government-licensing-framework/crown-copyright/).  \n\n```\nCopyright 2016-2023 Crown Copyright\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n  http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgchq%2Fgaffer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgchq%2Fgaffer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgchq%2Fgaffer/lists"}