{"id":25305720,"url":"https://github.com/philterd/entitydb","last_synced_at":"2025-10-28T10:31:42.715Z","repository":{"id":229006807,"uuid":"522288745","full_name":"philterd/entitydb","owner":"philterd","description":"A unified means for storing and querying entities","archived":true,"fork":false,"pushed_at":"2024-10-29T13:06:01.000Z","size":1220,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-07T05:47:10.167Z","etag":null,"topics":["database","entity","nlp","query"],"latest_commit_sha":null,"homepage":"https://www.philterd.io","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/philterd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-07T18:09:51.000Z","updated_at":"2024-11-30T01:34:58.000Z","dependencies_parsed_at":"2024-09-15T05:14:14.379Z","dependency_job_id":"86c15a28-e167-41bc-b853-7272f9b8bb6a","html_url":"https://github.com/philterd/entitydb","commit_stats":null,"previous_names":["philterd/entitydb"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/philterd/entitydb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philterd%2Fentitydb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philterd%2Fentitydb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philterd%2Fentitydb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philterd%2Fentitydb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/philterd","download_url":"https://codeload.github.com/philterd/entitydb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/philterd%2Fentitydb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281424736,"owners_count":26499021,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-28T02:00:06.022Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","entity","nlp","query"],"created_at":"2025-02-13T09:53:45.799Z","updated_at":"2025-10-28T10:31:41.938Z","avatar_url":"https://github.com/philterd.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# EntityDB\n\nEntityDB is an application that integrates several components to provide a unified means for storing and querying entities (people, places, and things). This project includes the Entity Query Language (EQL) which facilitates querying entities across various underlying databases through a single query language.\n\n## Architecture\n\nEntities are stored in an underlying database. Supported databases are MySQL, MongoDB, Cassandra, and DynamoDB. Entities are indexed in Elasticsearch for fast querying. A cache stores recently ingested and accessed entities to improve performance. A separate database, the data store, manages data such as users, groups, queries, and other information.\n\n## Features\n\nThe following are brief high-level descriptions of EntityDB's main features. Refer to the wiki for more detailed descriptions and information on how to configure and use the features.\n\n### API\n\nThe API is built on REST and JSON. The API allows for entity ingestion, status and health monitoring, and entity querying through the Entity Query Language (EQL).\n\n### Entity Store\n\nThe entity store is the master dataset of entities. It is an immutable data store. EntityDB provides a choice of MySQL, MongoDB, Cassandra, and DynamoDB for the underlying entity store. You are free to choose the database that best satisfies your use-case requirements.\n\n### Search Index\n\nAs entities are ingested they are indexed in a search engine. All queries are performed against the search engine. Currently, the only supported search index is Elasticsearch.\n\n### Entity Access Control\n\nEach ingested entity is assigned an ACL. The ACL determines the entity's visibility to users and groups of the system.\n\n### Audit\n\nVarious actions that occur in EntityDB are outputted as audit events. Some of the audited events include entity ingests, entities returned through queries, and entity ACL modifications.\n\n### Continuous Queries\n\nEntities received through the API are evaluated by the continuous queries. A continuous query is an EQL query that generates a notification when an entity meets the query's conditions. Continuous queries can be used to receive notifications that an ingested entity satisfies some conditions. Continuous queries are designed to be fast and efficient and promote a low time-to-alert (TTA).\n\nFor example, the continuous query `select * from entities where text = 'George'` will generate a notification when an entity having the text \"George\" is ingested.\n\n### Rules Engine\n\nSimilar to continuous queries, the rules engine is executed for each ingested entity. Rules are user-defined and can be created to take a specific action on entities that are found to match one or more conditions. However, unlike continuous queries, rules can contain complex logic and actions and are designed to be executed when time-to-alert is not critical.\n\n### Metric Reporting\n\nEntityDB can report metrics to AWS CloudWatch, InfluxDB, or the console. These metrics report values such as how long an entity is in the ingest queue before being ingested, how long continuous queries are taking to evaluate, and the counts of stored and indexed entities. These metrics provide a comprehensive overview of EntityDB's performance and statistics.\n\n### Scalability\n\nEntityDB is easily scaled horizontally since its components are all distributed. Simply stand up a new EntityDB instance to increase its throughput and performance. EntityDB's sample AWS CloudFormation creates an EntityDB autoscaling group behind an Elastic Load Balancer. The autoscaling group can be set to scale based on metrics such as the size of the ingest SQS queue, any of the EntityDB reported metrics, or any EC2 instance metrics.\n\n## Building EntityDB\n\nDuring EntityDB's build tests will be run. Some of the unit tests are more like integration tests and this is an area for improvement.\n\n```\nmvn clean install\n```\n\n## Running\n\nOnce successfully built, an `entitydb.jar` will be under `entitydb-app/target`. This is a runnable jar that can be started with `java -jar entitydb.jar`. By default, all components will use internal implementations but this can be changed in the `entitydb.properties`. See the [Documentation](https://github.com/mtnfog/entitydb/blob/master/documentation.md) for details on configuring the `entitydb.properties`.\n\n### Ingesting Entities\n\n#### Via the REST API\n\nEntities can be ingested through the API. Look under the `scripts/` directory for sample cURL scripts. Entities must be in the format defined in [entity-model](https://github.com/mtnfog/entity-model). Ingested entities are immutable.\n\n#### Via the Internal API\n\nWhen integrated directly with your application entities can be ingested through the queues bypassing the REST API. It is not recommended to ingest without queuing entities in order to prevent entity loss due to capacity or network issues.\n\nCopyright © 2024 Philterd, LLC.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphilterd%2Fentitydb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphilterd%2Fentitydb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphilterd%2Fentitydb/lists"}