{"id":20578790,"url":"https://github.com/cclient/elasticsearch-filter-limitbyfreq","last_synced_at":"2026-07-29T13:31:31.487Z","repository":{"id":93977104,"uuid":"102115834","full_name":"cclient/elasticsearch-filter-limitbyfreq","owner":"cclient","description":"elasticsearch token limit by freq,过滤出权重较重的top个词,为下一步simhash作准备","archived":false,"fork":false,"pushed_at":"2017-09-01T14:23:42.000Z","size":11,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-16T22:20:06.094Z","etag":null,"topics":["byfreq","elasticsearch-filter","limit"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cclient.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-01T13:30:57.000Z","updated_at":"2020-09-03T00:22:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"702c555a-6ba1-46f5-89ae-d97b54555499","html_url":"https://github.com/cclient/elasticsearch-filter-limitbyfreq","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cclient%2Felasticsearch-filter-limitbyfreq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cclient%2Felasticsearch-filter-limitbyfreq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cclient%2Felasticsearch-filter-limitbyfreq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cclient%2Felasticsearch-filter-limitbyfreq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cclient","download_url":"https://codeload.github.com/cclient/elasticsearch-filter-limitbyfreq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242206001,"owners_count":20089251,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["byfreq","elasticsearch-filter","limit"],"created_at":"2024-11-16T06:14:35.687Z","updated_at":"2025-10-12T14:43:34.194Z","avatar_url":"https://github.com/cclient.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"Limit Token Filter for Elasticsearch\n==================================\n\nFilter: limit_by_freq\n\nParameter: max_token_count(default:512)\n\nDesc: token order by freq desc and limit top \n\nfreq num is stored in 'payload' to be used in future\n\nInstall\n-------\n\n1.download or compile\n\n* download pre-build package from here: https://github.com/cclient/elasticsearch-filter-limitbyfreq/releases\n    \n    unzip plugin to folder `your-es-root/plugins/`\n\n2.restart elasticsearch\n\n\n#### Quick Example\n\n1.create a index\n\n```bash\ncurl -XPUT http://localhost:9200/test_index -d'\n{\n\t\"settings\": {\n\t\t\"analysis\": {\n            \"filter\": {\n                \"my_limit\": { \n                    \"type\":\"limit_by_freq\",\n                    \"max_token_count\":2\n                }\n            },\n            \"analyzer\": {\n                \"limit_test\": {\n                    \"tokenizer\": \"standard\",\n                    \"filter\": [\n\t\t\t\t\t\t\"my_limit\"\n                    ]\n                }\n            }\n        }\n\t},\n\t\"mappings\": {\n\t\t\"test\": {\n\t\t\t\"properties\": {\n\t\t\t\t\"desc\":\t\t{ \n\t\t\t\t\t\"type\": \"text\",\n\t\t\t\t\t\"analyzer\": \"limit_test\"\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n}'\n```\n\n2.test \n\n```bash\ncurl -XPOST http://localhost:9200/test_index/_analyze?tokenizer=standard\u0026filter=limit_by_freq -d'\nhello hyper log log'\n```\n\nResult\n\n```json\n{\n    \"tokens\": [\n        {\n            \"token\": \"log\",\n            \"start_offset\": 0,\n            \"end_offset\": 0,\n            \"type\": \"TOP_TOKEN\",\n            \"position\": 0\n        },\n        {\n            \"token\": \"hello\",\n            \"start_offset\": 0,\n            \"end_offset\": 0,\n            \"type\": \"TOP_TOKEN\",\n            \"position\": 1\n        },\n        {\n            \"token\": \"hyper\",\n            \"start_offset\": 0,\n            \"end_offset\": 0,\n            \"type\": \"TOP_TOKEN\",\n            \"position\": 2\n        }\n    ]\n}\n```\n\n\n\n```bash\ncurl -XPOST http://127.0.0.1:9200/test_index/_analyze?analyzer=limit_test -d'\nhello hyper log log'\n```\n\nResult\n\n```json\n{\n    \"tokens\": [\n        {\n            \"token\": \"log\",\n            \"start_offset\": 0,\n            \"end_offset\": 0,\n            \"type\": \"TOP_TOKEN\",\n            \"position\": 0\n        },\n        {\n            \"token\": \"hello\",\n            \"start_offset\": 0,\n            \"end_offset\": 0,\n            \"type\": \"TOP_TOKEN\",\n            \"position\": 1\n        }\n    ]\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcclient%2Felasticsearch-filter-limitbyfreq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcclient%2Felasticsearch-filter-limitbyfreq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcclient%2Felasticsearch-filter-limitbyfreq/lists"}