{"id":44689724,"url":"https://github.com/fanhua1994/DzFilter","last_synced_at":"2026-03-13T06:00:57.568Z","repository":{"id":27440255,"uuid":"113836623","full_name":"fanhua1994/DzFilter","owner":"fanhua1994","description":"【停止维护】使用DFA算法实现的内容安全,反垃圾,智能鉴黄,敏感词过滤,不良信息检测，文本校验，敏感词检测，包括关键词提取，过滤html标签等。","archived":false,"fork":false,"pushed_at":"2023-05-23T20:17:28.000Z","size":309,"stargazers_count":329,"open_issues_count":10,"forks_count":90,"subscribers_count":7,"default_branch":"master","last_synced_at":"2023-11-07T15:23:11.998Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fanhua1994.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-11T09:07:47.000Z","updated_at":"2023-11-05T11:09:36.000Z","dependencies_parsed_at":"2022-09-01T03:22:59.841Z","dependency_job_id":null,"html_url":"https://github.com/fanhua1994/DzFilter","commit_stats":null,"previous_names":[],"tags_count":16,"template":null,"template_full_name":null,"purl":"pkg:github/fanhua1994/DzFilter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fanhua1994%2FDzFilter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fanhua1994%2FDzFilter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fanhua1994%2FDzFilter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fanhua1994%2FDzFilter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fanhua1994","download_url":"https://codeload.github.com/fanhua1994/DzFilter/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fanhua1994%2FDzFilter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30459760,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-13T03:55:51.346Z","status":"ssl_error","status_checked_at":"2026-03-13T03:55:33.055Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-15T07:00:30.141Z","updated_at":"2026-03-13T06:00:57.549Z","avatar_url":"https://github.com/fanhua1994.png","language":"Java","funding_links":[],"categories":["人工智能"],"sub_categories":["自然语言处理"],"readme":"# copyright\n\u003e 不管您是个人还是公司都可以随意使用`DzFilter`，并且随意修改源代码。感谢大家的使用！\n\n# 温馨提示\n\u003e 用于生产环境时请替换项目的数据库连接代码，使用数据连接池的方式操作数据库！非常抱歉，本项目暂不维护！\n\n\n# 公告\n+ 目前1.0.7已经完善，暂时不支持sqlite强一致性，不能100%保证一致。\n+ 修复了集群情况下，重复通知的BUG。\n+ 1.0.9 版本移除关键词检索功能，如需使用请自行维护。\n+ 1.1.0 优化敏感词加载性能。\n\n# 使用指南\n+ [Spring配置指南](https://github.com/fanhua1994/DzFilter/blob/master/spring-use-guide.md)\n+ [SpringMvcDemo](https://github.com/fanhua1994/DzFilterSpringDemo)\n+ SpringBoot举一反三即可.\n\n# DzFilter\n目前最新版：1.1.0[[更新日志](https://github.com/fanhua1994/DzFilter/blob/master/log.md)]\n\n使用DFA算法实现的敏感词过滤。主要用于实现数据文本的内容安全,反垃圾,智能鉴黄,敏感词过滤,不良信息检测，携带文本的关键词获取。\n+ 过滤SQL脚本\n+ 过滤中文字符\n+ 过滤英文字符\n+ 过滤script标签\n+ 过滤html标签\n+ 过滤数字\n+ 过滤字母\n+ 过滤汉字\n+ 自定义过滤，可由后台自动删除添加。提供完善的API。\n\n\n# 如何使用\n## 导入以下仓库\n```\n\u003crepositories\u003e\n\t\u003crepository\u003e\n\t    \u003cid\u003ejitpack.io\u003c/id\u003e\n\t    \u003curl\u003ehttps://jitpack.io\u003c/url\u003e\n\t\u003c/repository\u003e\n\u003c/repositories\u003e\n```\n## 添加Dependency\n```\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.github.fanhua1994\u003c/groupId\u003e\n    \u003cartifactId\u003eDzFilter\u003c/artifactId\u003e\n    \u003cversion\u003e1.1.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## 功能详解\n### 过滤敏感词\n```\nString filter(String content)\n```\n### 判断是否包含敏感词\n```\nboolean existFilter(String content)\n```\n\n\n## 配置数据库\n\u003e 如果您需要使用在分布式的系统或同时在多个程序中使用```DzFilter```并且需要保持一致性，那么我们建议您使用mysql数据库，如果是单击应用我们建议您使用sqlite数据库。数据库文件在项目目录下的database下。放到对应目录  配置好就行了。如果想使用mysql的话，请自行导入mysql驱动，并将database目录下的java_filter.sql导入mysql然后在dzfilter_config.properties配置 用户名密码等信息即可。\n```\n\u003cdependency\u003e\n    \u003cgroupId\u003emysql\u003c/groupId\u003e\n    \u003cartifactId\u003emysql-connector-java\u003c/artifactId\u003e\n    \u003cversion\u003e5.1.45\u003c/version\u003e\n\u003c/dependency\u003e\n```\nMysql配置文件如下\n```\ndzfilter.db.driver=com.mysql.jdbc.Driver\ndzfilter.db.dburl=jdbc:mysql://120.77.245.xxx:3306/filter_word?useUnicode\\=true\u0026characterEncoding\\=utf-8\u0026autoReconnect\\=true\ndzfilter.db.is_mysql=true\n#如何您是使用mysql请配置以下信息\ndzfilter.db.dbuser=root\ndzfilter.db.dbpass=*****\n\n# 如果不是集群 请忽略以下配置\ndzfilter.cluster.open=true\ndzfilter.cluster.activemq=tcp://192.168.0.106:61616\ndzfilter.cluster.server_id=192.168.0.105\ndzfilter.cluster.channel_name=dzfilter\ndzfilter.cluster.username=admin\ndzfilter.cluster.password=admin\n```\nSQLite配置如下\n```\ndzfilter.db.driver=org.sqlite.JDBC\ndzfilter.db.dburl=jdbc:sqlite:/home/fanhua/database/data_filter20171211.db\ndzfilter.db.is_mysql=false\n#不是Mysql不用填写\ndzfilter.db.dbuser=root\ndzfilter.db.dbpass=***\n\n# 目前在SQLite数据库集群暂时没法保证一致性\ndzfilter.cluster.open=false\ndzfilter.cluster.activemq=tcp://192.168.0.106:61616\ndzfilter.cluster.server_id=192.168.0.105\ndzfilter.cluster.channel_name=dzfilter\ndzfilter.cluster.username=admin\ndzfilter.cluster.password=admin\n```\n\n## 集群配置\n如果您想要在多个项目中使用DzFilter请先安装activemq消息队列，在配置文件中进行配置，每个项目最好不要放到一台服务区，`dzfilter.cluster.server_id`参数对数据至关重要，请保证全局唯一。`dzfilter.cluster.open`请保证为true开启状态，\n多个项目之间需要保证`dzfilter.cluster.channel_name`参数一致。\n您只需要在多个项目中同时使用Activemq消费者监听`com.hengyi.dzfilter.listener.SyncMessageListener`即可。\n\n## 部分核心API\n\n### 添加自定义关键词过滤\n```\nBoolean res = TextUtils.addFilter(String keywords);\n```\n### 删除自定义关键词过滤\n```\nboolean res = TextUtils.delFilter(String keywords);\n```\n### 分页获取添加的关键词\n```\nList\u003cMap\u003cString,Object\u003e\u003e rows = TextUtils.getDataPage(int page,int num);//第一个参数页数 1开始 第二个参数每页条数\nList\u003cMap\u003cString,Object\u003e\u003e rows = TextUtils.getDataOffset(int offset,int limit);第一个参数偏移量 0开始 第二个参数每页条数\n```\n### 自定义关键词总数\n```\nint total = TextUtils.getDataTotal();\n```\n### 刷新敏感词\n```\nTextUtils.sync(boolean sync);\n```\n\n`sync`参数是否同步刷新整个集群。仅mysql数据库生效.\n\n\n```\nCopyright 2018 fanhua1994\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffanhua1994%2FDzFilter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffanhua1994%2FDzFilter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffanhua1994%2FDzFilter/lists"}