{"id":13848449,"url":"https://github.com/bbossgroups/bigdatas","last_synced_at":"2026-01-14T02:42:19.464Z","repository":{"id":35726048,"uuid":"40004394","full_name":"bbossgroups/bigdatas","owner":"bbossgroups","description":"this is a db-hdfs tools used to transfer big database  datas to hadoop hdfs like sqoop,but bboss bigdata tool is very nice monitor and event drivered model,and high perfermance,support Distributed executor tasks Ability.","archived":false,"fork":false,"pushed_at":"2025-11-17T03:13:34.000Z","size":71209,"stargazers_count":27,"open_issues_count":0,"forks_count":22,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-11-17T05:25:17.239Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://www.bbossgroups.com","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bbossgroups.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2015-07-31T12:31:24.000Z","updated_at":"2025-11-17T03:13:37.000Z","dependencies_parsed_at":"2023-02-10T10:15:35.188Z","dependency_job_id":"b69da317-dc44-4730-b49b-7a97ab1101d6","html_url":"https://github.com/bbossgroups/bigdatas","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/bbossgroups/bigdatas","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbossgroups%2Fbigdatas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbossgroups%2Fbigdatas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbossgroups%2Fbigdatas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbossgroups%2Fbigdatas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bbossgroups","download_url":"https://codeload.github.com/bbossgroups/bigdatas/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbossgroups%2Fbigdatas/sbom","scorecard":{"id":228076,"data":{"date":"2025-08-11","repo":{"name":"github.com/bbossgroups/bigdatas","commit":"9d925bd6e9aaacd017b0178cc524113bb4f75060"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.6,"checks":[{"name":"Maintained","score":0,"reason":"1 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Binary-Artifacts","score":9,"reason":"binaries present in source code","details":["Warn: binary detected: lib/ojdbc14.jar:1"],"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact 4.0.8 not signed: https://api.github.com/repos/bbossgroups/bigdatas/releases/1615799","Warn: release artifact 4.0.8 does not have provenance: https://api.github.com/repos/bbossgroups/bigdatas/releases/1615799"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'","Warn: branch protection not enabled for branch '4.0.9'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-17T04:12:37.447Z","repository_id":35726048,"created_at":"2025-08-17T04:12:37.447Z","updated_at":"2025-08-17T04:12:37.447Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28408712,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T01:52:23.358Z","status":"online","status_checked_at":"2026-01-14T02:00:06.678Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T19:00:49.510Z","updated_at":"2026-01-14T02:42:19.439Z","avatar_url":"https://github.com/bbossgroups.png","language":"Java","funding_links":[],"categories":["Java"],"sub_categories":[],"readme":"# bboss group website:\nhttp://www.bbossgroups.com\n\n# bboss group project blog:\nhttp://yin-bp.iteye.com/\n\n# 版本构建\n\n切换到命令行模式：\n\ncd D:\\bigdatas\n\n直接发布版本：\n\ngradle :bigdatamonitor:releaseRuntime\n\n构建发布的版本包存放在目录：\n\nD:\\bigdatas\\bigdatamonitor\\build\\distributions\n\n# 发布源码版本\n\ngradle clean publishToMavenLocal\n\n需要通过gradle构建发布版本,gradle安装配置参考文档：\n\nhttps://esdoc.bbossgroups.com/#/bboss-build\n\n# bboss大数据抽取工具功能特点如下：\n\n1.实现db到hadoop hdfs数据导入功能，提供高效的分布式并行处理能力，可以采用数据库分区、按字段分区、数据库分区结合字段分区、分页（不建议）四种方式并行批处理抽取db数据到hdfs文件系统中；\n\n2.能有效解决按字段分区抽取数据时，各分区数据不均匀导致作业任务处理节点负载不均衡的问题。\n\n3.灵活的作业任务处理模式：可以增量方式执行作业任务，作业可以停止后重新执行，重新执行时只需执行未完成的作业任务，也可以全部重新执行所有作业任务；当停止作业后，可以在原有作业切分的基础进一步切分出子任务，然后再重新执行作业，提升系统处理数据效率。\n\n4.作业处理服务器为每个作业分配独立的作业任务处理工作线程和任务执行队列，作业之间互不干扰\n\n5.采用异步事件驱动模式来管理和分发作业指令、采集作业状态数据。\n\n6.通过管理监控端，可以实时监控作业在各个数据处理节点作业任务的实时运行状态，查看作业的历史执行状态，方便地实现提交新的作业、重新执行作业、停止正在执行的作业、清除作业执行监控数据、同步作业任务执行状态等操作\n# 工具架构拓扑图： \n![Bigdata Architecture](http://dl2.iteye.com/upload/attachment/0110/6116/18258fcc-a9ff-3823-9c86-623585c90efc.png)\n\n# 工具采用的技术体系：\n\n 1.Bboss ioc：轻量级ioc容器，ioc扩展属性配置语法\n \n 2.Bboss持久层：高效数据查询行处理器，灵活动态数据源管理（连接池数据源/非连接池数据源），表分区信息查询等，动态创建作业配置表和作业监控记录表\n \n 3.Bboss分布式事件框架：基于JGroups，提供异步分布式事件驱动模型，动态管理作业节点（服务节点和数据处理节点），包括作业节点的动态加入、动态离开等；在管理节点、数据处理节点之间分发和接收各种作业处理指令事件\n \n 4.Bboss mvc：实现监控管理应用模块,在监控服务节点中，通过mvc 容器启动监听器启动作业管理节点\n \n 5.Bboss序列化组件：用来将作业监控数据序列化存储到sqllite中的作业监控表，同时在查看作业执行历史时将序列化存储的作业监控数据还原为对象状态的监控对象，便于界面展示\n \n 6.Bboss 标签库，jquery等：实现监控管理应用的视图层\n \n 7.Hadoop Hdfs客户端：用来连接hadoop hdfs文件系统\n \n 8.Sqllite：在监控节点中保存作业配置，保存作业执行状态数据\n \n 9.Jetty：运行监控管理应用模块的web应用容器\n \n 10.Bboss应用执行容器：用来启动作业管理监控应用、作业数据处理应用、启动jetty容器\n# bboss大数据抽取工具源码：\ngithub托管地址\nhttps://github.com/bbossgroups/bigdatas\noschina git地址\nhttps://git.oschina.net/bboss/bigdatas\n\n# 版本源码和发布包下载地址：\nhttps://github.com/bbossgroups/bigdatas/releases\n\n# 操作使用文档\n[大数据抽取工具管理操作手册.docx][].\n\n## bboss项目下载地址，参考文档：\nhttp://yin-bp.iteye.com/blog/1080824\n\n## License\n\nThe BBoss Framework is released under version 2.0 of the [Apache License][].\n\n[Apache License]: http://www.apache.org/licenses/LICENSE-2.0\n[大数据抽取工具管理操作手册.docx]: https://github.com/bbossgroups/bigdatas/blob/master/bigdatamonitor/%E5%A4%A7%E6%95%B0%E6%8D%AE%E6%8A%BD%E5%8F%96%E5%B7%A5%E5%85%B7%E7%AE%A1%E7%90%86%E6%93%8D%E4%BD%9C%E6%89%8B%E5%86%8C.docx?raw=true","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbossgroups%2Fbigdatas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbbossgroups%2Fbigdatas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbossgroups%2Fbigdatas/lists"}