{"id":18309799,"url":"https://github.com/yinfuyuan/docker-bigdata","last_synced_at":"2026-04-14T06:33:46.892Z","repository":{"id":240763214,"uuid":"136018311","full_name":"yinfuyuan/docker-bigdata","owner":"yinfuyuan","description":"This is a project created to build a big data cluster.","archived":false,"fork":false,"pushed_at":"2018-07-02T13:09:51.000Z","size":164,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-07-10T02:25:31.277Z","etag":null,"topics":["apache","docker","docker-compose","hadoop","hbase","kafak","zookeeper"],"latest_commit_sha":null,"homepage":"https://github.com/yinfuyuan/docker-bigdata","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yinfuyuan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-04T11:57:39.000Z","updated_at":"2022-05-03T02:54:06.000Z","dependencies_parsed_at":"2024-05-20T20:14:20.725Z","dependency_job_id":null,"html_url":"https://github.com/yinfuyuan/docker-bigdata","commit_stats":null,"previous_names":["yinfuyuan/docker-bigdata"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/yinfuyuan/docker-bigdata","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yinfuyuan%2Fdocker-bigdata","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yinfuyuan%2Fdocker-bigdata/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yinfuyuan%2Fdocker-bigdata/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yinfuyuan%2Fdocker-bigdata/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yinfuyuan","download_url":"https://codeload.github.com/yinfuyuan/docker-bigdata/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yinfuyuan%2Fdocker-bigdata/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31785677,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T02:24:21.117Z","status":"ssl_error","status_checked_at":"2026-04-14T02:24:20.627Z","response_time":153,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache","docker","docker-compose","hadoop","hbase","kafak","zookeeper"],"created_at":"2024-11-05T16:12:27.021Z","updated_at":"2026-04-14T06:33:46.876Z","avatar_url":"https://github.com/yinfuyuan.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Docker apache\r\n\r\nThis is a project created to build a apache large data cluster.\r\n\r\nThe services provided include but not limited to **hadoop** **hbase** **kafka** and **zookeeper**.\r\n\r\nYou can run this software on linux or windows or mac, recommending use of x86_64 linux platforms.\r\n\r\n## Dependencies\r\n\r\ngit version 1.8.3.1 or higher\r\n\r\ndocker version 17.09.0-ce or higher\r\n\r\ndocker-compose version 1.16.1 or higher\r\n\r\n## Get installed\r\n    \r\n    $ git clone https://github.com/yinfuyuan/docker-bigdata.git\r\n\r\n## Services\r\n\r\nYou can start all services like this:\r\n\r\n    #docker-compose up -d\r\n\r\nYou can start some of the services you need like this:\r\n\r\n    #docker-compose up -d SERVICE1 SERVICE2\r\n    \r\nexample:\r\n\r\n    #docker-compose up -d zookeeper1 zookeeper2 zookeeper3 hbase1 hbase2 hbase3\r\n    \r\nYou can also start them using yam files like this:\r\n\r\n    #docker-compose [-f docker-compose-SERVICE.yml] up -d\r\n    \r\nexample:\r\n\r\n    #docker-compose -f docker-compose-zookeeper.yml up -d\r\n    #docker-compose -f docker-compose-hbase.yml up -d\r\n    \r\nThe command above started the zookeeper and hbase service cluster,you can use\r\na browser to enter the host machine ip and 16010 port to view hbase status.\r\n    \r\nAll images have been built and in the dockerhub, you can directly download to use, \r\nbut if you use yml file start for the first time and you don't have local image, \r\ndocker will try to build them not to download, It may waste your time.\r\nIf this is not what you want, try starting after manually removing the build option from yml file.\r\n\r\n### zookeeper\r\n    \r\n    The service from the zookeeper official image\r\n    version 3.4.12\r\n    image：https://hub.docker.com/_/zookeeper/\r\n    Dockerfile：https://github.com/31z4/zookeeper-docker/blob/0e558d7a6df4031a6be7c1df4ba1e2367666d004/3.4.12/Dockerfile\r\n    \r\n### hadoop\r\n    \r\n    The service from the hadoop image, if you use docker-compose-hadoop.yml for the first time,it automatically builds the image.\r\n    You can comment out the build option to direct to use image.\r\n    version 2.7.6\r\n    image：https://hub.docker.com/r/yinfuyuan/hadoop/\r\n    Dockerfile：https://github.com/yinfuyuan/docker-bigdata/blob/master/services/hadoop/2.7/Dockerfile\r\n    \r\n    After you up the service you can use the host machine ip:50070 view the\r\n    hdfs and use the host machine ip:8088 view yarn\r\n    \r\n    Environment Variables\r\n    HADOOP_MASTER\r\n    HDFS_MASTER\r\n    YARN_MASTER\r\n    HISTORY_MASTER\r\n    HDFS_FS_DEFAULTFS\r\n    HDFS_SLAVES\r\n    HDFS_DFS_REPLICATION\r\n    HDFS_DFS_NAMENODE_SECONDARY_HTTP_ADDRESS\r\n    YARN_RESOURCEMANAGER_HOSTNAME\r\n    \r\n### hbase\r\n    \r\n**The service depends on zookeeper service, Start the zookeeper service before starting.**\r\n\r\n    The service from the hbase image, if you use docker-compose-hbase.yml for the first time,it automatically builds the image.\r\n    You can comment out the build option to direct to use image.\r\n    version 2.0.0\r\n    image：https://hub.docker.com/r/yinfuyuan/hbase/\r\n    Dockerfile：https://github.com/yinfuyuan/docker-bigdata/blob/master/services/hbase/2.0/Dockerfile\r\n    \r\n    After you up the service you can use the host machine ip:16010 view the hbase.\r\n    \r\n### kafka\r\n    \r\n**The service depends on zookeeper service, Start the zookeeper service before starting.**\r\n    \r\n    The service from the kafak image, if you use docker-compose-kafka.yml for the first time,it automatically builds the image.\r\n    You can comment out the build option to direct to use image.\r\n    version 1.1.0\r\n    image：https://hub.docker.com/r/yinfuyuan/kafka/\r\n    Dockerfile：https://github.com/yinfuyuan/docker-bigdata/blob/master/services/kafka/Dockerfile","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyinfuyuan%2Fdocker-bigdata","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyinfuyuan%2Fdocker-bigdata","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyinfuyuan%2Fdocker-bigdata/lists"}