{"id":24588003,"url":"https://github.com/kemalcanbora/foo_scientist","last_synced_at":"2026-04-05T08:35:57.079Z","repository":{"id":74559252,"uuid":"207587548","full_name":"kemalcanbora/foo_scientist","owner":"kemalcanbora","description":null,"archived":false,"fork":false,"pushed_at":"2019-09-13T14:59:58.000Z","size":9,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-24T06:42:26.731Z","etag":null,"topics":["big-data","json","pyspark","python","sh","shell","spark"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kemalcanbora.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-09-10T14:52:40.000Z","updated_at":"2019-09-20T17:35:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"1538b2c9-1163-4262-aa4b-c7dff6bded4f","html_url":"https://github.com/kemalcanbora/foo_scientist","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kemalcanbora/foo_scientist","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kemalcanbora%2Ffoo_scientist","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kemalcanbora%2Ffoo_scientist/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kemalcanbora%2Ffoo_scientist/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kemalcanbora%2Ffoo_scientist/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kemalcanbora","download_url":"https://codeload.github.com/kemalcanbora/foo_scientist/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kemalcanbora%2Ffoo_scientist/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31430009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T08:13:15.228Z","status":"ssl_error","status_checked_at":"2026-04-05T08:13:11.839Z","response_time":75,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","json","pyspark","python","sh","shell","spark"],"created_at":"2025-01-24T07:13:54.309Z","updated_at":"2026-04-05T08:35:57.070Z","avatar_url":"https://github.com/kemalcanbora.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Foo Scientist\n\nApache Spark follows a master/slave architecture with two main daemons\n  - Master Daemon — (Master/Driver Process)\n  - Worker Daemon –(Slave Process)\n\nCluster Manager\n![alt text](https://miro.medium.com/max/534/1*o80WzWJShjCV1RSfWzCiQA.png)\n\nSometimes architecture can be hard to build for this there are solutions such as cloudera etc. I wanted to do it my way.\nAll you have to do is edit the json file yourself it's name is **architecture.json** and run mimar.py\n\n\u003ch2\u003eNote:\u003c/h2\u003e Don't change type in json file and if you add another slave in json file just add new number end of the name like \nhadoop_slave_2, hadoop_slave_3 .. \n\n\u003ch2\u003eJSON File\u003c/h2\u003e\n\u003cpre\u003e\n[{\n  \"type\":\"hadoop-master\",\n  \"password\":\"123456\",\n  \"IP\": \"10.11.2.183\",\n  \"username\": \"root\",\n  \"name\": \"hadoop_master_mimar\"\n},\n{\n  \"type\":\"hadoop-slave\",\n  \"password\":\"123456\",\n  \"IP\": \"10.11.2.228\",\n  \"username\": \"root\",\n  \"name\": \"hadoop_slave_1\"\n}]\n\u003c/pre\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkemalcanbora%2Ffoo_scientist","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkemalcanbora%2Ffoo_scientist","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkemalcanbora%2Ffoo_scientist/lists"}