{"id":27203458,"url":"https://github.com/miliar/tech-stack-demand","last_synced_at":"2025-07-14T03:03:25.255Z","repository":{"id":71598051,"uuid":"203959297","full_name":"miliar/tech-stack-demand","owner":"miliar","description":"An example microservice pipeline to visualize the tech stack demand on the current job market","archived":false,"fork":false,"pushed_at":"2019-09-22T14:27:53.000Z","size":17693,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-09T22:45:03.530Z","etag":null,"topics":["apache-kafka","data-pipeline","docker","docker-compose","example","flask","html-css-javascript","indeed","jinja2","jobsearch","jobseeker","kafka","microservices","neo4j","python","python3","redis","scraper","stackexchange","tech-stacks"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miliar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-23T08:40:44.000Z","updated_at":"2024-09-16T19:38:26.000Z","dependencies_parsed_at":"2023-02-23T10:01:35.027Z","dependency_job_id":null,"html_url":"https://github.com/miliar/tech-stack-demand","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/miliar/tech-stack-demand","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miliar%2Ftech-stack-demand","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miliar%2Ftech-stack-demand/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miliar%2Ftech-stack-demand/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miliar%2Ftech-stack-demand/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miliar","download_url":"https://codeload.github.com/miliar/tech-stack-demand/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miliar%2Ftech-stack-demand/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265236741,"owners_count":23732493,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-kafka","data-pipeline","docker","docker-compose","example","flask","html-css-javascript","indeed","jinja2","jobsearch","jobseeker","kafka","microservices","neo4j","python","python3","redis","scraper","stackexchange","tech-stacks"],"created_at":"2025-04-09T22:39:12.839Z","updated_at":"2025-07-14T03:03:25.190Z","avatar_url":"https://github.com/miliar.png","language":"Python","readme":"# Tech stack demand\nAn example microservice pipeline to visualize the tech stack demand on the current job market.\n\n#### Show who is using certain technologies:\n![Tech stack demand tech demo](demo_pictures/search_by_tech.gif)\n\n\n#### Search for company:\n![Tech stack demand company demo](demo_pictures/search_by_company.gif)\n\n\n## Installation\n* [Install docker/docker-compose](https://docs.docker.com/compose/install/)\n* Assure docker has enough resources (Preferences \u003e Advanced; tested with 4 CPUs and 6 GiB RAM) \n* Run `docker-compose up`\n* Open in your browser http://localhost:5000/\n* Collect new data (reload page to see progress; this can take up to 20 min)\n![Tech stack demand collect demo](demo_pictures/collect_new_data.gif)\n\n\n## How to use the UI\n* Check out this [basic action wiki](https://github.com/Nhogs/popoto/wiki/Basic-action)\n\n\n## Technical sidenotes\n### Architecture \n![Tech stack demand architecture](demo_pictures/architecture.png)\n\n\n#### Browse kafka topics:\n* Kafka logs are not mounted\n* While collecting new data, check http://localhost:9000/\n\n\n#### Manage Neo4j data:\n* Data mounted to /neo4j_data_loader/neo4j_data\n* Open Neo4j browser: http://localhost:7474/\n* Login with user: neo4j password: password\n\n\n#### Run tests:\n* `docker-compose -f docker-compose-tests.yml up`\n\n\n#### Run keywords api update:\n* Delete old data saved in `keywords_api/redis_data`\n* Run: `docker-compose -f docker-compose-keywords-update.yml up`\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiliar%2Ftech-stack-demand","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiliar%2Ftech-stack-demand","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiliar%2Ftech-stack-demand/lists"}