{"id":27881882,"url":"https://github.com/src-d/dev-similarity","last_synced_at":"2026-03-06T10:32:26.308Z","repository":{"id":79453997,"uuid":"104088589","full_name":"src-d/dev-similarity","owner":"src-d","description":null,"archived":false,"fork":false,"pushed_at":"2019-09-18T11:18:42.000Z","size":713,"stargazers_count":11,"open_issues_count":0,"forks_count":6,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-09-16T16:59:08.297Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/src-d.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-09-19T14:51:55.000Z","updated_at":"2020-06-21T05:38:41.000Z","dependencies_parsed_at":"2023-04-23T12:31:38.106Z","dependency_job_id":null,"html_url":"https://github.com/src-d/dev-similarity","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/src-d/dev-similarity","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fdev-similarity","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fdev-similarity/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fdev-similarity/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fdev-similarity/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/src-d","download_url":"https://codeload.github.com/src-d/dev-similarity/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/src-d%2Fdev-similarity/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30171869,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T07:56:45.623Z","status":"ssl_error","status_checked_at":"2026-03-06T07:55:55.621Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-05T05:06:01.433Z","updated_at":"2026-03-06T10:32:26.288Z","avatar_url":"https://github.com/src-d.png","language":"Jupyter Notebook","readme":"# Developer Similarity\n\nThis is a legacy from [source{d}](http://www.sourced.tech/)'s research in developer similarity when its main goal was to [match developers and companies](http://web.archive.org/web/20161020170426/http://sourced.tech/).\nWe were obsessed with developer embeddings and recommending similar ones. Not everything we created\nis included here. For now, it's the profile images to visually compare the experience.\n\nsource{d} runs a massive-scale data retrieval pipeline that among other sources nearly mirrors all GitHub repositories.\nEverything related to working with developer metadata requires understanding which email addresses correspond\nto which GitHub users. [GitHub ToU](https://help.github.com/articles/github-terms-of-service/) directly forbids to use the API for recruitment purposes, which at the time we had, so there was no easy way to obtain that information.\n\nLuckily, we were able to perform identity matching using various heuristics for Git commit signatures. For example, given two different signatures with the same name in the same repository, it is likely that the different email addresses belong to the same user.\nWe never measured the accuracy of our heuristics - back then we were a humble startup - but our CTO claims that it worked reasonably well in practice.\n\nGiven the [topic modeling results](https://arxiv.org/abs/1704.00135), we were able to cluster 6 millions\nof filtered repositories with K-means algorithm ([src-d/kmcuda](https://github.com/src-d/kmcuda)) into a\nreasonable number of clusters, in our particular case 2916. We arranged those clusters using t-SNE\nand mapped into a regular 54x54 = 2916 grid and described the latter in the [blog post](https://blog.sourced.tech/post/lapjv/).\nThus every repository appears in a single specific cell of that grid.\n\nGiven the identity matching, we had the contribution history and could even roughly estimate the impact as the ratio of\ncontributed commits to the whole number of commits in the main branch of a project. We color the corresponding cell for each weighted contribution by the chosen user and compose the profile image.\n\nThe complete pipeline is in the [legacy notebook](legacy/from_scratch.ipynb) which can not be directly executed without the datasets and the last step which draws the images is fully functional and you can [play with it](profile.ipynb).\n\nAs an example, this is how our CTO [Máximo Cuadros](https://github.com/mcuadros) developer fingerprint looks like under this approach:\n\n![Máximo's profile](máximo.png)\n\nWe see PHP trace in the upper right area and Go trace in the lower right.\n\nThis is [Fabien Potencier](https://github.com/fabpot), the man behind [Symfony](https://symfony.com/) PHP framework:\n\n![Fabien's profile](fabien.png)\n\nWe see the strong PHP trace. Thus looking on both profiles we see that Máximo and Fabien have something in common -\nthey both have experience in PHP.\n\nThis is [Russ Cox](https://github.com/rsc), one of the creators of [Go](https://golang.org/) language:\n\n![Russ' profile](russ.png)\n\nWe see the strong Go trace. Thus looking on his and Máximo's profiles we see that they have something in common -\nthey both have experience in Go.\n\nAt the same time, the profile images are not limited to languages. It just happens so that each language has some\ncommon usage patterns and people with much experience follow all of them. For example, this is\n[François Chollet](https://github.com/fchollet), the author of [Keras](https://keras.io/),\nthe deep learning Python library:\n\n![François' profile](françois.png)\n\nJust a few pixels. In contrast, this is [Armin Ronacher](https://github.com/mitsuhiko),\na well known universal Python developer:\n\n![Armin' profile](armin.png)\n\nThere is an expressed Python trace + web development. We see that François always contributes very specific things.\nAt the same time, Python deep learning pixels reside in the cloud of Python related projects.\n\n\n### Docker image\n\n```\ndocker run -it --rm -p 8888:8888 srcd/dev-similarity\n```\n\nThen open the last printed link in your web browser.\n\n### License\n\nApache 2.0.\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrc-d%2Fdev-similarity","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsrc-d%2Fdev-similarity","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsrc-d%2Fdev-similarity/lists"}