{"id":18447405,"url":"https://github.com/paperswithcode/paperswithcode-client","last_synced_at":"2025-04-06T01:08:17.804Z","repository":{"id":51261517,"uuid":"283026422","full_name":"paperswithcode/paperswithcode-client","owner":"paperswithcode","description":"API Client for paperswithcode.com","archived":false,"fork":false,"pushed_at":"2024-05-10T02:25:48.000Z","size":101,"stargazers_count":162,"open_issues_count":9,"forks_count":24,"subscribers_count":10,"default_branch":"develop","last_synced_at":"2025-03-30T00:07:43.358Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paperswithcode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-27T21:58:46.000Z","updated_at":"2025-03-18T03:41:44.000Z","dependencies_parsed_at":"2024-06-21T17:37:35.453Z","dependency_job_id":"210df6f7-6474-443c-9aff-47ab102d1c99","html_url":"https://github.com/paperswithcode/paperswithcode-client","commit_stats":{"total_commits":51,"total_committers":4,"mean_commits":12.75,"dds":"0.43137254901960786","last_synced_commit":"70bd7ee5157fcaeff39145d2e03cc9ed5bb7421b"},"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paperswithcode%2Fpaperswithcode-client","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paperswithcode%2Fpaperswithcode-client/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paperswithcode%2Fpaperswithcode-client/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paperswithcode%2Fpaperswithcode-client/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paperswithcode","download_url":"https://codeload.github.com/paperswithcode/paperswithcode-client/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247419860,"owners_count":20936012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T07:13:16.355Z","updated_at":"2025-04-06T01:08:17.791Z","avatar_url":"https://github.com/paperswithcode.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# paperswithcode.com API client\n\nThis is a client for [PapersWithCode](https://paperswithcode.com/api/v1/docs/)\nread/write API.\n\nThe API is completely covered by the client and it wraps all the API models\ninto python objects and communicates with the API by getting and passing those\nobjects from and to the api client.\n\nDocumentation can be found on the\n[ReadTheDocs](https://paperswithcode-client.readthedocs.io/en/latest/) website.\n\nIt is published to the\n[Python Package Index](https://pypi.org/project/paperswithcode-client/) and\ncan be installed by simply calling `pip install paperswithcode-client`.\n\n## Quick usage example\n\nTo install:\n\n```bash\npip install paperswithcode-client\n```\n\nTo list papers indexed on Papers with Code:\n\n```python\n\nfrom paperswithcode import PapersWithCodeClient\n\nclient = PapersWithCodeClient()\npapers = client.paper_list()\nprint(papers.results[0])\nprint(papers.next_page)\n```\n\nFor full docs please see our [ReadTheDocs](https://paperswithcode-client.readthedocs.io/en/latest/) page.\n\n## How to mirror your competition\n\nPapers with Code offers a mirroring service for ongoing competitions that allows competition administrators\nto automatically upload the results to Papers with Code using an API. \n\nTo use the API in the write mode you'll need to first [obtain an API token](https://paperswithcode.com/accounts/generate_api_token).\n\nUsing the API token you'll be able to use the client in write mode:\n\n```python\nfrom paperswithcode import PapersWithCodeClient\n\nclient = PapersWithCodeClient(token=\"your_secret_api_token\")\n```\n\nTo mirror a live competition, you'll need to make sure the corresponding task (e.g. \"Image Classification\") \nexists on Papers with Code. You can use the search to check if it exists, and if it doesn't, you can add a \nnew task on the [Task addition page](https://paperswithcode.com/add/task). \n\nIf you cannot find your dataset on the website, you can create it with the API like this:\n\n```python\nfrom paperswithcode.models.dataset import *\nclient.dataset_add(\n    DatasetCreateRequest(\n        name=\"VeryTinyImageNet\",\n    )\n)\n```\n\nNow we are ready to programatically create the competition on Papers with Code. Here is an example of how we would do\nthis on a fictional VeryTinyImageNet dataset.\n\n```python\nfrom paperswithcode import PapersWithCodeClient\nfrom paperswithcode.models.evaluation.synchronize import *\n\nclient = PapersWithCodeClient(token=\"your_secret_api_token\")\n\nr = EvaluationTableSyncRequest(\n    task=\"Image Classification\",\n    dataset=\"VeryTinyImageNet\",\n    description=\"Optional description of your challenge in markdown format\",\n    metrics=[\n        MetricSyncRequest(\n            name=\"Top 1 Accuracy\",\n            is_loss=False,\n        ),\n        MetricSyncRequest(\n            name=\"Top 5 Accuracy\",\n            is_loss=False,\n        )\n    ],\n    results=[\n        ResultSyncRequest(\n            metrics={\n                \"Top 1 Accuracy\": \"85\",\n                \"Top 5 Accuracy\": \"95\"\n            },\n            paper=\"\",\n            methodology=\"My Unpublished Model Name\",\n            external_id=\"competition-submission-id-4321\",\n            evaluated_on=\"2020-11-20\",\n            external_source_url=\"https://my.competition.com/leaderboard/entry1\"\n        ),\n        ResultSyncRequest(\n            metrics={\n                \"Top 1 Accuracy\": \"75\",\n                \"Top 5 Accuracy\": \"81\"\n            },\n            paper=\"https://arxiv.org/abs/1512.03385\",\n            methodology=\"ResNet-50 (baseline)\",\n            external_id=\"competition-submission-id-1123\",\n            evaluated_on=\"2020-09-20\",\n            external_source_url=\"https://my.competition.com/leaderboard/entry2\"\n        )\n    ]\n)\n\nclient.evaluation_synchronize(r)\n```\nThis is going to add two entries to the leaderboard, a `ResNet-50` baseline that is referenced by the provided \narXiv paper link, and an unpublished entry for model `My Unpublished Model Name`. \n\nTo decompose it a bit more:\n\n```python\nmetrics=[\n    MetricSyncRequest(\n        name=\"Top 1 Accuracy\",\n        is_loss=False,\n    ),\n    MetricSyncRequest(\n        name=\"Top 5 Accuracy\",\n        is_loss=False,\n    )\n],\n```\n\nThis defines two global metrics that are going to be used in the leaderboard. The table will be ranked based on the \nfirst provided metric. The paramter `is_loss` indicates if the metric is a loss metric, i.e. if smaller-is-better. \nSince in this case both are accuracy metric where higher-is-better, we set `is_loss=False` which will produce the\ncorrect sorting order in the table. \n\nAn individual row in the leaderboard is represented by:\n\n```python\nResultSyncRequest(\n    metrics={\n        \"Top 1 Accuracy\": \"85\",\n        \"Top 5 Accuracy\": \"95\"\n    },\n    paper=\"\",\n    methodology=\"My Unpublished Model Name\",\n    external_id=\"competition-submission-id-4321\",\n    evaluated_on=\"2020-11-20\",\n    external_source_url=\"https://my.competition.com/leaderboard/entry1\"\n)\n```\n\nMetrics is simply a dictionary of metric values for each of the global metrics. The `paper` parameter can be a link\nto an arXiv paper, conference paper, or a paper page on Papers with Code. Any code that's associated with the paper\nwill be linked automatically. The `methodology` parameter should contain\nthe model name that is informative to the reader. `external_id` is your ID of this submission - this ID should be\nunqiue and is used when you make repeated calls to merge results if they changed. `evaluated_on` is the date in `YYYY-MM-DD`\nformat on which the method was evaluated on - we use this to create progress graphs. Finally, `external_source_url` \nis the URL to your website, ideally linking back to this individual entry. This will be linked in the \"Result\" column\nof the leaderboard and will enable users to navigate back to your website. \n\nFinally, this line of code:\n\n```python\nclient.evaluation_synchronize(r)\n```\n\nThis will execute the request on our API and will return you the ID of your leaderboard on Papers with Code. \nYou can then access it by going to `https://paperswithcode.com/sota/\u003cyour_leaderboard_id\u003e` or find it using \nthe site search.  \n\nTo keep your Papers with Code leaderboard in sync, you can simply re-post all the entries in your competition\non regular intervals. If a row already exists, it will be merged and no duplicates will be created. \n\nFor in-depth API docs please refer to our [ReadTheDocs](https://paperswithcode-client.readthedocs.io/en/latest/) page.\n\nBy using the API you agree that any competition data you submit will be licenced under [CC-BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/).\n\nIf you need any help contact us on hello@paperswithcode.com. \n\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaperswithcode%2Fpaperswithcode-client","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaperswithcode%2Fpaperswithcode-client","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaperswithcode%2Fpaperswithcode-client/lists"}