{"id":15904757,"url":"https://github.com/dmitryduev/penquins","last_synced_at":"2025-09-11T22:31:27.536Z","repository":{"id":46204986,"uuid":"265066748","full_name":"dmitryduev/penquins","owner":"dmitryduev","description":"A python client for Kowalski","archived":false,"fork":false,"pushed_at":"2024-06-25T09:39:45.000Z","size":377,"stargazers_count":6,"open_issues_count":3,"forks_count":8,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-10-06T12:41:58.866Z","etag":null,"topics":["kowalski","python-client","ztf","ztf-ii"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmitryduev.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-18T21:29:17.000Z","updated_at":"2024-09-18T22:03:04.000Z","dependencies_parsed_at":"2024-04-01T23:31:11.537Z","dependency_job_id":"1cc269ff-24a8-466c-a29d-7430237811b8","html_url":"https://github.com/dmitryduev/penquins","commit_stats":{"total_commits":28,"total_committers":5,"mean_commits":5.6,"dds":0.4285714285714286,"last_synced_commit":"3de7d8fc9959ecafe9cb0f24401b3751f7b203b3"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitryduev%2Fpenquins","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitryduev%2Fpenquins/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitryduev%2Fpenquins/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmitryduev%2Fpenquins/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmitryduev","download_url":"https://codeload.github.com/dmitryduev/penquins/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232666179,"owners_count":18557991,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kowalski","python-client","ztf","ztf-ii"],"created_at":"2024-10-06T12:41:47.494Z","updated_at":"2025-01-06T03:05:33.956Z","avatar_url":"https://github.com/dmitryduev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# penquins: a python client for [Kowalski](https://github.com/skyportal/kowalski)\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5651471.svg)](https://doi.org/10.5281/zenodo.5651471)\n\n`penquins` is a python client for [Kowalski](https://github.com/skyportal/kowalski), a multi-survey data archive and alert broker for time-domain astronomy.\n\n## Quickstart\n\nInstall `penquins` from [PyPI](https://pypi.org/project/penquins/):\n\n```bash\npip install penquins --upgrade\n```\n\nConnect to a Kowalski instance:\n\n```python\nfrom penquins import Kowalski\n\nusername = \"\u003cusername\u003e\"\npassword = \"\u003cpassword\u003e\"\n\nprotocol, host, port = \"https\", \"\u003chost\u003e\", 443\n\nkowalski = Kowalski(\n    username=username,\n    password=password,\n    protocol=protocol,\n    host=host,\n    port=port\n)\n```\n*When connecting to only one instance, it will be labeled as \"default\". Keep this in mind when retrieving the results of your queries.*\n\nConnect to multiple Kowalski instances:\n\n```python\nfrom penquins import Kowalski\n\ninstances = {\n    \"kowalski\": {\n        \"name\": \"kowalski\",\n        \"host\": \"\u003chost\u003e\",\n        \"protocol\": \"https\"\n        \"port\": 443,\n        \"token\": \"\u003ctoken\u003e\" # or username and password\n    },\n    ...\n}\n\nkowalski = Kowalski(instances=instances)\n```\n\n*When using multiple instances at once, you can specify a single instance to query using its name when calling `query(name=...)`, or no name at all. If no name is provided and the catalog(s) being queried is/are available on multiple instances, penquins will divide the load between instances automagically.*\n\n*When retrieving the results, you'll have to use the instance(s) name instead of \"default\", or simply iterate over the results by instance and merge the results.*\n\n\nIt is recommended to authenticate once and then just reuse the generated token:\n\n```python\ntoken = kowalski.token\nprint(token)\n\nkowalski = Kowalski(\n    token=token,\n    protocol=protocol,\n    host=host,\n    port=port\n)\n```\n\nCheck connection:\n\n```python\nkowalski.ping()\n```\n\n### Querying a Kowalski instance\n\nMost users will be interacting with Kowalski using the `Kowalski.query` method.\n\nRetrieve available catalog names:\n\n```python\nquery = {\n    \"query_type\": \"info\",\n    \"query\": {\n        \"command\": \"catalog_names\",\n    }\n}\n\nresponse = kowalski.query(query=query)\ndata = response.get(\"default\").get(\"data\")\n```\n\nQuery for 7 nearest sources to a sky position, sorted by the spheric distance, with a `near` query:\n\n```python\nquery = {\n    \"query_type\": \"near\",\n    \"query\": {\n        \"max_distance\": 2,\n        \"distance_units\": \"arcsec\",\n        \"radec\": {\"query_coords\": [281.15902595, -4.4160933]},\n        \"catalogs\": {\n            \"ZTF_sources_20210401\": {\n                \"filter\": {},\n                \"projection\": {\"_id\": 1},\n            }\n        },\n    },\n    \"kwargs\": {\n        \"max_time_ms\": 10000,\n        \"limit\": 7,\n    },\n}\n\nresponse = kowalski.query(query=query)\ndata = response.get(\"default\").get(\"data\")\n```\n\nRetrieve available catalog names:\n\n```python\nquery = {\n    \"query_type\": \"info\",\n    \"query\": {\n        \"command\": \"catalog_names\",\n    }\n}\n\nresponse = k.query(query=query)\ndata = response.get(\"default\").get(\"data\")\n```\n\nQuery for 7 nearest sources to a sky position, sorted by the spheric distance, with a `near` query:\n\n```python\nquery = {\n    \"query_type\": \"near\",\n    \"query\": {\n        \"max_distance\": 2,\n        \"distance_units\": \"arcsec\",\n        \"radec\": {\"query_coords\": [281.15902595, -4.4160933]},\n        \"catalogs\": {\n            \"ZTF_sources_20210401\": {\n                \"filter\": {},\n                \"projection\": {\"_id\": 1},\n            }\n        },\n    },\n    \"kwargs\": {\n        \"max_time_ms\": 10000,\n        \"limit\": 7,\n    },\n}\n\nresponse = k.query(query=query)\ndata = response.get(\"default\").get(\"data\")\n```\n\nRun a `cone_search` query:\n\n```python\nquery = {\n    \"query_type\": \"cone_search\",\n    \"query\": {\n        \"object_coordinates\": {\n            \"cone_search_radius\": 2,\n            \"cone_search_unit\": \"arcsec\",\n            \"radec\": {\n                \"ZTF20acfkzcg\": [\n                    115.7697847,\n                    50.2887778\n                ]\n            }\n        },\n        \"catalogs\": {\n            \"ZTF_alerts\": {\n                \"filter\": {},\n                \"projection\": {\n                    \"_id\": 0,\n                    \"candid\": 1,\n                    \"objectId\": 1\n                }\n            }\n        }\n    },\n    \"kwargs\": {\n        \"filter_first\": False\n    }\n}\n\nresponse = kowalski.query(query=query)\ndata = response.get(\"default\").get(\"data\")\n```\n\nRun a `find` query:\n\n```python\nq = {\n    \"query_type\": \"find\",\n    \"query\": {\n        \"catalog\": \"ZTF_alerts\",\n        \"filter\": {\n            \"objectId\": \"ZTF20acfkzcg\"\n        },\n        \"projection\": {\n            \"_id\": 0,\n            \"candid\": 1\n        }\n    }\n}\n\nresponse = kowalski.query(query=q)\ndata = response.get(\"default\").get(\"data\")\n```\n\nRun a batch of queries in parallel:\n\n```python\nqueries = [\n    {\n        \"query_type\": \"find\",\n        \"query\": {\n            \"catalog\": \"ZTF_alerts\",\n            \"filter\": {\n                \"candid\": alert[\"candid\"]\n            },\n            \"projection\": {\n                \"_id\": 0,\n                \"candid\": 1\n            }\n        }\n    }\n    for alert in data\n]\n\nresponses = k.query(queries=queries, use_batch_query=True, max_n_threads=4)\n```\n\n### Querying multiple instances at once\n\nWhen using multiple instances at once, you can specify a single instance to query using its name when calling `query(name=...)`, or no name at all. If no name is provided, and the catalog(s) being queried is/are available on multiple instances, penquins will divide the load between instances automagically.\n\nWhen retrieving the results, you'll have to use the instance(s) name instead of \"default\", or simply iterate over the results by instance and merge the results.\n\nAny of the queries mentioned for single instance querying also work here.\n\n#### Examples\n\nNo instance name specified:\n\n```python\nq = {\n    \"query_type\": \"find\",\n    \"query\": {\n        \"catalog\": \"ZTF_alerts\",\n        \"filter\": {\n            \"objectId\": \"ZTF20acfkzcg\"\n        },\n        \"projection\": {\n            \"_id\": 0,\n            \"candid\": 1\n        }\n    }\n}\nresponse = kowalski.query(query=q)\ndata = response.get(\u003cinstance_name).get(\"data\") # retrieving data from one instance\n\n# OR\n\ndata = [] # or {} depending on the query's expected result, differs by query type\nfor instance, instance_results in response.items():\n    for result in instance_results:\n        data.append(result.get('data'))\n```\n\nInstance name specified:\n```python\nq = {\n    \"query_type\": \"find\",\n    \"query\": {\n        \"catalog\": \"ZTF_alerts\",\n        \"filter\": {\n            \"objectId\": \"ZTF20acfkzcg\"\n        },\n        \"projection\": {\n            \"_id\": 0,\n            \"candid\": 1\n        }\n    }\n}\nresponse = kowalski.query(query=q, name=\u003cinstance_name\u003e)\ndata = response.get(\u003cinstance_name).get(\"data\") # retrieving data from one instance\n```\n\n### Interacting with the API\n\nUsers can interact with [Kowalski's API](https://kowalski.caltech.edu/docs/api/)\nin a more direct way using the `Kowalski.api` method.\n\nUsers with admin privileges can add/remove users to/from the system:\n\n```python\nusername = \"noone\"\npassword = \"nopas!\"\nemail = \"user@caltech.edu\"\n\nrequest = {\n  \"username\": username,\n  \"password\": password,\n  \"email\": email\n}\n\nresponse = kowalski.api(method=\"post\", endpoint=\"/api/users\", data=request)\n\nresponse = kowalski.api(method=\"delete\", endpoint=f\"/api/users/{username}\")\n```\n\n## Publish new version\n\nPlease refer to https://realpython.com/pypi-publish-python-package/\nfor a detailed guide.\n\n```shell script\npip install bumpversion\nexport PENQUINS_VERSION=2.4.2\n\nbumpversion --current-version $PENQUINS_VERSION minor setup.py penquins/penquins.py\npython setup.py sdist bdist_wheel\n\ntwine check dist/*$PENQUINS_VERSION*\ntwine upload dist/*$PENQUINS_VERSION*\n\nusername: __token__\ntoken: \u003cTOKEN\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmitryduev%2Fpenquins","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmitryduev%2Fpenquins","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmitryduev%2Fpenquins/lists"}