{"id":19746358,"url":"https://github.com/astrolabsoftware/fink-science-portal","last_synced_at":"2025-04-30T08:31:06.641Z","repository":{"id":37799915,"uuid":"299210900","full_name":"astrolabsoftware/fink-science-portal","owner":"astrolabsoftware","description":"Fink Science Portal","archived":false,"fork":false,"pushed_at":"2024-10-24T14:53:25.000Z","size":108503,"stargazers_count":7,"open_issues_count":80,"forks_count":4,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-10-25T02:49:17.365Z","etag":null,"topics":["apache-hbase","astronomy","dash","fink"],"latest_commit_sha":null,"homepage":"https://fink-portal.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/astrolabsoftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-28T06:30:59.000Z","updated_at":"2024-10-24T14:52:50.000Z","dependencies_parsed_at":"2023-02-16T20:31:49.338Z","dependency_job_id":"bf7341ec-e34b-4911-8b06-00badd23882a","html_url":"https://github.com/astrolabsoftware/fink-science-portal","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astrolabsoftware%2Ffink-science-portal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astrolabsoftware%2Ffink-science-portal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astrolabsoftware%2Ffink-science-portal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/astrolabsoftware%2Ffink-science-portal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/astrolabsoftware","download_url":"https://codeload.github.com/astrolabsoftware/fink-science-portal/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224202819,"owners_count":17272807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-hbase","astronomy","dash","fink"],"created_at":"2024-11-12T02:14:21.449Z","updated_at":"2024-11-12T02:14:22.175Z","avatar_url":"https://github.com/astrolabsoftware.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fink Science Portal\n\n[![Sentinel](https://github.com/astrolabsoftware/fink-science-portal/workflows/Sentinel/badge.svg)](https://github.com/astrolabsoftware/fink-science-portal/actions?query=workflow%3ASentinel)\n\n![fronted](assets/frontend.png)\n\nThe Fink Science Portal allows users to browse and display alert data collected and processed by Fink from a web browser: [https://fink-portal.org](https://fink-portal.org).\n\nThe backend is using [Apache HBase](https://hbase.apache.org/), a distributed non-relational database. The frontend is based on [Dash](https://plotly.com/dash/), a Python web framework built on top of Flask, Plotly and React. The frontend has also integrated components to perform fit on the data, such as [gatspy](https://www.astroml.org/gatspy/) for variable stars, [pyLIMA](https://github.com/ebachelet/pyLIMA) for microlensing, or the [imcce](https://ssp.imcce.fr/webservices/miriade/) tools for Solar System objects.\n\n## Backend structure\n\nAfter each observation night, the data is aggregated and pushed into Apache HBase tables. The main table contains all alert data processed by Fink since 2019-11-01. This represents more than 217 million alerts collected, and about 147 million scientifically valid (8.0 TB) as of 01/2024. The main table data is indexed along the `objectId` of alerts, and the emission date `jd`.\n\nIn order to allow multi-indexing with HBase, we create _index tables_. These tables are indexed along different properties (time, sky position, classification, ...). They contain the same number of rows than the main table but fewer columns. These index tables are used to perform fast search along arbitrary properties and isolate interesting candidates, while the main table is used to display final data.\n\nWe developed custom HBase clients to manipulate the data efficiently (Lomikel, FinkBrowser, more information [here](https://hrivnac.web.cern.ch/hrivnac/Activities/index.html)).\n\n## Tests\n\nYou can test the REST API using:\n\n```bash\n./run_tests.sh --url https://fink-portal.org\n```\n\nThe folder [tests](tests) contain many example on how to use the REST API.\n\n## Deployment\n\nThe portal has been tested on Python 3.11. Other versions might work.\n\n### Local deployment\n\nIf you want to deploy on your machine for test purposes, you can follow the [tutorial](docs/howto_deploy_locally.md). Note that a Dockerfile should be ready at some point.\n\n### Production\n\nThe frontend is host at the VirtualData cloud at Université Paris-Saclay, France. To deploy it, just edit `config.yml`.\n\n```python\nAPIURL: https://fink-portal.org\nIP: fink-portal.org\nPORT: 24000\nHBASEIP: hbase-1.lal.in2p3.fr\nZOOPORT: 2183\nSCHEMAVER: \"schema_3.1_5.0.0\"\ntablename: ztf\n```\n\nand the launch is supervised by gunicorn:\n\n```bash\ngunicorn index:server -b :24000 --workers=4\n```\n\nIn practice we also use a reverse-proxy (nginx).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fastrolabsoftware%2Ffink-science-portal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fastrolabsoftware%2Ffink-science-portal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fastrolabsoftware%2Ffink-science-portal/lists"}