{"id":20332233,"url":"https://github.com/t1m4/starwars_api","last_synced_at":"2026-05-14T07:35:03.587Z","repository":{"id":168882144,"uuid":"394910541","full_name":"t1m4/starwars_api","owner":"t1m4","description":"Django Starwars API interview project(Django + Aiohttp)","archived":false,"fork":false,"pushed_at":"2022-12-29T08:09:31.000Z","size":79,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-14T15:40:41.557Z","etag":null,"topics":["aiohttp","csv","django","django-rest-framework","petl","pytest-django"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/t1m4.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-11T07:58:11.000Z","updated_at":"2022-12-26T10:19:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"21a32f85-922d-46b9-9ac0-66e3a49fbb3d","html_url":"https://github.com/t1m4/starwars_api","commit_stats":null,"previous_names":["t1m4/starwars_api"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/t1m4%2Fstarwars_api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/t1m4%2Fstarwars_api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/t1m4%2Fstarwars_api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/t1m4%2Fstarwars_api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/t1m4","download_url":"https://codeload.github.com/t1m4/starwars_api/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241851372,"owners_count":20030963,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiohttp","csv","django","django-rest-framework","petl","pytest-django"],"created_at":"2024-11-14T20:25:37.809Z","updated_at":"2026-05-14T07:34:58.565Z","avatar_url":"https://github.com/t1m4.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Starwars API interview project\n\n### Technical requirements\n\n1. The user should have a way to download the latest  complete  dataset of characters from the API by clicking on a button, the collected and transformed data should be stored as a CSV file in the file system. Metadata for downloaded datasets (e.g. filename, date, etc.) should be stored inside the database. Fetching and transformations should be implemented  efficiently  , minimize the amount of requests, your app should be able to process  large amounts of data  .\n2. Add a  date column ( %Y-%m-%d ) based on edited date\nResolve the  homeworld field into the homeworld's name ( /planets/1/ -\u003e\nTatooine )\nFields referencing different resources and date fields other than  date/birth_year\ncan be dropped\n3. The user should be able to inspect all previously downloaded datasets, as well as do simple\nexploratory operations on it.\n4. By default the table should only show the first 10 rows of the dataset, by clicking on a button\n“ Load more  ” additionally 10 rows should be shown - reloading the page is fine.\n5. Provide the functionality to count the occurrences of values (combination of values) for columns.\nFor example when selecting the columns  date and  homeworld the table should show the\ncounts as follows:\n\n### Tools\n\n- Django and DRF \n- Celery and Redis\n- [petl](https://petl.readthedocs.io/en/stable/ \"Petl library\")\n- pytest (WIP)\n- Asincio, aiohttp and httpx(WIP)\n- Amazon S3 or Minio(WIP)\n\n\n### Run using shell\n1. Create virtual envinronment\n    ```shell\n    python3.9 -m venv venv\n    ```\n2. Activate virtual envinronment\n    ```shell\n    source /venv/bin/activate\n    ```\n3. Install requirements\n    ```shell\n    pip install -r requirements.txt\n    ```\n4. Create .env file base on .env.dev\n5. Export environment variables in your current shell\n    ```shell\n    source docker/export.sh\n    ```\n6. Start Django application \n    ```shell\n    ./docker/start.sh\n    ``` \n\n### Run using docker\n1. Run application\n    ```shell\n    docker-compose up app\n    ```\n2. Start flower to check celery tasks\n    ```shell\n    docker-compose up flower\n    ```\n\n### Aiohttp client\nAiohttp AsyncAPIClient in the file [external_api/starwars_api/api.py](https://github.com/t1m4/starwars_api/blob/master/external_api/starwars_api/api.py)\n\nTips:\n- After timeout, pending tasks will be canceled inside execute() function.\n- Retry tasks. Just save retry_tasks_list and wait() them again. \n\nThe old sync version of API can be found in [feature/sync_api](https://github.com/t1m4/starwars_api/tree/feature/sync_api) branch\n\n### WIP\n\n1. Write more tests using pytest \n2. Read and write very large files using custom indexes and .seek()\n3. Add documentation using sphinx\n4. Minio or AWS S3 integration for saving files","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ft1m4%2Fstarwars_api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ft1m4%2Fstarwars_api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ft1m4%2Fstarwars_api/lists"}