{"id":28512153,"url":"https://github.com/opengeoscience/dataqs","last_synced_at":"2026-03-01T14:34:22.315Z","repository":{"id":145026972,"uuid":"43162399","full_name":"OpenGeoscience/dataqs","owner":"OpenGeoscience","description":"Classes and scheduled tasks to automate the import of regularly updated online spatial data into GeoNode.","archived":false,"fork":false,"pushed_at":"2018-10-15T14:43:41.000Z","size":3477,"stargazers_count":9,"open_issues_count":5,"forks_count":7,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-07-04T01:36:48.332Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenGeoscience.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2015-09-25T17:01:19.000Z","updated_at":"2024-12-07T04:15:00.000Z","dependencies_parsed_at":"2023-05-28T05:45:12.341Z","dependency_job_id":null,"html_url":"https://github.com/OpenGeoscience/dataqs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/OpenGeoscience/dataqs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGeoscience%2Fdataqs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGeoscience%2Fdataqs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGeoscience%2Fdataqs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGeoscience%2Fdataqs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenGeoscience","download_url":"https://codeload.github.com/OpenGeoscience/dataqs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenGeoscience%2Fdataqs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29970994,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T14:11:48.712Z","status":"ssl_error","status_checked_at":"2026-03-01T14:11:48.352Z","response_time":124,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-09T00:30:50.190Z","updated_at":"2026-03-01T14:34:22.307Z","avatar_url":"https://github.com/OpenGeoscience.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"=====\nData Queues\n=====\n\ndataqs (Data Queues) is a simple Django app to download, process,\nand import spatial data into GeoServer/GeoNode.\n\n\nQuick start\n-----------\n\n1. Add \"dataqs\" to your INSTALLED_APPS setting like this::\n\n    INSTALLED_APPS = (\n        ...\n        'dataqs',\n        'dataqs.forecastio',\n        'dataqs.gfms',\n        'dataqs.airnow',\n        'dataqs.wqp',\n        'dataqs.aqicn',\n        #etc\n    )\n    \n2. In your settings.py or local_settings.py file, add a CELERYBEAT_SCHEDULE\n   setting to specify when Celery should run data_queues tasks::\n   \n   \tfrom celery.schedules import crontab\n\tCELERYBEAT_SCHEDULE = {\n\t    'gfms': {\n\t        'task': 'dataqs.gfms.tasks.gfms_task',\n\t        'schedule': crontab(minute='3'),\n\t        'args': ()\n\t    },\n\t    'forecast_io': {\n\t        'task': 'dataqs.forecastio.tasks.forecast_io_task',\n\t        'schedule': crontab(minute='1'),\n\t        'args': ()\n\t    },\n\t        'task': 'dataqs.aqicn.tasks.aqicn_task',\n\t        'schedule': crontab(hour='*/6', minute='0'),\n\t        'args': ([],)\n\t    },\n\t}\n\n3. Also add the following settings::\n  \n\t#Location of GeoServer data directory\n\tGS_DATA_DIR = '/usr/share/geoserver/data'\n\n\t#Directory where temporary dataqs geoprocessing files should be downloaded\n\tGS_TMP_DIR = GS_DATA_DIR + '/tmp'\n\n\t#AirNow API username:password\n\t#(sign up for a free account at http://airnowapi.org/account/request/)\n\tAIRNOW_ACCOUNT = 'your_airnow_username:your_airnow_password'\n\n\t#NASA GPM FTP ACCOUNT\n\t#(sign up at http://registration.pps.eosdis.nasa.gov/registration/)\n\tGPM_ACCOUNT = 'your_gpm_email_account'\n\n\t#HIFLD: Dictionary of layers to process in the form of:\n\tHIFLD_LAYERS = [\n\t    {\n            'name': 'US State Boundaries',\n            'url': 'https://hifld-dhs-gii.opendata.arcgis.com/datasets/718791120f6549708cb642dac6ff0dbf_0.geojson',\n            'table': 'state_boundaries',\n            'sld': 'polygon'\n        },\n        {\n            'name': 'Cities and Towns NTAD',\n            'url': 'https://hifld-dhs-gii.opendata.arcgis.com/datasets/6a1e404a10754e59bac4bfa50db3f487_0.geojson',\n            'table': 'cities_towns',\n            'sld': 'point'\n        },\n        {\n            'name': 'Roads and Railroad Tunnels',\n            'url': 'https://hifld-dhs-gii.opendata.arcgis.com/datasets/2f6abb736360437ba363e0a1210b4d36_0.geojson',\n            'table': 'roads_tunnels',\n            'sld': 'line'\n        }\n\t]\n\n\t#Time to wait before updating Geoserver mosaic (keep at 0 unless Geoserver\n\t#is on a different server. In that case, there will need to be an automated\n\t#rsync between GS_TMP_DIR where celery is running and\n\t#GS_DATA_DIR where GeoServer is running.\n\tRSYNC_WAIT_TIME = 0\n\n4. In order to run the spei processor, the following must be installed::\n\n    sudo apt-get install netcdf-bin\n    sudo apt-get install cdo\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopengeoscience%2Fdataqs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopengeoscience%2Fdataqs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopengeoscience%2Fdataqs/lists"}