{"id":13482366,"url":"https://github.com/propublica/django-collaborative","last_synced_at":"2025-07-30T16:36:10.749Z","repository":{"id":36112373,"uuid":"180206287","full_name":"propublica/django-collaborative","owner":"propublica","description":"ProPublica's collaborative tip-gathering framework. Import and manage CSV, Google Sheets and Screendoor data with ease.","archived":false,"fork":false,"pushed_at":"2023-01-27T14:00:59.000Z","size":9279,"stargazers_count":100,"open_issues_count":9,"forks_count":18,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-06-11T20:40:59.398Z","etag":null,"topics":["csv-import","django","google-sheets","journalism","screendoor"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/propublica.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-08T18:10:20.000Z","updated_at":"2025-03-07T17:18:10.000Z","dependencies_parsed_at":"2023-02-15T10:00:50.603Z","dependency_job_id":null,"html_url":"https://github.com/propublica/django-collaborative","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/propublica/django-collaborative","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propublica%2Fdjango-collaborative","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propublica%2Fdjango-collaborative/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propublica%2Fdjango-collaborative/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propublica%2Fdjango-collaborative/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/propublica","download_url":"https://codeload.github.com/propublica/django-collaborative/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/propublica%2Fdjango-collaborative/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267899759,"owners_count":24163000,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv-import","django","google-sheets","journalism","screendoor"],"created_at":"2024-07-31T17:01:01.288Z","updated_at":"2025-07-30T16:36:10.694Z","avatar_url":"https://github.com/propublica.png","language":"Python","readme":"# Collaborate\n\n![ProPublica](https://raw.githubusercontent.com/propublica/django-collaborative/master/docs/images/ProPublica.png) ![Google News Initiative](https://raw.githubusercontent.com/propublica/django-collaborative/master/docs/images/Google-News-Initiative.png)\n\nThis is a web application for managing and building stories based on\ntips solicited from the public. This project is meant to be easy to\nsetup for non-programmer, intuitive to use and highly extendable.\n\nHere are a few use cases:\n- Collection of data from various sources (Google Form via Google Sheets, Screendoor, Private Google Spreadsheets)\n- An easy to setup data entry system\n- Organizing data from multiple sources and allowing many users to view and annotate it\n\nThe project is broken up into several components:\n- A system for transforming CSV files into managed database records\n- A default and automatic Django admin panel built for rapid and easy editing,\n  managing and browsing of data\n- Customizable fields for tagging, querying, annotating and tracking tips\n\nThis is a project of [ProPublica](https://www.propublica.org/),\nsupported by the [Google News Initiative](https://newsinitiative.withgoogle.com/).\n\n# Documentation\n\nWe have a GitBook with a full user guide that covers running Collaborate, importing and refining data, and setting up Google services. [You can read the documentation here.](https://propublica.gitbook.io/collaborative/)\n\n# Deploy it\n\nCollaborate has builtin support for one-click installs in both Google Cloud and\nHeroku. During the setup process for both deployments, *make sure to\nfill in the email, username and password fields so you can log in.*\n\n## Heroku\n\n[![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy?template=https://github.com/propublica/django-collaborative/tree/master)\n\nThe Heroku deploy button will create a small, \"free-tier\" Collaborate\nsystem. This consists of a small web server, a database which\nsupports between 10k-10M records (depending on data size) and\nautomatically configures scheduled data re-importing.\n\n## Google Cloud\n\n[![Run on Google Cloud](https://storage.googleapis.com/cloudrun/button.svg)](https://console.cloud.google.com/cloudshell/editor?shellonly=true\u0026cloudshell_image=gcr.io/cloudrun/button\u0026cloudshell_git_repo=https://github.com/propublica/django-collaborative.git\u0026cloudshell_git_branch=master\u0026cloudshell_working_dir=deploy/google-cloud)\n\nThe Google Cloud Run button launches Collaborate into the Google Cloud\nenvironment. This deploy requires you to [setup a Google Project][gc-proj],\nenable [Google Cloud billing][gc-bill] and enable the [Cloud Run API](gc-run).\n[Full set up instructions are here][gc-docs].\n\nThis deploy does not automatically configure scheduled re-importing, but\nyou can add it via Cloud Scheduler by [following these instructions][gc-sched].\n\nOnce you've deployed your Cloud Run instance, you can manage your running\ninstance from the [Google Developer's Console][run-dashboard].\n\n## Getting Started (Local Testing/Development)\n\nGetting the system set up and running locally begins with cloning this\nrepository and installing the Python dependencies. Python 3.6 or 3.7 and Django 2.2 are assumed here.\n\n    # virtual environment is recommended\n    mkvirtualenv -p /path/to/python3.7 collaborative\n    # install python dependencies\n    pip install -r requirements.txt\n\nAssuming everything worked, let's bootstrap and then start the local server:\n\n    # get the database ready\n    python manage.py migrate\n\n    # create a default admin account\n    python manage.py createsuperuser\n\n    # gather up django and collaborate assets\n    python manage.py collectstatic --noinput\n\n    # start the local application\n    python manage.py runserver\n\nYou can then access the application `http://localhost:8000` and log\nin with the credentials you selected in the `createsuperuser` step\n(above). Logging in will bring you to a configuration wizard where\nyou will import your first Google Sheet and import its contents.\n\n## Production Deploy (Nginx/Docker)\n\nIf you want to deploy this to a production environment, we've included\nconfiguration templates and scripts for [Docker][docker] and [Nginx][nginx].\n\nA Collaborate `Dockerfile` (the same one used by the Google Cloud Run\ndeploy) can be found here:\n\n    deploy/google-cloud/Dockerfile\n\nThis creates a basic production environment with nginx and gunicorn. By\ndefault, it uses SQLite3, but you can configure the database by adding a\n`DATABASE_URL` environment variable. You can read more about the format\nfor this variable [here][dj-database-url].\n\nWe also included a configuration script for plain Nginx deploys here:\n\n    deploy/google-cloud/django_nginx.conf\n\nThis can be copied to your main Nginx sites configuration directory (e.g.,\n`/etc/nginx/sites-available/`).\n\nIn order to get auto-updating data sources, make sure to add a cron job\nthat runs the following `manage.py` command:\n\n    manage.py refresh_data_sources\n\nThere's an example cron file that, when added to your `/etc/crontab`, will\nupdate data every 15 minutes:\n\n    ./deploy/cron/refresh_data_sources\n\nNote that if you use the above example, you probably want to add logrotate\nfor the logfile the above cron config adds. You can find the logrotate\nscript here (add it to `/etc/logrotate.d/refresh_data_sources`):\n\n    ./deploy/logrotate/refresh_data_sources\n\n\n[gc-proj]: https://console.cloud.google.com/projectselector2/home/dashboard\n    \"Google Cloud Project Selector\"\n\n[gc-bill]: https://cloud.google.com/billing/docs/how-to/modify-project\n    \"Google Cloud Billing set up\"\n\n[gc-run]: https://console.cloud.google.com/flows/enableapi?apiid=cloudbuild.googleapis.com,run.googleapis.com\u0026redirect=https://console.cloud.google.com\n    \"Enable Google Cloud Run API\"\n\n[gc-docs]: https://cloud.google.com/run/docs/quickstarts/build-and-deploy\n    \"Google Cloud Run quickstart\"\n\n[gc-sched]: https://cloud.google.com/run/docs/events/using-scheduler\n    \"Google Cloud Scheduler\"\n\n[run-dashboard]: https://console.cloud.google.com/run\n    \"Google Cloud Run console\"\n\n[docker]: https://www.docker.com/get-started\n    \"Docker website\"\n\n[nginx]: https://nginx.org/\n    \"Nginx website\"\n\n[dj-database-url]: https://github.com/jacobian/dj-database-url\n    \"Django database environment variable instructions\"\n","funding_links":[],"categories":["\u003ca id=\"tag-communication\" href=\"#tag-communication\"\u003eCommunication\u003c/a\u003e","Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpropublica%2Fdjango-collaborative","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpropublica%2Fdjango-collaborative","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpropublica%2Fdjango-collaborative/lists"}