{"id":25739207,"url":"https://github.com/TechforgoodCAST/find-that-charity","last_synced_at":"2025-02-26T08:02:02.254Z","repository":{"id":21607066,"uuid":"93387617","full_name":"kanedata/find-that-charity","owner":"kanedata","description":"Reconciliation for UK Charities and other nonprofit organisations, with elasticsearch back end. ","archived":false,"fork":false,"pushed_at":"2025-02-13T12:01:06.000Z","size":13361,"stargazers_count":28,"open_issues_count":46,"forks_count":9,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-02-13T23:03:27.803Z","etag":null,"topics":["charity","charity-commission","elasticsearch","python-django","reconciliation","reconciliation-api","scottish-charities"],"latest_commit_sha":null,"homepage":"https://findthatcharity.uk/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kanedata.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-05T09:27:15.000Z","updated_at":"2025-02-13T12:01:11.000Z","dependencies_parsed_at":"2023-02-13T20:17:09.986Z","dependency_job_id":"5544248e-d785-4975-b990-54f67a9daf7d","html_url":"https://github.com/kanedata/find-that-charity","commit_stats":null,"previous_names":["techforgoodcast/find-that-charity"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanedata%2Ffind-that-charity","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanedata%2Ffind-that-charity/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanedata%2Ffind-that-charity/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kanedata%2Ffind-that-charity/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kanedata","download_url":"https://codeload.github.com/kanedata/find-that-charity/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240814850,"owners_count":19861955,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["charity","charity-commission","elasticsearch","python-django","reconciliation","reconciliation-api","scottish-charities"],"created_at":"2025-02-26T08:01:41.480Z","updated_at":"2025-02-26T08:02:02.156Z","avatar_url":"https://github.com/kanedata.png","language":"Python","funding_links":[],"categories":["Awesome tech for good projects"],"sub_categories":[],"readme":"# Find that charity\n\nElasticsearch-powered search engine for looking for charities and other non-profit organisations. Allows for:\n\n- importing data nearly 20 sources in the UK, ensuring that duplicates\n  are matched to one record.\n- An elasticsearch index that can be queried.\n- [Org-ids](http://org-id.guide/about) are added to organisations.\n- Reconciliation API for searching organisations, based on an optimised search query.\n- Facility for uploading a CSV of charity names and adding the (best guess) at a\n  charity number.\n- HTML pages for searching for a charity\n\n## Installation\n\n1. [Clone repository](https://github.com/kanedata/find-that-charity)\n2. Create virtual environment (`python -m venv env`)\n3. Activate virtual environment (`env/bin/activate` or `env/Scripts\\activate`)\n4. Install requirements (`pip install -r requirements.txt`)\n5. [Install postgres](https://www.postgresql.org/download/)\n6. Start postgres\n7. Create 2 postgres databases - one for admin (eg `ftc_admin` and one for data eg `ftc_data`)\n8. [Install elasticsearch 7](https://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html) - you may need to increase available memory (see below)\n9. Start elasticsearch\n10. Create `.env` file in root directory. Contents based on `.env.example`.\n11. Create the database tables (`python ./manage.py migrate --database=data \u0026\u0026 python ./manage.py migrate --database=admin \u0026\u0026 python ./manage.py createcachetable --database=admin`)\n12. Import data on charities (`python ./manage.py import_charities`)\n13. Import data on nonprofit companies (`python ./manage.py import_ch`)\n14. Import data on other non-profit organisations (`python ./manage.py import_all`)\n15. Add organisations to elasticsearch index (`python ./manage.py es_index`) - (Don't use the default `search_index` command as this won't setup aliases correctly)\n\n## Dokku Installation\n\n### 1. Set up dokku server\n\nSSH into server and run:\n\n```bash\n# create app\ndokku apps:create ftc\n\n# postgres\nsudo dokku plugin:install https://github.com/dokku/dokku-postgres.git postgres\ndokku postgres:create ftc-db-data\ndokku postgres:link ftc-db-data ftc --alias \"DATABASE_URL\"\ndokku postgres:create ftc-db-admin\ndokku postgres:link ftc-db-admin ftc --alias \"DATABASE_ADMIN_URL\"\n\n# elasticsearch\nsudo dokku plugin:install https://github.com/dokku/dokku-elasticsearch.git elasticsearch\necho 'vm.max_map_count=262144' | sudo tee -a /etc/sysctl.conf; sudo sysctl -p\nexport ELASTICSEARCH_IMAGE=\"elasticsearch\"\nexport ELASTICSEARCH_IMAGE_VERSION=\"7.7.1\"\ndokku elasticsearch:create ftc-es\ndokku elasticsearch:link ftc-es ftc\n# configure elasticsearch 7:\n# https://github.com/dokku/dokku-elasticsearch/issues/72#issuecomment-510771763\n\n# setup elasticsearch increased memory (might be needed)\nnano /var/lib/dokku/services/elasticsearch/ftc-es/config/jvm.options\n# replace `-Xms512m` with `-Xms2g`\n# replace `-Xms512m` with `-Xmx2g`\n# restart elasticsearch\ndokku elasticsearch:restart ftc-es\n\n# Redirect\ndokku plugin:install https://github.com/dokku/dokku-redirect.git\ndokku redirect:set ftc www.findthatcharity.uk findthatcharity.uk\n\n# SSL\nsudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git\ndokku letsencrypt:set ftc email your@email.tld\ndokku letsencrypt:enable ftc\ndokku letsencrypt:cron-job --add\n```\n\n### 2. Add as a git remote and push\n\nOn local machine:\n\n```bash\ngit remote add dokku dokku@SERVER_HOST:ftc\ngit push dokku main\n```\n\n### 3. Setup and run import\n\nOn Dokku server run:\n\n```bash\n# setup\ndokku run ftc python ./manage.py migrate --database=data\ndokku run ftc python ./manage.py migrate --database=admin\ndokku run ftc python ./manage.py createcachetable --database=admin\n\n# run import\ndokku run ftc python ./manage.py charity_setup\ndokku run ftc python ./manage.py import_oscr\ndokku run ftc python ./manage.py import_charities\ndokku run ftc python ./manage.py import_ch\ndokku run ftc python ./manage.py import_other_data\ndokku run ftc python ./manage.py import_all\ndokku run ftc python ./manage.py es_index\n```\n\n## Server\n\nThe server uses [django](https://www.djangoproject.com/). Run it with the\nfollowing command:\n\n`python ./manage.py runserver`\n\nThe server offers the following API endpoints:\n\n- `/reconcile`: a [reconciliation service API](https://github.com/OpenRefine/OpenRefine/wiki/Reconciliation-Service-API)\n  conforming to the OpenRefine reconciliation API specification.\n\n- `/charity/12345`: Look up information about a particular charity\n\n## Todo\n\nPriorities:\n\n- tests for ensuring data is correctly imported\n- server tests\n- use results of `server/recon_test.py` to produce the best reconciliation\n  search query for use in the server (`recon_test_7` seems the best at the moment)\n- threshold for when to use the result vs discard\n\nFuture development:\n\n- upload a CSV file and reconcile each row with a charity\n- allow updating a charity with additional possible names\n\n## Testing\n\n```sh\ncoverage run pytest \u0026\u0026 coverage html\npython -m http.server -d htmlcov --bind 127.0.0.1 8001\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTechforgoodCAST%2Ffind-that-charity","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTechforgoodCAST%2Ffind-that-charity","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTechforgoodCAST%2Ffind-that-charity/lists"}