{"id":45142789,"url":"https://github.com/chateval/application","last_synced_at":"2026-03-04T22:00:44.781Z","repository":{"id":41948069,"uuid":"137398628","full_name":"chateval/application","owner":"chateval","description":"A platform for the warehousing and evaluation of neural open domain chatbot models.","archived":false,"fork":false,"pushed_at":"2024-04-01T21:36:21.000Z","size":2289,"stargazers_count":6,"open_issues_count":5,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-01T22:38:47.100Z","etag":null,"topics":["chatbot-framework","django-application","nlp"],"latest_commit_sha":null,"homepage":"https://chateval.org","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chateval.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-14T19:12:35.000Z","updated_at":"2022-11-18T08:35:42.000Z","dependencies_parsed_at":"2023-01-25T03:00:57.229Z","dependency_job_id":null,"html_url":"https://github.com/chateval/application","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/chateval/application","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chateval%2Fapplication","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chateval%2Fapplication/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chateval%2Fapplication/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chateval%2Fapplication/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chateval","download_url":"https://codeload.github.com/chateval/application/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chateval%2Fapplication/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30095688,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-04T21:59:23.547Z","status":"ssl_error","status_checked_at":"2026-03-04T21:57:50.415Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot-framework","django-application","nlp"],"created_at":"2026-02-20T01:00:27.320Z","updated_at":"2026-03-04T22:00:44.772Z","avatar_url":"https://github.com/chateval.png","language":"Python","readme":"# ChatEval\nA scientific framework for evaluating neural open domain chatbots.\n\n## Dependencies\n- Django (web framework)\n- Bulma (CSS framework)\n- MySQL (database)\n- Amazon S3 (Dataset storage)\n- Magnitude (word embedding format)\n\n## Usage\n### (Optional) Docker Installation\nChatEval supports the use of Docker as both a development and deployment tool.\n\n0. Install [Docker](https://docker.com/).\n1. Configure environment variables in `Dockerfile` by adding `ENV variable value` for each environment variable.\n2. Build Docker image by using `docker build -t chateval .` (this may take some time).\n3. Run ChatEval on port 8000 by using `docker run chateval`\n4. Access app at [localhost:8000](localhost:8000) and admin SQL page at [localhost:8000/admin](localhost:8000/admin).\n5. Run the [evaluation microservice](https://github.com/chateval/evaluation) at localhost:8001.\n\n### Running `application`\nChatEval's primary microservice is `application`, which handles our Django back-end and SQL database. To run this:\n\n0. Install dependencies using `pip install -r requirements.txt`.\n2. Edit database information in `/chateval/settings.py`.\n3. Run server migrations using `python manage.py makemigrations \u0026\u0026 python manage.py migrate`.\n4. Create an `env.sh` (and source using `source env.sh`) containing:\n```\nexport DB_NAME=\nexport DB_USER=\nexport DB_PASSWORD=\nexport DB_HOST=\nexport DB_PORT=\nexport AWS_ACCESS_KEY_ID=\nexport AWS_SECRET_ACCESS_KEY=\nexport AWS_STORAGE_BUCKET_NAME=\nexport EMAIL_PASSWORD=\nexport EVAL_LOCATION=\n```\n5. Run server using `python manage.py runserver`\n6. Download `.magnitude` word embeddings from [here](http://magnitude.plasticity.ai/word2vec/GoogleNews-vectors-negative300.magnitude) and place the vectors in `/eval/scripts/files/google_news.magnitude`.\n7. Populate database using `python manage.py loaddata init.json`.\n8. Access app at [localhost:8000](localhost:8000) and admin SQL page at [localhost:8000/admin](localhost:8000/admin).\n\n### Running `evaluation`\nChatEval's evaluation microservice is available [here](https://github.com/chateval/evaluation) and handles the evaluation of  models to abstract Mechanical Turk interactions and to offload computation for loading/querying word embeddings. The location for this API is configurable as an environment variable named `EVAL_LOCATION` and is defaulted to [localhost:8001](localhost:8001). Note the location must be configured for both `application` and `evaluation`.\n","funding_links":[],"categories":["🤖 LLM \u0026 Chatbot Testing"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchateval%2Fapplication","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchateval%2Fapplication","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchateval%2Fapplication/lists"}