{"id":15772401,"url":"https://github.com/c-ehrlich/vocab-learn-order","last_synced_at":"2026-04-09T01:31:10.778Z","repository":{"id":134327252,"uuid":"459278089","full_name":"c-ehrlich/vocab-learn-order","owner":"c-ehrlich","description":"Determine the optimal order to learn vocabulary and find sample sentences","archived":false,"fork":false,"pushed_at":"2022-07-04T12:32:25.000Z","size":46911,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-11T15:48:09.705Z","etag":null,"topics":["express","material-ui","mongodb","nodejs","react","supertest","typescript","zod","zustand"],"latest_commit_sha":null,"homepage":"https://c-ehrlich.github.io/vocab-learn-order/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/c-ehrlich.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-14T18:25:45.000Z","updated_at":"2022-10-12T16:35:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"44931acb-bce9-48a7-a3d3-e46c38d3b943","html_url":"https://github.com/c-ehrlich/vocab-learn-order","commit_stats":{"total_commits":132,"total_committers":1,"mean_commits":132.0,"dds":0.0,"last_synced_commit":"af0cd485b19926d5c11e5de7dab713714466d507"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/c-ehrlich/vocab-learn-order","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c-ehrlich%2Fvocab-learn-order","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c-ehrlich%2Fvocab-learn-order/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c-ehrlich%2Fvocab-learn-order/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c-ehrlich%2Fvocab-learn-order/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/c-ehrlich","download_url":"https://codeload.github.com/c-ehrlich/vocab-learn-order/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/c-ehrlich%2Fvocab-learn-order/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31581864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-08T14:31:17.711Z","status":"ssl_error","status_checked_at":"2026-04-08T14:31:17.202Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["express","material-ui","mongodb","nodejs","react","supertest","typescript","zod","zustand"],"created_at":"2024-10-04T15:22:14.190Z","updated_at":"2026-04-09T01:31:10.759Z","avatar_url":"https://github.com/c-ehrlich.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vocab Learn Order\n\nLive Demo: [https://c-ehrlich.github.io/vocab-learn-order](https://c-ehrlich.github.io/vocab-learn-order)\n\nThe backend is on a free Heroku instance, so please allow it a few seconds to spin up :)\n\n![Vocab Learn Order Screenshot](https://user-images.githubusercontent.com/8353666/155544745-745b1201-b071-426d-aac8-60875831d57e.png)\n\nVocab Learn Order is a web app for Japanese learners. Many immersion learners know the experience of listening to podcasts or other audio material in their target language, looking up unknown words, and then saving those words in a Notes file with the intent to learn them later with a tool such as Anki. But suddenly the file is several hundred words long and it's difficult to know which words to learn first to get the greatest benefit.\n\nFrequency lists help with this somewhat, but as each frequency list is computed from a type of media, no single frequency list can give a realistic representation of how useful a given word is. Additionally, different learners have different goals - for some the highest priority is reading, for some it's business conversation, for some it's casual conversation, and so on.\n\nBy letting users decide how highly to weigh each frequency list, this app is able to create a much more accurate learn order than other methods. Additionally it makes Anki card creation much easier by giving access to sample sentences for each word through YouGlish and ImmersionKit. It's suitable for learners of all skill levels as the database is built on a dataset created specifically for this app, which contains frequency information for about 200,000 words, significantly more than most individual frequency lists.\n\n## Table of Contents\n  - [Technologies](#technologies)\n  - [Installation and Setup](#installation-and-setup)\n    - [Development](#development)\n    - [Deployment](#deployment)\n  - [Reflection](#reflection)\n\n## Technologies\n* __Backend__: MongoDB, Node / Express with TypeScript, Mongoose / TypeGoose, Zod, Jest / Supertest\n* __Frontend__: React with TypeScript, Zustand, React Router, Material UI, Jest / React Testing Library\n* __Sample Deployment__: MongoDB Atlas (Database), Heroku (Backend), GitHub Pages (Frontend)\n\n## Installation and Setup\n\n### Development\n\n1. Clone the repo\n```\ngit clone https://github.com/c-ehrlich/vocab-learn-order.git\n```\n2. Run a local instance of MongoDB\n```\nmongod\n```\n3. Create .env files in `/backend` and `/frontend`, based on respective `sample.env` files\n\n4. Install backend dependencies and start the backend development server\n```\ncd backend\nyarn\nyarn dev\n```\n\n5. Populate the development database\n```\ncurl -X POST http://localhost:1337/api/filldatabase\n   -H 'Content-Type: application/json'\n   -d '{\"privateKey\": \"\u003cprivate key\u003e\"}'\n```\n\n6. Install frontend dependencies and start the frontend development server\n```\ncd frontend\nyarn\nyarn start\n```\n\n### Deployment\n\n#### Database (MongoDB Atlas)\n1. Create a new Database on MongoDB Atlas\n2. Get the URI for the database and put in `/backend/.env`\n3. After deploying the backend, populate the database\n```\ncurl -X POST \u003cbackend url\u003e/api/filldatabase\n   -H 'Content-Type: application/json'\n   -d '{\"privateKey\": \"\u003cprivate key\u003e\"}'\n```\n\n#### Backend (Heroku)\n1. Create a new Heroku app\n2. Add environment variables to Heroku's \"Config Vars\" in the app settings, see `sample.env` for example\n3. Add a `heroku` remote\n```\nheroku git:remote -a \u003cname of heroku app\u003e\n```\n4. From the project root directory, deploy the `/backend` folder\n```\ngit subtree push --prefix backend heroku main\n```\n\n#### Frontend (GitHub Pages)\nSee the [Create React App documentation](https://create-react-app.dev/docs/deployment/) for more details.\n1. Add a \"homepage\" item to `package.json` that points to where the app will be deployed. Example:\n```\n\"homepage\": \"http://mywebsite.com/relativepath\",\n```\n\n2. Add predeploy and deploy scripts to package.json\n```\n  \"scripts\": {\n+   \"predeploy\": \"npm run build\",\n+   \"deploy\": \"gh-pages -d build\",\n    \"start\": \"react-scripts start\",\n    ...\n```\n3. Run the `deploy` script\n```\ncd frontend\nyarn deploy\n```\n\n#### Create custom Database Seed data\nThis is not necessary for a normal deployment. However it might be desireable in the future, for example in order to implement additional frequency lists, or to create a similar application for a different langauge. To achieve this:\n\n* Most frequency lists come with a `\u003clistname\u003e-index.json` file. That file includes a `format` key. For this process to work, the format must be `3`. Fortunately most frequency lists are in that format. For frequency lists in a non-standard format, you will need to analyse their structure and adapt one of the existing algorithms in `/create-data/src/combine-frequency-lists.ts` to parse them.\n* In `combine-frequency-lists.ts`:\n1. Add the new frequency list to the `TWord` interface\n2. Import the list\n```ts\nconst listName: Format3Entry[] = require(`${fileLocation}`);\n```\n3. If the list is made up of multiple JSON files, import them all and concatenate into one array\n4. Add the list to the `lists` variable\n```ts\nconst lists: { list: Format3Entry[]; name: string }[] = [\n  {\n    list: \u003cvariable name of the imported list\u003e,\n    name: '\u003ckey name of the list in the database\u003e',\n  },\n  ...\n```\n* Create the combined Dictionary / Frequency List file __(warning: this will take quite long as we are iterating over several million items)__\n```\ncd create-data/src\nnpx ts-node combine-frequency-lists.ts\n```\n* Place the resulting `words-jmdict.json` file in `/backend/src/data`\n* Register the new model fields in the backend in `/backend/src/model/word.model.ts` and `/backend/src/schema/word.schema.ts`\n* Register the new model fields in the frontend in `/frontend/src/types/TWord.type.ts` and `/frontend/src/types/TFrequencyListWeights.type.ts`\n* Delete the database if it is already populated\n```\ncurl -X POST \u003cbackend url\u003e/api/deletedatabase\n   -H 'Content-Type: application/json'\n   -d '{\"privateKey\": \"\u003cprivate key\u003e\"}'\n```\n* Populate the database\n```\ncurl -X POST \u003cbackend url\u003e/api/filldatabase\n   -H 'Content-Type: application/json'\n   -d '{\"privateKey\": \"\u003cprivate key\u003e\"}'\n```\n\n## Reflection\nThis app represented several new steps in my development as a Full Stack Engineer:\n  * Backend and Frontend developed fully in TypeScript\n  * Test suites for both backend and frontend\n  * Uses an established backend design framework (Route - Middleware - Controller - Service - Schema - Database)\n\nI am proud of building another app that has real users, and of creating a well structured and maintaible full stack app for the first time. I made a lot of progress from the spaghetti code backends of my previous projects, and now have a template for developing good backends in the future. I was impressed by both the Developer Experience of using TypeScript, and the new development possiblities it gives, such as validating requests based on DB schema in a single line of code.\n\nOn the frontend, it was my first time building a significant UI using a Component Library (Material UI). In the past I had mostly just built my own UIs from scratch using Styled Components. I was impressed by how much time and how many lines of code it is possible to save by creating a themeing/design system before starting to build individual components, and how much a library helps with accessibility. It takes a bit of work to make a Material UI project not look like a stock Google app, but the result is worth it.\n\nThe part of the project that is quite messy is the `create-data` component. However given the current dictionaries and frequency lists it creates the desired output, and I don't think it will be necessary to change in the future. So in the spirit of YAGNI I decided to leave it as is for now and build it out in a more proper way if I do ever decide to build a new dataset.]\n\nDue to how GitHub Pages hosts static content, I used HashRouter instead of BrowserRouter for url routing. This means that accessing for example `https://c-ehrlich.github.io/vocab-learn-order/foo` will result in a blank page instead of redirecting to the homepage or showing a 404. If this app were to be deployed somewhere else, this would be easy to fix by switching the routing implementation to BrowserRouter. \n\nPopulating the database through a backend route that is protected by a private key is not ideal as such a feature should not have a public endpoint at all. Keeping this functionality within the backend package means the Database Schema can be used to verify the incoming data, but putting this functionality in its own file that needs to be executed from the command line on the server would be a better approach in terms of security.\n\n## Acknowledgements\n* JLPT Vocab List from [stephenmk/yomichan-jkpt-vocab](https://github.com/stephenmk/yomichan-jlpt-vocab)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fc-ehrlich%2Fvocab-learn-order","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fc-ehrlich%2Fvocab-learn-order","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fc-ehrlich%2Fvocab-learn-order/lists"}