{"id":23270809,"url":"https://github.com/mcxxmc/simple-webpage-document-storage-sys","last_synced_at":"2026-04-05T20:32:22.999Z","repository":{"id":230676967,"uuid":"411152889","full_name":"mcxxmc/simple-webpage-document-storage-sys","owner":"mcxxmc","description":"A simple implementation to simulate a document-based database system (instead of tables).","archived":false,"fork":false,"pushed_at":"2021-11-15T07:21:31.000Z","size":343,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-12T15:20:02.702Z","etag":null,"topics":["database","golang","react"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcxxmc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-28T05:47:05.000Z","updated_at":"2022-02-27T10:55:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"642bb6cd-17a5-4f86-87c5-67a2a53b962b","html_url":"https://github.com/mcxxmc/simple-webpage-document-storage-sys","commit_stats":null,"previous_names":["mcxxmc/simple-webpage-document-storage-sys"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcxxmc%2Fsimple-webpage-document-storage-sys","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcxxmc%2Fsimple-webpage-document-storage-sys/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcxxmc%2Fsimple-webpage-document-storage-sys/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcxxmc%2Fsimple-webpage-document-storage-sys/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcxxmc","download_url":"https://codeload.github.com/mcxxmc/simple-webpage-document-storage-sys/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247463804,"owners_count":20942942,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","golang","react"],"created_at":"2024-12-19T18:55:31.342Z","updated_at":"2025-12-30T23:06:47.571Z","avatar_url":"https://github.com/mcxxmc.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# simple-webpage-document-storage-sys\n\n\n## General\n\n\nKeyword: document-based database system, Golang, Gin framework, React.\n\n\nA simple implementation to simulate a document-based database system (instead of tables).\n\n\nBackend supported by go and includes a gin server. Frontend is currently developed in React.\n\n\nUnlike the file explorer in most systems, the hierarchy of the files \u0026 directories are not reflected in the physical level.\n\n\nAll the files owned by the same user are put under the same \"physical\" folder and their relations are managed by the profile of that user. All the directories are logical and virtual as they do not physically exist. Each user profile is a JSON file containing the relative relationships of all the real files and virtual directories.\n\n\nUser profiles are maintained by a bigger index (also a JSON file). Adding or deleting users will modify that big index, but register and unregister will not.\n\n\nThe two important parts of the backend is manager and cache. \"manager\" maintains the user collections (the profiles) for login user only. \"cache\" keeps all the basic user info (name, id, password, etc.) regardless of logging in or not. I am considering that replacing \"cache\" with an outer SQL server or Redis may simplify the system.\n\n\n## Frontend\n\n\nImplemented in React and javascript. This is my first time using React and I am still learning. Currently, I have been familiar with the concepts of state, props and callback functions.\n\n\nBasically, it is a single page application and some certain flags to control which information to show the users. \n\n\nIt has login and logout (using tokens). It visualizes the hierarchy structure of users' data and allow creating, reading, renaming, moving and deleting. The interactions with the backend are achieved using `fetch`.\n\n\n## Backend\n\n\nThis is the most important part of this project and I am mostly proud of and interested in it. \n\n\nAs is mentioned in the \"General\" section, it simulates a document-based database system. This system consists of 3 parts: the index of all users (a single JSON file), the profile of each user (lots of JSON files, 1 for each user) and the collections of user documents (each user own his/her own collection, which is an independent folder containing all the documents). \n\n\nThough a user may sort and store his/her documents into different categories and put them under different folders, those \"folders\" are not real and do not exist on the disk physically. Those relationships are stored as JSON objects in the user profile and all the documents are stored in the same collection folder. So, if you directly open the collection folder, you will see all the documents in parallel and you won't see any hierarchy structure here.\n\n\nNote that each user and each document he/she owns has a unique id (64-character long by default) to avoid collision. Ids are randomly generated by the backend server.\n\n\nUsers use ids as the names of their collection folders instead of their customized usernames; documents use ids as their filename on disk instead of their filenames given by user. The document owned by different users may have the same id as they are distinguished by different owners and are put under different collection folders.\n\n\nTo be honest, I am not sure whether such design will bring significant benefits compared to the traditional way of storing documents under nested folders. I think the main advantage will be the convenience provided by the abstraction between the hierarchy structure and real physical storages. \n\n\nIt will be less costly to create, delete, rename directories and files and update the relationships between them, as they (called `Image` in my codes) are simply some loosely organized JSON objects. \n\n\nYou do not need to move the file physically on your disk; the only thing you need to do is to modify the attributes of the corresponding JSON objects (e.g., `Parent` and `Children`). It is even more convenient for directories, as there are no real folders for them. Most modification is made and only made to the JSON profile. Exceptions are creating, deleting and modifying a file (not including renaming).\n\n\nMainly, the whole implementation consists of 4 packages. The controller, the file system (`filesys`), the logging, and the manager.\n\n\n### controller\n\n\nIt has achieved Restful API using Golang Gin framework. It uses middleware to allow CORS, deal with Options and handle tokens.\n\n\n### filesys\n\n\nNote that this `filesys` is different from the `file-sys` folder. The later is the physical storage while the former is my golang used for interacting with the os.\n\n\nIt defined many fundamental structs for the whole project, and is a brigde between the os and the \"manager\". \n\n\nNote that the \"controller\" should never calls this package directly; it should call \"manager\" instead.\n\n\n### logging\n\n\nBasically a wrap-up of the Uber zap log. \n\n\n### manager\n\n\nThe most important part that achieves most functions of the database system. It defines a \"manager\" class to manager all the collections for login users. It also has a \"cached\" object to manager all the user information (e.g., username, password). \n\n\nThe \"manager\" must be initialized at the beginning of each execution. It calls `filesys` to load user collections for users that have logged in. When the server is shutting down, the \"manager\" also needs to be called to save the modified user collections into the disk.\n\n\nBefore commiting any operation (e.g., create a new file), the \"manager\" will do necessary checkings to ensure that the parameters are valid (e.g., the user id exists, the parent id is valid, the file id is unique, ...). Then, it will update the `Image` and the collection before any real change is made to the disk. \n\n\nAfter several updates and fixes, the logic behind those operations should be error-free now.\n\n\nIn very rare cases (probably an accidential shutdown), the collection may contain a false `Image` that leads to a file that does not exist. Currently there is no mechanism to compensate for that.\n\n\n### token\n\n\nThere is also a token package for JWT tokens.\n\n\n## TODO\n\n\n### 1. Most Important: React App Data Structure Reconstruction (frontend)\n\n\nReconstruct the way React App stores the data (`state.organized`) from array to a tree, which will make the computation much easier for collapsing (using `state.show` and `state.collapse`). A possible way is to rewrite the `recursivelyAppend()` method. Also, it will make it easier to update the view after creating, renaming or deleting a file or a directory, as there is no need to call `fetchAndSort()` each time, which is very expensive.\n\n\n### 2. Collapse (frontend)\n\n\nNeed to achieve the visualization effects: click on an icon to collapse the children nodes of a parent node.\n\n\n### 3. Hardcoding (frontend \u0026 backend)\n\n\nHardcoding some static strings as variables and use these variables instead of directly using those strings.\n\n\n### 4. Better CSS (frontend)\n\n\nUnluckily, the webpage is ugly now.\n\n\n### 5. Middleware (backend)\n\n\nImprove the middleware by applying router groups. Should not check token for the login request.\n\n\n### 6. (Optional, backend) Changing `cached` to SQL  \n\n\n### 7. Rare Cases for \"manager\" (backend)\n\n\nDevelop a mechanism to handle the real cases mentioned in the end of the \"manager\" section above. The simplest way is to enable the user to just delete that phantom file from the collection. \n\n\nI have included the corresponding codes in the comments in `manager/strcut.go deleteTxt`. However, it may lead to uncontrolled behaviours that are not known yet. Considering the rare cases should be really rare, please use it at you own risk.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcxxmc%2Fsimple-webpage-document-storage-sys","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcxxmc%2Fsimple-webpage-document-storage-sys","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcxxmc%2Fsimple-webpage-document-storage-sys/lists"}