{"id":20856826,"url":"https://github.com/techlism/headout-assignment","last_synced_at":"2025-03-12T13:43:37.189Z","repository":{"id":219254647,"uuid":"748576324","full_name":"techlism/headout-assignment","owner":"techlism","description":null,"archived":false,"fork":false,"pushed_at":"2024-01-26T10:07:01.000Z","size":13,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-19T07:13:10.313Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/techlism.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-26T09:41:22.000Z","updated_at":"2024-01-26T09:41:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"5aea4a30-d067-468d-b7a3-b80cd1ea761a","html_url":"https://github.com/techlism/headout-assignment","commit_stats":null,"previous_names":["techlism/headout-assignment"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techlism%2Fheadout-assignment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techlism%2Fheadout-assignment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techlism%2Fheadout-assignment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techlism%2Fheadout-assignment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/techlism","download_url":"https://codeload.github.com/techlism/headout-assignment/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243230062,"owners_count":20257640,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-18T04:34:44.729Z","updated_at":"2025-03-12T13:43:37.167Z","avatar_url":"https://github.com/techlism.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# To run the docker container with resource limit as specified and compatibility with x86 and ARM\n\nClone this repository.\n\n``` docker build -t headout-assignment . ```\n\nThe --memory and --cpus flags are used to set resource limits\n\n``` docker run --memory=1500m --cpus=2 --platform=linux/arm64,linux/amd64 headout-assignment ```\n\n# Optimizations in the Assignment\nIn my assignment, I made two significant optimizations to improve the efficiency of the application.\n\n## Switch from readFileSync to createReadStream and readline\nInitially, I was using fs.readFileSync to read files, which loads the entire file into memory. This was inefficient and could cause the application to run out of memory for large files.\n\nTo optimize this, I switched to fs.createReadStream combined with readline. This approach reads the file in chunks and allows processing to start as soon as the first chunk is available.\nIt's more memory-efficient and performs better with large files.\n\n## Script to Generate Large Text Files\nTo test the application's performance with large files, I created a script (generateTxt.js) that generates 31 text files, each 100MB in size. It uses the fs and crypto library.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechlism%2Fheadout-assignment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftechlism%2Fheadout-assignment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechlism%2Fheadout-assignment/lists"}