{"id":25168262,"url":"https://github.com/himanegi/dirscan","last_synced_at":"2026-05-02T03:32:11.937Z","repository":{"id":275086727,"uuid":"920445583","full_name":"himanegi/dirscan","owner":"himanegi","description":"A NestJS-based project for scanning directories and their subdirectories in parallel using worker threads. This project demonstrates how to efficiently scan large directory structures while leveraging multi-core CPUs for improved performance.","archived":false,"fork":false,"pushed_at":"2025-01-31T04:56:02.000Z","size":439,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T18:52:31.016Z","etag":null,"topics":["nestjs","typescript","workerpool"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/himanegi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-22T06:52:12.000Z","updated_at":"2025-01-31T04:56:06.000Z","dependencies_parsed_at":"2025-01-31T05:26:22.140Z","dependency_job_id":null,"html_url":"https://github.com/himanegi/dirscan","commit_stats":null,"previous_names":["himanegi/dirscan"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/himanegi/dirscan","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanegi%2Fdirscan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanegi%2Fdirscan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanegi%2Fdirscan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanegi%2Fdirscan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/himanegi","download_url":"https://codeload.github.com/himanegi/dirscan/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/himanegi%2Fdirscan/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32522245,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-02T01:12:54.858Z","status":"online","status_checked_at":"2026-05-02T02:00:05.923Z","response_time":132,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nestjs","typescript","workerpool"],"created_at":"2025-02-09T07:16:30.373Z","updated_at":"2026-05-02T03:32:11.913Z","avatar_url":"https://github.com/himanegi.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Directory Scanning using Nest.js\n\nA NestJS-based project for scanning directories and their subdirectories in parallel using worker threads. This project demonstrates how to efficiently scan large directory structures while leveraging multi-core CPUs for improved performance.\n\n---\n\n## Features\n\n- **Parallel Directory Scanning**: Utilizes worker threads to scan directories in parallel, ensuring optimal performance.\n- **Depth-First Search (DFS) and Breadth-First Search (BFS)**: Supports both DFS and BFS approaches for directory traversal.\n- **Worker Pool**: Uses `workerpool` library to manage a pool of worker threads for concurrent scanning.\n- **File Metadata**: Retrieves metadata for each file/directory, including:\n  - File name\n  - File path\n  - File size\n  - Last modified time\n  - Whether it's a file or directory\n\n---\n\n## Workflow\n\n![workflow](./src/public/workflow.png)\n\n## Prerequisites\n\nBefore running the project, ensure you have the following installed:\n\n- [Node.js](https://nodejs.org/) (v16 or higher)\n- [NestJS CLI](https://docs.nestjs.com/cli/overview) (recommended)\n- [Yarn](https://yarnpkg.com/) or [npm](https://www.npmjs.com/) (package manager)\n\n---\n\n## Installation\n\n1\\. Clone the repository:\n\n```bash\ngit clone https://github.com/himanegi/dirscan.git\ncd dirscan\n```\n\n2\\. Install dependencies:\n\n```bash\nnpm install\n```\n\n3\\. Build the project:\n\n```bash\nnpm run build\n```\n\n---\n\n## Usage\n\n### Running the Project\n\n1\\. Start the NestJS application:\n\n```bash\nnpm run start\n```\n\n2\\. The application will start a server on `http://localhost:3000`.\n\n---\n\n### Scanning a Directory\n\nTo scan a directory, send a GET request to the `/scan` endpoint with `/dfs`, `/bfs` or `/sync` route.\n\n#### Example Request\n\n```bash\ncurl -X GET http://localhost:3000/scan/dfs\ncurl -X GET http://localhost:3000/scan/bfs\ncurl -X GET http://localhost:3000/scan/sync\n```\n\n#### Example Response\n\n```json\n[\n  {\n    \"file\": \"file1.txt\",\n    \"filePath\": \"test-dir/file1.txt\",\n    \"size\": 1024,\n    \"modifiedTime\": \"2023-10-01T12:34:56.000Z\",\n    \"isDirectory\": false,\n    \"isFile\": true\n  },\n\n  {\n    \"file\": \"dir1\",\n    \"filePath\": \"test-dir/dir1\",\n    \"size\": 4096,\n    \"modifiedTime\": \"2023-10-01T12:34:56.000Z\",\n    \"isDirectory\": true,\n    \"isFile\": false\n  }\n]\n```\n\n---\n\n### Configuration\n\nYou can configure the worker pool settings in the `ScanService` class:\n\n```typescript\nthis.pool = workerpool.pool(join(__dirname, 'scan-worker'), {\n  maxWorkers: 5, // Adjust the number of workers\n});\n```\n\n---\n\n## Project Structure\n\n```\n\ndirscan/\n\n├── src/\n│   ├── scan/\n│   │   ├── scan.service.ts       # Service for directory scanning\n│   │   ├── scan.controller.ts    # Controller for handling scan requests\n│   │   ├── scan.worker.ts        # Worker script for parallel scanning\n│   ├── app.module.ts             # Root module\n│   ├── main.ts                   # Application entry point\n├── test-dir/                     # Example directory for testing\n├── README.md                     # This file\n├── package.json                  # Project dependencies and scripts\n├── tsconfig.json                 # TypeScript configuration\n\n```\n\n---\n\n## Performance\n\nThe project uses worker threads to parallelize directory scanning, making it suitable for large directory structures. You can adjust the number of workers in the worker pool to optimize performance based on your system's CPU cores.\n\n---\n\n## Contributing\n\nContributions are welcome! If you'd like to contribute, please follow these steps:\n\n1\\. Fork the repository.\n\n2\\. Create a new branch for your feature or bugfix.\n\n3\\. Commit your changes and push to the branch.\n\n4\\. Submit a pull request.\n\n---\n\n## Acknowledgments\n\n- [NestJS](https://nestjs.com/) for the awesome framework.\n\n- [workerpool](https://github.com/josdejong/workerpool) for simplifying worker thread management.\n\n---\n\nEnjoy scanning directories with NestJS! 🚀\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhimanegi%2Fdirscan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhimanegi%2Fdirscan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhimanegi%2Fdirscan/lists"}