{"id":16700460,"url":"https://github.com/glasswalk3r/linux-nfs-bigdir","last_synced_at":"2025-10-22T06:51:22.232Z","repository":{"id":56839473,"uuid":"75545338","full_name":"glasswalk3r/Linux-NFS-BigDir","owner":"glasswalk3r","description":"Perl module to quickly read a large number of entries from directories under NFS","archived":true,"fork":false,"pushed_at":"2024-04-07T03:29:53.000Z","size":43,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-21T00:33:33.498Z","etag":null,"topics":["linux","nfs-client","perl5","syscall"],"latest_commit_sha":null,"homepage":"","language":"Perl","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/glasswalk3r.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-04T15:17:26.000Z","updated_at":"2024-08-13T04:31:09.000Z","dependencies_parsed_at":"2025-01-02T18:15:37.101Z","dependency_job_id":"24eba06a-07f6-4d01-838b-c93bee3d1e95","html_url":"https://github.com/glasswalk3r/Linux-NFS-BigDir","commit_stats":{"total_commits":15,"total_committers":3,"mean_commits":5.0,"dds":"0.19999999999999996","last_synced_commit":"895ff2cd03b27c0b3589eae353b85db816f157ec"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glasswalk3r%2FLinux-NFS-BigDir","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glasswalk3r%2FLinux-NFS-BigDir/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glasswalk3r%2FLinux-NFS-BigDir/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glasswalk3r%2FLinux-NFS-BigDir/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/glasswalk3r","download_url":"https://codeload.github.com/glasswalk3r/Linux-NFS-BigDir/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239132545,"owners_count":19587106,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["linux","nfs-client","perl5","syscall"],"created_at":"2024-10-12T18:10:41.360Z","updated_at":"2025-10-22T06:51:16.909Z","avatar_url":"https://github.com/glasswalk3r.png","language":"Perl","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Perl application](https://github.com/glasswalk3r/Linux-NFS-BigDir/actions/workflows/unit-test.yml/badge.svg)](https://github.com/glasswalk3r/Linux-NFS-BigDir/actions/workflows/unit-test.yml)\n\n# Linux-NFS-BigDir\n\nThis module was created to solve a very specific problem: you have a directory over NFS, mounted by\na Linux OS, and that directory has a very large number of items (files, directories, etc). The number of entries\nis so large that you have trouble to list the contents with `readdir` or even `ls` from the shell. In extreme\ncases, the operation just \"hangs\" and will provide a feedback hours later.\n\nI observed this behavior only with NFS version 3 (and wasn't able to simulate it with local EXT3/EXT4): you might find in different situations, \nbut in that case it migh be a wrong configuration regarding the filesystem. Ask your administrator first.\n\nIf you can't fix (or get fixed) the problem, then you might want to try to use this module. It will use the `getdents`\nsyscall from Linux. You can check the documentation about this syscall with `man getdents` in a shell.\n\nIn short, this syscall will return a data structure, but you probably will want to use only the name of each entry in the directory.\n\nHow can this be useful? Here are some directions:\n\n1. You want to remove all directory content.\n2. You want to remove files from the directory with a pattern in their filename (using regular expressions, for example).\n3. You want to select specific files by their filenames and then test something else (like atime).\n\nThese are examples, but it should cover the vast majority of what you want to do. `getdents` syscall will be more effective because\nit will not call `stat` of each of those files before returning the information to you. That means, you will have the opportunity to filter\nwhatever you need and then call `stat` if you really need.\n\nI came up at `getdents` after researching about \"how to remove million of files\". After a while I reached an C program example that uses `getdents`\nto print the filenames under the directory. By using it, I was able to cleanup directories with thousands (or even millions) of files in a couple of minutes, \ninstead of many hours.\n\nThis module is a Perl implementation of that.\n\n## Install\n\nThis distribution will only work in Linux OS with perl version 5.14.4 or higher.\n\nYou will need [Dist::Zilla](https://dzil.org/) installed first. Check the `dist.ini` file for details on that.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglasswalk3r%2Flinux-nfs-bigdir","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fglasswalk3r%2Flinux-nfs-bigdir","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglasswalk3r%2Flinux-nfs-bigdir/lists"}