{"id":31058427,"url":"https://github.com/marcelmay/hfsa","last_synced_at":"2025-09-15T07:50:58.895Z","repository":{"id":57730704,"uuid":"95222532","full_name":"marcelmay/hfsa","owner":"marcelmay","description":"Hadoop FSImage Analyzer (HFSA)","archived":false,"fork":false,"pushed_at":"2025-09-02T10:40:30.000Z","size":3581,"stargazers_count":62,"open_issues_count":4,"forks_count":24,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-09-02T11:29:47.975Z","etag":null,"topics":["fsimage","hadoop","hdfs","tool"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcelmay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE.txt","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2017-06-23T13:26:59.000Z","updated_at":"2025-09-02T10:40:33.000Z","dependencies_parsed_at":"2023-11-07T22:03:10.081Z","dependency_job_id":"23f035d7-25a5-4d9a-885d-b57d5cbda24a","html_url":"https://github.com/marcelmay/hfsa","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"purl":"pkg:github/marcelmay/hfsa","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelmay%2Fhfsa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelmay%2Fhfsa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelmay%2Fhfsa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelmay%2Fhfsa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcelmay","download_url":"https://codeload.github.com/marcelmay/hfsa/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelmay%2Fhfsa/sbom","scorecard":{"id":593131,"data":{"date":"2025-08-11","repo":{"name":"github.com/marcelmay/hfsa","commit":"64ac1ee8f08d1589f92d96d2ccd88f13b1c711fb"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.6,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/5 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: jobLevel 'actions' permission set to 'read': .github/workflows/codeql.yml:21","Info: jobLevel 'contents' permission set to 'read': .github/workflows/codeql.yml:22","Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Warn: no topLevel permission defined: .github/workflows/codeql.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":9,"reason":"11 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 9","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/marcelmay/hfsa/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:25: update your workflow using https://app.stepsecurity.io/secureworkflow/marcelmay/hfsa/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:35: update your workflow using https://app.stepsecurity.io/secureworkflow/marcelmay/hfsa/codeql.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:39: update your workflow using https://app.stepsecurity.io/secureworkflow/marcelmay/hfsa/codeql.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:50: update your workflow using https://app.stepsecurity.io/secureworkflow/marcelmay/hfsa/codeql.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:71: update your workflow using https://app.stepsecurity.io/secureworkflow/marcelmay/hfsa/codeql.yml/master?enable=pin","Info:   0 out of   6 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.txt:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE.txt:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":10,"reason":"SAST tool is run on all commits","details":["Info: SAST configuration detected: CodeQL","Info: all commits (25) are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-389x-839f-4rhx","Warn: Project is vulnerable to: GHSA-xq3w-v528-46rv","Warn: Project is vulnerable to: GHSA-4g8c-wm8x-jfhw"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-20T22:27:57.191Z","repository_id":57730704,"created_at":"2025-08-20T22:27:57.192Z","updated_at":"2025-08-20T22:27:57.192Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275225867,"owners_count":25427000,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-15T02:00:09.272Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fsimage","hadoop","hdfs","tool"],"created_at":"2025-09-15T07:50:54.942Z","updated_at":"2025-09-15T07:50:58.883Z","avatar_url":"https://github.com/marcelmay.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hadoop FSImage Analyzer (HFSA)\n\n[![Maven Central](https://img.shields.io/maven-central/v/de.m3y.hadoop.hdfs.hfsa/hfsa-parent.svg)](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22de.m3y.hadoop.hdfs.hfsa%22%20AND%20a%3A%22hfsa-parent%22)  [![javadoc](https://javadoc.io/badge2/de.m3y.hadoop.hdfs.hfsa/hfsa-lib/javadoc.svg)](https://javadoc.io/doc/de.m3y.hadoop.hdfs.hfsa/hfsa-lib)\n\n### Intro\n\nHadoop FSImage Analyzer (HFSA) complements the [Apache Hadoop 'hadoop-hdfs' tool](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html)\nby providing [HDFS fsimage](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html#The_Persistence_of_File_System_Metadata)\n* [tooling](tool) support for summary overview of the HDFS data files and directories of users and groups\n  (answering 'who has how many/big/small files...')\n* a [library](lib) for fast and partly multithreaded fsimage processing API file-, directory- and symlink aware visitor,\n  derived from [Apache HDFS FSImageLoader](https://github.com/apache/hadoop/blob/master/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java)\n* a helper [FSImage file generator](fsimage-generator) for creating synthetic test data\n\n## Example usage for library\n\nSee [FSImageLoaderTest.java](lib/src/test/java/de/m3y/hadoop/hdfs/hfsa/core/FsImageLoaderTest.java) for example usage.  \n\nThe following lines visit all directory-, file- and symlink inodes:\n```java\nRandomAccessFile file = new RandomAccessFile(\"src/test/resources/fsi_small.img\", \"r\");\n\n// Load file into memory\nFsImageData fsimageData = new FsImageLoader.Builder()\n    .parallel().build()\n    .load(file);\n\n// Traverse file hierarchy\nnew FsVisitor.Builder()\n    .parallel()\n    .visit(fsimageData, new FsVisitor() {\n        @Override\n        public void onFile(FsImageProto.INodeSection.INode inode, String path) {\n            // Do something\n            String fileName = (\"/\".equals(path) ? path : path + '/') + inode.getName().toStringUtf8();\n            System.out.println(fileName);\n            FsImageProto.INodeSection.INodeFile f = inode.getFile();\n            PermissionStatus p = loader.getPermissionStatus(f.getPermission());\n            ...\n        }\n             \n        @Override\n        public void onDirectory(FsImageProto.INodeSection.INode inode, String path) {\n            // Do something\n            final String dirName = (\"/\".equals(path) ? path : path + '/') + inode.getName().toStringUtf8();\n            System.out.println(\"Directory : \" + fileName);\n            \n            FsImageProto.INodeSection.INodeDirectory d = inode.getDirectory();\n            PermissionStatus p = loader.getPermissionStatus(d.getPermission());\n            ...\n        }\n             \n        @Override\n        public void onSymLink(FsImageProto.INodeSection.INode inode, String path) {\n            // Do something\n        }\n    }\n);\n```\n        \n### Requirements\n\n- JDK 1.8 (11 recommended for running)\n- Hadoop 2.x or 3.x fsimage  \n  Note: hfsa lib version 1.2+ has Hadoop 3.x dependencies but still works for Hadoop 2.x fsimages\n- Maven 3.9.x (for building from source)\n\n### Building\n\n```\nmvn clean install\n```\n\n### Roadmap\n\n- Configurable strategy for fast-but-memory-intensive or slow-but-memory-friendly fsimage loading\n- Report and config options for topk/sorting/selection/...\n\n### License\n\nHFSA is released under the [Apache 2.0 license](LICENSE.txt).\n\n```\nCopyright 2017-2025 Marcel May and project contributors\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    https://www.apache.org/licenses/LICENSE-2.0\n\n    Unless required by applicable law or agreed to in writing, software\n    distributed under the License is distributed on an \"AS IS\" BASIS,\n    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n    See the License for the specific language governing permissions and\n    limitations under the License.\n```\n\nContains work derived from Apache Hadoop.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcelmay%2Fhfsa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcelmay%2Fhfsa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcelmay%2Fhfsa/lists"}