{"id":22922406,"url":"https://github.com/macsteini/file-content-aggregator","last_synced_at":"2026-05-07T01:03:54.944Z","repository":{"id":276272268,"uuid":"901966751","full_name":"MacSteini/File-Content-Aggregator","owner":"MacSteini","description":"zsh and bash scripts that combine the names and cleaned content of all non-hidden files in a folder into a single output file, excluding the output file and the script itself.","archived":false,"fork":false,"pushed_at":"2024-12-11T16:42:49.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-07T09:14:40.996Z","etag":null,"topics":["aggregator","automation","bash","bash-script","file","files","merge","zsh"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MacSteini.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-11T16:42:39.000Z","updated_at":"2024-12-11T16:51:23.000Z","dependencies_parsed_at":"2025-02-08T21:30:26.110Z","dependency_job_id":null,"html_url":"https://github.com/MacSteini/File-Content-Aggregator","commit_stats":null,"previous_names":["macsteini/file-content-aggregator"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MacSteini%2FFile-Content-Aggregator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MacSteini%2FFile-Content-Aggregator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MacSteini%2FFile-Content-Aggregator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MacSteini%2FFile-Content-Aggregator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MacSteini","download_url":"https://codeload.github.com/MacSteini/File-Content-Aggregator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246655704,"owners_count":20812683,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aggregator","automation","bash","bash-script","file","files","merge","zsh"],"created_at":"2024-12-14T08:10:06.221Z","updated_at":"2026-05-07T01:03:54.903Z","avatar_url":"https://github.com/MacSteini.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# File Content Aggregator\n\n`zsh` and `bash` scripts that combine the names and cleaned content of all non-hidden files in a folder into a single output file, excluding the output file and the script itself.\n\n## Table of Contents\n\n- [Overview](#overview)\n- [Features](#features)\n- [Requirements](#requirements)\n- [Usage Instructions](#usage-instructions)\n- [How It Works](#how-it-works)\n- [Troubleshooting](#troubleshooting)\n- [Contributing](#contributing)\n- [Licence](#licence)\n\n## Overview\n\nThis project automates the creation of a combined file. The script reads through all non-hidden files in the current folder and appends their cleaned content to the output file, ensuring compatibility and clarity.\n\nAdditional functionality includes:\n\n- File extension filtering\n- Recursive folder traversal\n- Dynamic count of processed files by file type\n\n## Features\n\n- Compatible with **macOS** and the **zsh** shell\n- Cleans file content by removing BOMs and carriage return characters\n- Excludes hidden files, folders, the output file, and the script file itself\n- Supports:\n  - File extension filtering (e.g., only .txt files)\n  - Recursive folder traversal\n  - Dynamic count of processed files by extension\n- Generates a file with headers and cleaned content for each file\n\n## Requirements\n\n- **Operating System**: macOS or Linux\n- **Shell**: _zsh_ or _bash_ (v4.0+)\n- **Tools Required**:\n  - `find`\n  - `sed`\n  - `printf`\n\nThese tools are included by default in macOS and most Linux distributions.\n\n## Usage Instructions\n\n1. Open the terminal on your macOS device.\n1. Go to the folder with the files to process:\n   ```bash\n   cd /path/to/your/folder\n   ```\n1. Choose one of the following options:\n   1. Run the command in the zsh shell:\n      ```zsh\n      outfile=\"combined.txt\";extensions=\"\";rm -f \"$outfile\";[[ \"$0\" == \"-bash\" || \"$0\" == \"-zsh\" ]] \u0026\u0026 echo \"Executed as a command in the shell.\" || { script_name=$(basename \"$0\"); echo \"Executed from a script file: $script_name\"; };ext_filters=();typeset -A ext_counts;count=0;if [[ -n \"$extensions\" ]]; then ext_array=(\"${(@s/,/)extensions}\");for ext in \"${ext_array[@]}\";do ext_filters+=(-name \"*.$ext\");done;fi;LC_ALL=C find . -type f $([[ -n \"$extensions\" ]] \u0026\u0026 printf \"%s\" \"\\( ${ext_filters[@]} -o -false \\)\") -not -path '*/.*' -not -name \"$outfile\" -not -name \"$script_name\" -print0 | while IFS= read -r -d '' file;do printf \"###\\n### %s\\n###\\n\\n\" \"$file\" \u003e\u003e \"$outfile\";sed '1s/^\\xEF\\xBB\\xBF//; s/\\r$//' \"$file\" \u003e\u003e \"$outfile\";printf \"\\n\\n\" \u003e\u003e \"$outfile\";ext=\"${file##*.}\";((ext_counts[\"$ext\"]++));count=$((count + 1));done;echo \"Processing complete. $count files processed. Split by extension:\";for ext in ${(k)ext_counts};do echo \"$ext: ${ext_counts[$ext]} files\";done;echo \"Output written to $outfile.\"\n      ```\n   1. Run the following command in the bash shell:\n      ```bash\n      outfile=\"combined.txt\"; extensions=\"\"; rm -f \"$outfile\"; [[ \"$0\" == \"-bash\" || \"$0\" == \"-zsh\" ]] \u0026\u0026 echo \"Executed as a command in the shell.\" || { script_name=$(basename \"$0\"); echo \"Executed from a script file: $script_name\"; }; declare -A ext_counts; count=0; ext_filters=(); if [[ -n \"$extensions\" ]]; then IFS=',' read -ra ext_array \u003c\u003c\u003c \"$extensions\"; for ext in \"${ext_array[@]}\"; do ext_filters+=(-name \"*.$ext\"); done; fi; find_cmd=\"find . -type f\"; [[ -n \"$extensions\" ]] \u0026\u0026 find_cmd+=\" \\( ${ext_filters[@]} -o -false \\)\"; find_cmd+=\" -not -path '*/.*' -not -name '$outfile' -not -name '$script_name' -print0\"; eval \"$find_cmd\" | while IFS= read -r -d '' file; do printf \"###\\n### %s\\n###\\n\\n\" \"$file\" \u003e\u003e \"$outfile\"; sed '1s/^\\xEF\\xBB\\xBF//; s/\\r$//' \"$file\" \u003e\u003e \"$outfile\"; printf \"\\n\\n\" \u003e\u003e \"$outfile\"; ext=\"${file##*.}\"; ext_counts[\"$ext\"]=$((ext_counts[\"$ext\"] + 1)); count=$((count + 1)); done; echo \"Processing complete. $count files processed. Split by extension:\"; for ext in \"${!ext_counts[@]}\"; do echo \"$ext: ${ext_counts[$ext]} files\"; done; echo \"Output written to $outfile.\"\n      ```\n   1. Save and execute the scripts as a file:\n      1. Save the zsh or bash script in the folder with the files to process\n      1. Make it executable: `chmod +x ./file-content-aggregator.zsh` resp. `chmod +x file-content-aggregator.sh`\n      1. Run the script: `./file-content-aggregator.zsh` resp. `./file-content-aggregator.sh`\n\n## How It Works\n\n1. **Initialisation:**\n   - Sets the output file and optional file extensions\n   - Excludes hidden files, the output file, and the script file itself\n1. **File Selection:**\n   - Dynamically includes all files if extensions are empty\n   - Otherwise, filters files based on specified extensions\n1. **File Processing:**\n   - Iterates through selected files\n   - For each file:\n     - Appends a header with the file name\n     - Cleans the file content by removing BOM and carriage returns\n     - Counts files by extension\n1. **Completion:**\n   - Outputs a summary of processed files by extension\n\n## Troubleshooting\n\n- **Permission denied:** Update folder permissions with `chmod` if required and ensure the script has executable permissions: `chmod +x ./file-content-aggregator.zsh` resp. `chmod +x ./file-content-aggregator.sh`\n- **Output file not generated:** Verify that the `find`, `sed`, and `printf` commands are working on your system by testing them individually\n- **Empty output:** Ensure your file extension filters or folder contain matching files\n\n## Contributing\n\nContributions are welcome! Please follow these steps:\n\n1. Fork this repository\n1. Create a feature branch: `git checkout -b feature-branch`\n1. Commit your changes: `git commit -m \"Add feature\"`\n1. Push the branch: `git push origin feature-branch`\n1. Submit a pull request\n\nPlease ensure all changes are well-documented and tested.\n\n**Suggestions for improvements are highly encouraged!** Please ensure that your contributions adhere to the project’s coding standards and include appropriate documentation.\n\n## Licence\n\nThis project is licenced under the [MIT Licence](https://opensource.org/license/mit \"MIT Licence\"). You are free to use, modify, and distribute this project in compliance with the licence terms.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmacsteini%2Ffile-content-aggregator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmacsteini%2Ffile-content-aggregator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmacsteini%2Ffile-content-aggregator/lists"}