{"id":34087771,"url":"https://github.com/kumarmunish/pycoreux","last_synced_at":"2025-12-14T13:53:28.003Z","repository":{"id":241525228,"uuid":"473053194","full_name":"kumarmunish/pycoreux","owner":"kumarmunish","description":"Python shell utilities for file, text, and subprocess operations inspired by coreutils","archived":false,"fork":false,"pushed_at":"2025-07-31T14:46:57.000Z","size":1454,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-08T10:59:34.393Z","etag":null,"topics":["coreutils-programs","python","scripts","unix-shell"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/pycoreux","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kumarmunish.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-03-23T05:48:37.000Z","updated_at":"2025-08-03T20:30:18.000Z","dependencies_parsed_at":"2024-05-28T22:01:59.575Z","dependency_job_id":"efecbb06-5175-4903-840f-e944428a5528","html_url":"https://github.com/kumarmunish/pycoreux","commit_stats":null,"previous_names":["kumarmunish/pycoreutils","kumarmunish/pycoreux"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kumarmunish/pycoreux","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumarmunish%2Fpycoreux","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumarmunish%2Fpycoreux/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumarmunish%2Fpycoreux/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumarmunish%2Fpycoreux/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kumarmunish","download_url":"https://codeload.github.com/kumarmunish/pycoreux/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumarmunish%2Fpycoreux/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27729163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-14T02:00:11.348Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coreutils-programs","python","scripts","unix-shell"],"created_at":"2025-12-14T13:53:25.188Z","updated_at":"2025-12-14T13:53:27.994Z","avatar_url":"https://github.com/kumarmunish.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pycoreux\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"pycoreux.png\" alt=\"pycoreux - Python Unix Core Utilities\" /\u003e\n\u003c/div\u003e\n\n[![Build Status](https://github.com/kumarmunish/pycoreux/workflows/CI/badge.svg)](https://github.com/kumarmunish/pycoreux/actions)\n[![PyPI version](https://img.shields.io/pypi/v/pycoreux.svg)](https://pypi.org/project/pycoreux/)\n[![Python versions](https://img.shields.io/pypi/pyversions/pycoreux.svg)](https://pypi.org/project/pycoreux/)\n[![Tests](https://img.shields.io/github/actions/workflow/status/kumarmunish/pycoreux/ci.yml?label=tests\u0026logo=pytest)](https://github.com/kumarmunish/pycoreux/actions)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n---\n\n**A modern Python library that provides shell-like utilities for file operations, text processing, and subprocess management. Inspired by Unix coreutils, pycoreux offers a Pythonic API for building portable, scriptable command-line workflows with ease.**\n\n## Features\n\n- **File Operations**: Read, write, and manipulate files with ease\n- **Text Processing**: Count lines/words, search patterns, head/tail operations\n- **Process Management**: Execute subprocesses with simple APIs\n- **String Utilities**: Pattern matching, text manipulation\n- **Path Operations**: Directory listing, file system navigation\n- **Archive Operations**: Create and extract tar/zip archives\n- **Pipeline Support**: Chain operations together like shell pipes\n\n## Installation\n\n```bash\npip install pycoreux\n```\n\n## Quick Start: Unix Equivalents\n\nIf you're already familiar with shell scripting and the Unix toolset, here is a comprehensive guide to the equivalent pycoreux operation for each Unix command:\n\n| Unix / shell | pycoreux equivalent |\n|--------------|-------------------|\n| `cat file.txt` | `FileOps.cat(\"file.txt\")` |\n| `head -n 10 file.txt` | `FileOps.head(\"file.txt\", 10)` |\n| `tail -n 10 file.txt` | `FileOps.tail(\"file.txt\", 10)` |\n| `wc file.txt` | `FileOps.wc(\"file.txt\")` |\n| `ls -la` | `FileOps.ls(\".\", show_hidden=True, long_format=True)` |\n| `grep pattern file.txt` | `TextUtils.grep(\"pattern\", \"file.txt\")` |\n| `grep -i pattern file.txt` | `TextUtils.grep(\"pattern\", \"file.txt\", ignore_case=True)` |\n| `grep -n pattern file.txt` | `TextUtils.grep(\"pattern\", \"file.txt\", line_numbers=True)` |\n| `grep -v pattern file.txt` | `TextUtils.grep(\"pattern\", \"file.txt\", invert=True)` |\n| `sort file.txt` | `TextUtils.sort(lines)` |\n| `sort -r file.txt` | `TextUtils.sort(lines, reverse=True)` |\n| `sort -n file.txt` | `TextUtils.sort(lines, numeric=True)` |\n| `uniq file.txt` | `TextUtils.uniq(lines)` |\n| `uniq -c file.txt` | `TextUtils.uniq(lines, count=True)` |\n| `nl file.txt` | `TextUtils.nl(\"file.txt\")` |\n| `echo \"hello world\"` | `TextUtils.echo(\"hello\", \"world\")` |\n| `cut -d',' -f1 file.txt` | `TextUtils.cut(line, delimiter=\",\", fields=1)` |\n| `sed 's/old/new/g' file.txt` | `TextUtils.replace(text, \"old\", \"new\")` |\n| `find . -name \"*.txt\"` | `PathUtils.find(\".\", name=\"*.txt\")` |\n| `find . -type f` | `PathUtils.find(\".\", type_filter=\"f\")` |\n| `which python` | `ProcessUtils.which(\"python\")` |\n| `ps aux` | `ProcessUtils.ps()` |\n| `tar -czf archive.tar.gz files/` | `ArchiveUtils.tar_create(\"archive.tar.gz\", [\"files/\"], \"gz\")` |\n| `tar -xzf archive.tar.gz` | `ArchiveUtils.tar_extract(\"archive.tar.gz\")` |\n| `gzip file.txt` | `ArchiveUtils.gzip_file(\"file.txt\")` |\n| `gunzip file.txt.gz` | `ArchiveUtils.gunzip_file(\"file.txt.gz\")` |\n\n## Some Examples\n\nLet's see some simple examples. Suppose you want to read the contents of a file as a string:\n\n```python\nfrom pycoreux import FileOps\n\ncontent = FileOps.cat(\"test.txt\")\n```\n\nThat looks straightforward enough, but suppose you now want to count the lines in that file:\n\n```python\nlines, words, chars = FileOps.wc(\"test.txt\")\nprint(f\"File has {lines} lines\")\n```\n\nFor something a bit more challenging, let's try finding all lines in the file that contain \"Error\":\n\n```python\nfrom pycoreux import TextUtils\n\nerror_lines = TextUtils.grep(\"Error\", \"test.txt\")\nprint(error_lines)\n```\n\nWant to get just the first 10 lines of a file?\n\n```python\nfirst_ten = FileOps.head(\"test.txt\", 10)\nprint(first_ten)\n```\n\nLet's combine operations - read a file, find lines containing \"Error\", and get only the first 5 matches:\n\n```python\n# Read file and split into lines\ncontent = FileOps.cat(\"test.txt\")\nlines = content.split('\\n')\n\n# Filter for error lines\nerror_lines = [line for line in lines if \"Error\" in line]\n\n# Get first 5\nfirst_five_errors = error_lines[:5]\nprint('\\n'.join(first_five_errors))\n\n# Or using grep directly for the same result\nerror_output = TextUtils.grep(\"Error\", \"test.txt\")\nerror_lines = error_output.split('\\n')\nfirst_five = error_lines[:5]\nprint('\\n'.join(first_five))\n```\n\nWant to sort some data? No problem:\n\n```python\nlines = [\"zebra\", \"apple\", \"banana\", \"cherry\"]\nsorted_output = TextUtils.sort(lines)\nprint(sorted_output)\n# Output:\n# apple\n# banana\n# cherry\n# zebra\n```\n\nLet's try something more complex - count unique occurrences:\n\n```python\nlines = [\"apple\", \"apple\", \"banana\", \"apple\", \"cherry\", \"banana\"]\nunique_output = TextUtils.uniq(lines, count=True)\nprint(unique_output)\n# Output:\n#       2 apple\n#       1 banana\n#       1 apple\n#       1 cherry\n#       1 banana\n```\n\nRunning external commands:\n\n```python\nfrom pycoreux import ProcessUtils\n\n# Simple command execution\nresult = ProcessUtils.run(\"ls -la\")\nif result.success:\n    print(result.stdout)\n\n# Capture output directly\noutput = ProcessUtils.capture(\"date\")\nprint(f\"Current date: {output.strip()}\")\n\n# Find executable location\npython_path = ProcessUtils.which(\"python3\")\nprint(f\"Python is at: {python_path}\")\n```\n\nWorking with archives:\n\n```python\nfrom pycoreux import ArchiveUtils\n\n# Create a tar.gz archive\nArchiveUtils.tar_create(\"backup.tar.gz\", [\"important_files/\"], compression=\"gz\")\n\n# Extract it later\nextracted_files = ArchiveUtils.tar_extract(\"backup.tar.gz\", \"restore/\")\nprint(f\"Extracted: {extracted_files}\")\n\n# Compress a single file\ncompressed_file = ArchiveUtils.gzip_file(\"large_file.txt\")\nprint(f\"Compressed to: {compressed_file}\")\n```\n\n## A Realistic Use Case\n\nLet's use pycoreux to write a program that system administrators might actually need. Suppose we want to analyze web server logs to find the most frequent visitors. Given an Apache log file, we want to extract IP addresses and count their occurrences.\n\nIn a shell script, you might do:\n\n```bash\ncut -d' ' -f 1 access.log | sort | uniq -c | sort -rn | head -10\n```\n\nHere's the equivalent using pycoreux:\n\n```python\nfrom pycoreux import FileOps, TextUtils\n\n# Direct pipeline style - each step mirrors the Unix command\ndef analyze_access_log(log_file):\n    content = FileOps.cat(log_file)                                    # cat access.log\n    ips = [TextUtils.cut(line, ' ', 1) for line in content.split('\\n') if line.strip()]  # cut -d' ' -f 1\n    sorted_ips = TextUtils.sort(ips)                                   # sort\n    unique_counts = TextUtils.uniq(sorted_ips.split('\\n'), count=True) # uniq -c\n    reverse_sorted = TextUtils.sort(unique_counts.split('\\n'), reverse=True, numeric=True)  # sort -rn\n    return FileOps.head(content=reverse_sorted, lines=10)             # head -10\n\n# Usage\nprint(analyze_access_log(\"access.log\"))\n\n# Or using the TextUtils.pipe function for functional composition:\nfrom functools import partial\n\npipeline = TextUtils.pipe(\n    FileOps.cat,\n    lambda content: [TextUtils.cut(line, ' ', 1) for line in content.split('\\n') if line.strip()],\n    TextUtils.sort,\n    lambda ips: TextUtils.uniq(ips.split('\\n'), count=True),\n    lambda counts: TextUtils.sort(counts.split('\\n'), reverse=True, numeric=True),\n    lambda sorted_counts: FileOps.head(content=sorted_counts, lines=10)\n)\nresult = pipeline(\"access.log\")\n```\n\nOutput:\n\n```text\n      16 176.182.2.191\n       7 212.205.21.11\n       1 190.253.121.1\n       1 90.53.111.17\n```\n\n## Pipeline-Style Operations\n\nYou can chain operations together for powerful data processing, mimicking Unix shell pipelines:\n\n```python\nfrom pycoreux import FileOps, TextUtils\n\n# Example 1: Simple one-liner chaining\n# Shell equivalent: cat app.log | grep \"ERROR\" | head -5\nfirst_errors = FileOps.head(content=TextUtils.grep(\"ERROR\", content=FileOps.cat(\"app.log\")), lines=5)\nprint(first_errors)\n\n# Example 2: Multi-step chaining (easier to read)\n# Shell equivalent: cat app.log | grep \"ERROR\" | sort | head -3\ncontent = FileOps.cat(\"app.log\")                           # cat app.log\nerrors = TextUtils.grep(\"ERROR\", content=content)          # | grep \"ERROR\"\nsorted_errors = TextUtils.sort(errors.split('\\n'))         # | sort\ntop_errors = FileOps.head(content=sorted_errors, lines=3)  # | head -3\nprint(top_errors)\n\n# Example 3: Processing multiple files\n# Shell equivalent: cat *.log | grep \"WARN\" | head -10\nimport glob\nall_logs = '\\n'.join([FileOps.cat(f) for f in glob.glob(\"*.log\")])  # cat *.log\nwarnings = TextUtils.grep(\"WARN\", content=all_logs)                 # | grep \"WARN\"\nfirst_warnings = FileOps.head(content=warnings, lines=10)           # | head -10\nprint(first_warnings)\n```\n\n## CLI Usage\n\npycoreux also provides command-line tools:\n\n```bash\n# Using the CLI scripts\npython -m pycoreux.scripts.pycoreux_cli cat myfile.txt\npython -m pycoreux.scripts.pycoreux_cli head -n 5 myfile.txt\npython -m pycoreux.scripts.pycoreux_cli grep \"pattern\" myfile.txt\npython -m pycoreux.scripts.pycoreux_cli wc myfile.txt\n```\n\n## API Reference\n\n### FileOps\n\n- `cat(filepath)` - Read and return file contents\n- `head(filepath=None, lines=10, content=None)` - Return first N lines as string (from file or content)\n- `tail(filepath, lines=10)` - Return last N lines as string\n- `ls(path=\".\", show_hidden=False, long_format=False)` - List directory contents\n- `wc(filepath)` - Count lines, words, and characters\n- `touch(filepath)` - Create empty file or update timestamp\n- `mkdir(dirpath, parents=False)` - Create directory\n- `rm(filepath, recursive=False)` - Remove files or directories\n\n### TextUtils\n\n- `echo(*args, sep=\" \", end=\"\\n\")` - Join and return arguments as string\n- `grep(pattern, filepath=None, content=None, ignore_case=False, line_numbers=False, invert=False)` - Search for patterns (in file or content)\n- `nl(filepath, start=1, skip_empty=True)` - Add line numbers\n- `sort(lines, reverse=False, numeric=False)` - Sort lines\n- `uniq(lines, count=False)` - Remove duplicate consecutive lines\n- `cut(line, delimiter=\"\\t\", fields=1)` - Extract fields from line\n- `replace(text, pattern, replacement, count=0, ignore_case=False)` - Replace patterns in text\n- `wc(text)` - Count words, lines, characters in text\n- `pipe(*functions)` - Create a pipeline of functions for chaining operations\n\n### ProcessUtils\n\n- `run(command, shell=True, **kwargs)` - Execute command and return ProcessResult\n- `capture(command, **kwargs)` - Execute command and return stdout\n- `pipe(commands)` - Chain commands with pipes\n- `which(program)` - Find program in PATH\n- `kill(pid, signal=15)` - Send signal to process\n- `ps()` - List running processes\n\n### PathUtils\n\n- `find(path=\".\", name=None, type_filter=None, max_depth=None)` - Find files and directories\n- `which_all(program)` - Find all instances of program in PATH\n- `du(path, human_readable=False)` - Calculate disk usage\n- `chmod(path, mode)` - Change file permissions\n- `stat_info(path)` - Get detailed file information\n- `copy(src, dst, recursive=False)` - Copy files or directories\n- `move(src, dst)` - Move/rename files or directories\n- `symlink(target, link_name)` - Create symbolic link\n- `readlink(path)` - Read symbolic link target\n\n### ArchiveUtils\n\n- `tar_create(archive_path, files, compression=None)` - Create tar archive\n- `tar_extract(archive_path, extract_to=\".\")` - Extract tar archive\n- `tar_list(archive_path)` - List tar archive contents\n- `zip_create(archive_path, files, compression_level=6)` - Create zip archive\n- `zip_extract(archive_path, extract_to=\".\")` - Extract zip archive\n- `zip_list(archive_path)` - List zip archive contents\n- `gzip_file(file_path, output_path=None)` - Compress file with gzip\n- `gunzip_file(file_path, output_path=None)` - Decompress gzip file\n- `compress_file(file_path, method=\"gz\")` - Compress with specified method\n- `decompress_file(file_path)` - Auto-detect and decompress file\n\n## Development\n\n```bash\n# Clone the repository\ngit clone https://github.com/kumarmunish/pycoreux.git\ncd pycoreux\n\n# Install in development mode\npip install -e \".[dev]\"\n\n# Set up pre-commit hooks (optional)\npre-commit install\n\n# Run tests\npytest\n\n# Format code\nblack .\nisort .\n\n# Type checking\nmypy pycoreux\n\n# Run all checks (like CI)\nblack --check .\nisort --check-only .\nflake8 pycoreux\nmypy pycoreux\npytest\n```\n\n## Package Information\n\n**pycoreux** is published on PyPI and can be installed using pip:\n\n```bash\npip install pycoreux\n```\n\n### Package Details\n\n- **Latest Version**: 0.1.1\n- **License**: MIT\n- **Python Support**: 3.8+\n- **Platform**: Cross-platform (Windows, macOS, Linux)\n- **Dependencies**: No external dependencies required for core functionality\n\n### Development Status\n\n- **Status**: Alpha\n- **Intended Audience**: Developers, System Administrators, DevOps Engineers\n- **Use Cases**: Shell scripting in Python, file processing, log analysis, automation\n\n### Release History\n\n- **v0.1.1** (2025-07-31): Refactored to renamed functions for consistency\n- **v0.1.0** (2025-07-31): Initial release with core shell-like utilities\n\n## License\n\nMIT License – see [LICENSE](./LICENSE) file for details.\n\n## Contributing\n\nHave a shell utility you wish you could use from Python?\nSpotted a gap in the toolkit or have an idea for a new feature?\n\nContributions are welcome! Feel free to open an issue or submit a Pull Request — whether it’s for a new utility, improvement, or bug fix. \n\nLet's make pycoreux even more powerful together.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkumarmunish%2Fpycoreux","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkumarmunish%2Fpycoreux","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkumarmunish%2Fpycoreux/lists"}