{"id":24922872,"url":"https://github.com/liddiard/stream-editor","last_synced_at":"2025-10-11T20:50:15.722Z","repository":{"id":34961147,"uuid":"39036700","full_name":"liddiard/stream-editor","owner":"liddiard","description":"A web tool for interactively using and chaining command-line text manipulation utilities like sed, grep, and awk.","archived":false,"fork":false,"pushed_at":"2024-07-18T22:38:55.000Z","size":8617,"stargazers_count":32,"open_issues_count":7,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-09T19:15:47.533Z","etag":null,"topics":["bash","flask","grep","react","regular-expressions","sed","shell","stream-editor"],"latest_commit_sha":null,"homepage":"https://streameditor.io","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/liddiard.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-07-13T20:51:57.000Z","updated_at":"2025-01-11T18:08:14.000Z","dependencies_parsed_at":"2024-07-18T23:14:32.263Z","dependency_job_id":null,"html_url":"https://github.com/liddiard/stream-editor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/liddiard%2Fstream-editor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/liddiard%2Fstream-editor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/liddiard%2Fstream-editor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/liddiard%2Fstream-editor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/liddiard","download_url":"https://codeload.github.com/liddiard/stream-editor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248094987,"owners_count":21046770,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","flask","grep","react","regular-expressions","sed","shell","stream-editor"],"created_at":"2025-02-02T11:33:59.902Z","updated_at":"2025-10-11T20:50:10.674Z","avatar_url":"https://github.com/liddiard.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cstrong\u003e\u003cem\u003eFound a bug? [See info below on how to report it.](#reporting-bugs)\u003c/em\u003e\u003c/strong\u003e\n\n# \u003cimg src=\"https://github.com/liddiard/stream-editor/blob/master/client/public/img/logo-light.svg?raw=true\" height=\"36\" alt=\"Stream Editor logo\" /\u003e Stream Editor | [streameditor.io](https://streameditor.io/)\n\n**Stream Editor** is a web tool for interactively using and chaining command-line text manipulation utilities, such as `sed`, `grep`, and `awk`.\n\nText transformation tools provided by Unix operating systems are incredibly powerful for pattern extraction, formatting, and data manipulation, but a command line isn't always the best interface for using them when it comes to experimentation and debugging, especially if you want to chain several commands together with pipes and understand what each is doing.\n\nStream Editor provides a user-friendly web interface for tinkering with these text editing commands that dynamically updates output as you type. It enables you to chain multiple commands together and observe the output after each step, optionally seeing a diff of added/deleted text with green/red highlights. \n\nOnce you've finalized the operations you want to use, Stream Editor lets you export them as a series of command-line pipes with a single click, or share them with a unique URL.\n\n[**Read more on my website.**](https://harrisonliddiard.com/project/stream-editor/)\n\n## Screenshots\n\n### Stream Editor default view (dark theme)\n\n![Stream Editor screenshot](screenshots/initial.png)\n\n### Stream Editor light theme, showcasing its full Unicode support\n\n![Stream Editor screenshot](screenshots/unicode.png)\n\n### Stream Editor being used to analyze its own logs, showing the effects of multiple commands chained together\n\n![Stream Editor screenshot](screenshots/chain.png)\n\n## Reporting bugs\n\n### Security bugs\n\nFound a security bug related to this codebase or how Stream Editor is deployed at [streameditor.io](https://streameditor.io)? I highly encourage and kindly request that you report it by emailing:\n\n```\nsecurity [at] streameditor [dot] io\n```\n\nPlease privately email me instead of posting about it publicy on GitHub Issues or elsewhere, and please include your steps to reproduce.\n\nThe way Stream Editor is set up at [streameditor.io](https://streameditor.io) is intended to **prevent**:\n\n- arbitrary command execution\n- writing files to the server\n- reading any *sensitive* data from the server\n- otherwise gaining access to the server\n- adversely affecting others' ability to use [streameditor.io](https://streameditor.io) in a significant way \n\nValid security bugs are likely to include anything that allows you to do these things. Note that reading certain *non-sensitive* files from the server is possible and expected.\n\n### Other bugs\n\nIf your bug **does not** involve any security concerns, please report it on [GitHub Issues](https://github.com/liddiard/stream-editor/issues).\n\n## Installation (local development)\n\n### System requirements / prerequisites\n\n- MacOS or Linux*\n- Python 3.12 w/ pip 3\n- Node.js 20 w/ npm 10\n\n\\* The Unix text editing commands that Stream Editor supports are not all available on Windows, though you *may* be able to get everything to work in a [Cygwin](https://www.cygwin.com/) kind of environment. It just hasn't been tested.\n\n### Instructions\n\n1. Clone the repo.\n2. (optional but recommended) [Set up a Python 3 virtual environment](https://docs.python.org/3/tutorial/venv.html) for the project, and activate it.\n3. Install the server dependencies: `pip3 install -r requirements.txt`\n4. Start the Flask development server: `FLASK_ENV=development python3 dev_server.py`\n5. Change to the client directory: `cd client`\n6. Install the client dependencies: `npm install`\n7. Start the client-side development server: `npm start`\n\n## Notes for server deployment\n\n**Note:** This isn't a comprehensive guide; this section is intended mainly for my personal reference.\n\nServer is running with [gunicorn](https://gunicorn.org/):\n\n```shell\nJAIL_PATH=/root/jail gunicorn --name stream-editor server:app\n```\n\nTo auto-restart (recommended), it can be installed as a systemd service at `/etc/systemd/system/stream-editor.service`:\n\n```ini\n[Unit]\nDescription=Stream Editor\nAfter=network.target\n\n[Service]\nUser=root\nGroup=www-data\nWorkingDirectory=/home/liddiard/stream-editor/repo\nEnvironment=\"JAIL_PATH=/root/jail\"\n# https://askubuntu.com/a/1014501\nExecStart=/bin/bash -c \"PATH=/home/liddiard/stream-editor/bin:$PATH exec /home/liddiard/stream-editor/bin/gunicorn --name stream-editor server:app\"\nRestart=always\nRestartSec=5\n\n[Install]\nWantedBy=multi-user.target\n```\n\nEnable the service by running the following:\n\n```bash\nsudo systemctl daemon-reload\nsudo systemctl start stream-editor.service\nsudo systemctl enable stream-editor.service\nsudo systemctl status stream-editor.service\n```\n\nIt is also running behind a simple Nginx reverse proxy. Config:\n\n```Nginx\nserver {\n\n    server_name api.streameditor.io;\n\n    location / {\n        proxy_pass http://localhost:8000;\n        proxy_set_header Host $host;\n        proxy_set_header X-Real-IP $remote_addr;\n        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n        proxy_set_header X-Forwarded-Proto $scheme;\n    }\n\n}\n```\n\n- chroot jail is configured with [Jailkit](https://olivier.sessink.nl/jailkit/); [this post](http://www.mattheakis.com/blog/view.php?name=setting_up_a_jail_to_safely_execute_code) was a useful reference\n- after Jailkit is installed (you have to build it from source), the script in `scripts/create_jail.sh` is supposed to do all the setup for the jail\n- the man pages generated in `client/public/manpages/` are specific to the versions running on whatever machine you're using\n- generate new man pages with the script in `scripts/generate_man_html.sh`. I did a lot of massaging of its output with regex find-and-replacing.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fliddiard%2Fstream-editor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fliddiard%2Fstream-editor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fliddiard%2Fstream-editor/lists"}