{"id":17748540,"url":"https://github.com/purarue/bgproc","last_synced_at":"2026-03-06T06:03:44.320Z","repository":{"id":38682037,"uuid":"279770778","full_name":"purarue/bgproc","owner":"purarue","description":"bash loop to run tasks in the background. used as an anacron alternative","archived":false,"fork":false,"pushed_at":"2024-11-12T16:23:14.000Z","size":116,"stargazers_count":13,"open_issues_count":2,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-22T18:08:46.554Z","etag":null,"topics":["anacron","background-jobs","bash","cron","task-scheduler"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/purarue.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-07-15T05:14:34.000Z","updated_at":"2025-05-01T13:33:27.000Z","dependencies_parsed_at":"2025-06-22T18:08:48.197Z","dependency_job_id":"a1820880-e96f-442f-8bf5-564d3005b56c","html_url":"https://github.com/purarue/bgproc","commit_stats":null,"previous_names":["purarue/bgproc","seanbreckenridge/bgproc"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/purarue/bgproc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/purarue%2Fbgproc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/purarue%2Fbgproc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/purarue%2Fbgproc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/purarue%2Fbgproc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/purarue","download_url":"https://codeload.github.com/purarue/bgproc/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/purarue%2Fbgproc/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30164532,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T04:43:31.446Z","status":"ssl_error","status_checked_at":"2026-03-06T04:40:30.133Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anacron","background-jobs","bash","cron","task-scheduler"],"created_at":"2024-10-26T10:02:14.297Z","updated_at":"2026-03-06T06:03:43.403Z","avatar_url":"https://github.com/purarue.png","language":"Shell","funding_links":[],"categories":["Shell"],"sub_categories":[],"readme":"# bgproc\n\nA `bash` loop to run tasks in the background. Used as a `anacron` alternative.\n\nThis uses [`evry`](https://github.com/purarue/evry) to schedule commands/run things periodically. `evry` saves persistent files with timestamps to the computer for each job, which means this follows `anacron`s philosophy - the computer doesn't have to be running 24 x 7. `evry` checks when tasks were last run, and if that duration has elapsed (e.g. `2 days`), it runs the task.\n\n## How?\n\nThis runs any other files it finds recursively with `find` from the current directory that end with `.job`. You can alternatively provide directories which contain `.job` files as positional arguments.\n\nA potential `.job` file might look like:\n\n```bash\n#!/bin/bash\n# backup the logfile from my server once a day\n\nevry 1 day -backup_logfile \u0026\u0026 {\n  scp vps_server:~/app.log ~/.cache/app.log\n}\n```\n\nThis runs each `.job` file explicitly with `bash`, but you could easily write a wrapper like:\n\n```bash\n#!/bin/bash\n# every 2 days, run some python script\n\nevry 2 days -my_task \u0026\u0026 {\n  printlog \"running python script...\"\n  exec python3 /usr/local/bin/run_task.py\n}\n```\n\n## Usage\n\n```\nUsage: bgproc [-h] [-nodpqjJ] [-F \u003cn\u003e] [-t \u003cf\u003e] [DIR...]\nRuns tasks in the background. Run without flags to start the background loop\n\t-n\tDon't search directories recursively (add -maxdepth 1)\n\t-o\tRuns the task loop once\n\t-d\tRuns the task loop once, in debug mode\n\t-p\tRuns the task loop thrice, to pretty print debug info\n\t-F \u003cn\u003e\tRuns the jobs in parallel, with \u003cn\u003e jobs at a time\n\t-t \u003cf\u003e\tRuns the job file \u003cf\u003e, can be used to test a job before adding it to your directory\n\t-q\tQuiet mode, silences any logs\n\t-j\tPrint paths of all jobs, then exit\n\t-J\tPrint paths of all job directories, then exit\nAny additional arguments should be directories which contain '.job' files\nIf no directories are provided, searches from the current directory recursively\nSee https://github.com/purarue/bgproc for more info\n```\n\nThe `-F` option does not attempt to print/schedule jobs in order, it just forks and waits for them to finish. So, outputs from the commands may overlap\n\nTo test a `.job` file before adding it to your `bgproc` directory, you can use the `-t` option. For example:\n\n`EVRY_DEBUG=1 bgproc -t ./dir/something.job`\n\nSee [here](https://gist.github.com/purarue/e7ad77320c065d96f282f6d45deaa842) for example debug output.\n\nThis offers a few ways to run the task loop once, `-o` (once), `-d` (with lots of debug information), `-F \u003cn\u003e` to run `\u003cn\u003e` jobs in parallel, or `-p`, in pretty mode, which tells you when each job will run next:\n\n```\n$ bgproc -p ~/.local/scripts/supervisor_jobs/linux\nglue-update-cubing-json - 5 days, 1 hour, 49 minutes, 47 seconds\nwarn_mailsync - 9 minutes, 27 seconds\nbg-my-feed-index - 34 minutes, 48 seconds\ncopy_images - 4 minutes, 27 seconds\nlinkmusic - 46 minutes, 13 seconds\nguestbook_comments - 10 minutes, 18 seconds\nupdaterss - 2 minutes, 45 seconds\n```\n\n### Install\n\nCopy the `bgproc` script onto your `$PATH` somewhere and make it executable. To automate that, you could use [`basher`](https://github.com/basherpm/basher):\n\n```\nbasher install purarue/bgproc\n```\n\n### Logs\n\nIf you want to save logs somewhere else, you can set the `BGPROC_LOGFILE` environment variable to a different location. Defaults to saving temporary logs at `/tmp/bgproc.log`\n\nLogs are very basic, just saves the timestamp and the message passed like:\n\n```\n1613892690:Starting loop...\n1613892693:updaterss:updated RSS feeds:0\n```\n\nBoth the [`printlog` and `send-error`](https://github.com/purarue/bgproc/blob/2b4a2a021bd0ccf0d7ea8d2557e8c5c816e05b49/bgproc#L34-L54) functions are exported into the bash environment, so they're accessible from any bash scripts `bgproc` runs. Both of those accept one argument - the text to print. `send-error` sends a OS notification if possible, using `notify-send` on linux and `osascript` (AppleScript) on mac.\n\nFor reference, my jobs often follow a structure like this:\n\n```bash\n#!/usr/bin/env bash\n\nevry 2 hours -somecommand \u0026\u0026 {\n  printlog \"some command: running...\"  # saves timestamp to logfile\n  somecommand || send-error \"some command failed...\"  # notifies me if this fails\n}\n```\n\n### Configuration\n\nThis waits for `60 seconds` between running jobs, if you want to increase/change that, you can set the `BGPROC_SLEEPTIME` environment variable. To wait for 10 minutes between trying to run jobs: `BGPROC_SLEEPTIME=600 bgproc`\n\nIf you want to run multiple `bgproc` instances for different directories/jobs, put `bgproc` on your `$PATH`, and set the `BGPROC_LOCKFILE` environment variable to allow multiple instances of `bgproc` to run at the same time:\n\n```bash\nBGPROC_LOCKFILE=/tmp/personal_jobs.lock BGPROC_LOGFILE=/tmp/personal_logs bgproc /some/other/directory\n```\n\nTo change the date format, you can set the `BGPROC_DATE_FMT` environment variable, that is passed to the `date` command, for more info see `man date`:\n\n```bash\nBGPROC_DATE_FMT='+%Y-%m-%dT%H-%M-%S' ./bgproc -o ./jobs\n```\n\nWhen running a loop, whenever this finishes running each job and before it sleeps, it updates the timestamp on `$BGPROC_LASTRUNFILE` (defaults to `~/.cache/bgproc.lastrun`). You can use this to check when the loop last finished, or to possibly notify you if some task is hanging your loop entirely.\n\n```bash\n# to figure out how long its been since loop last finished in seconds\necho \"$(date +%s)\" - \"$(stat -c'%Y' ~/.cache/bgproc.lastrun)\" | bc\n220\n```\n\n## bgproc_on_machine\n\nI use `bgproc` on all of my machines and my phone, so `bgproc_on_machine` handles the task of figuring out which machine I'm currently on, so the correct background `.job`s can run. That uses [`on_machine`](https://github.com/purarue/on_machine) internally, which generates a unique hash, like: `linux_arch` or `android_termux`.\n\nAfter setting the `$BGPROC_PATH` environment variable:\n\n```bash\n$ export BGPROC_PATH=\"${HPIDATA}/jobs:${HOME}/.local/scripts/supervisor_jobs:${REPOS}/HPI-personal/jobs\"\n$ bgproc_on_machine -o\n1655993990:Searching for jobs in:\n1655993990:/home/user/data/jobs/all\n1655993990:/home/user/data/jobs/linux\n1655993990:/home/user/.local/scripts/supervisor_jobs/all\n1655993990:/home/user/.local/scripts/supervisor_jobs/linux\n1655993990:/home/user/Repos/HPI-personal/jobs/all\n1655993990:/home/user/Repos/HPI-personal/jobs/linux\n```\n\nYou can see examples of those directory structures [in my dotfiles](https://github.com/purarue/dotfiles/tree/master/.local/scripts/supervisor_jobs) and [in my personal HPI repo](https://github.com/purarue/HPI-personal/tree/master/jobs):\n\n```\njobs\n├── all\n│   ├── backup_bash.job\n│   ├── backup_browser_history.job\n│   ├── backup_ipython.job\n│   ├── backup_zsh_history.job\n│   ├── doctor_snapshot.job\n│   ├── minecraft_advancements.job\n│   └── runelite_screenshots.job\n├── android\n├── linux\n│   ├── backup_albums.job\n│   ├── backup_bash_server_history.job\n│   ├── backup_chess.job\n│   ├── backup_garmin.job.disabled\n│   ├── backup_ghexport.job\n│   ├── backup_git_doc_history.job\n│   ├── backup_listenbrainz.job\n│   ├── backup_rexport.job\n│   ├── backup_stexport.job\n│   ├── backup_trakt.job\n│   └── mint.job\n└── mac\n    ├── backup_imessages.job\n    └── backup_safari_history.job\n```\n\n#### Background Service\n\nThis doesn't ship way to run this automatically for each operating system. Could potentially use a `systemd` service (on linux flavors that have that) or an [Automator script](https://stackoverflow.com/questions/6442364/running-script-upon-login-mac) on macOS.\n\nPersonally, I run this with [`supervisor`](https://github.com/Supervisor/supervisor) (since it being cross platform means my background processes are cross platform as well) at the beginning of my X session on linux, and, otherwise [checking if the pid file exists when I open a terminal](https://github.com/purarue/dotfiles/blob/master/.local/scripts/README.md#supervisor)\n\nOn `android` where handling background tasks is a bit more complicated, instead of using supervisor to run `bgproc` in the background, I use the `-F` flag to run the loop once [when I open my terminal](https://github.com/purarue/dotfiles/blob/master/.config/zsh/android.zsh). In my shell profile for `termux`, I have:\n\n`evry 1 hour -run_android_jobs \u0026\u0026 bgproc_on_machine -onqF 4`\n\nSince that uses `-F 4` (run 4 jobs in parallel), jobs finish relatively quickly and I don't have to wait long for jobs to run before I can interact with the terminal.\n\n### Performance\n\nDespite this running `evry` all the time to check if times have elapsed, it doesn't cause much of a footprint, each call takes about `1ms`:\n\n```\n$ hyperfine --warmup 3 -i 'evry 1 day -run_benchmark'\nBenchmark #1: evry 1 day -run_benchmark\n  Time (mean ± σ):       1.0 ms ±   0.7 ms    [User: 0.6 ms, System: 0.6 ms]\n  Range (min … max):     0.1 ms …   3.0 ms    1329 runs\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpurarue%2Fbgproc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpurarue%2Fbgproc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpurarue%2Fbgproc/lists"}