{"id":37076329,"url":"https://github.com/diegofps/pywup","last_synced_at":"2026-01-14T08:58:45.835Z","repository":{"id":57458601,"uuid":"187112989","full_name":"diegofps/pywup","owner":"diegofps","description":"Parallelize any experiment and collect results from their stdout","archived":false,"fork":false,"pushed_at":"2020-03-11T14:04:24.000Z","size":438,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-09-29T09:02:12.251Z","etag":null,"topics":["cluster-computing","multiprocessing","plotting","regex"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/diegofps.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-16T23:10:24.000Z","updated_at":"2023-04-28T12:39:58.000Z","dependencies_parsed_at":"2022-09-09T23:12:23.894Z","dependency_job_id":null,"html_url":"https://github.com/diegofps/pywup","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/diegofps/pywup","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegofps%2Fpywup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegofps%2Fpywup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegofps%2Fpywup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegofps%2Fpywup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/diegofps","download_url":"https://codeload.github.com/diegofps/pywup/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/diegofps%2Fpywup/sbom","scorecard":{"id":341360,"data":{"date":"2025-08-11","repo":{"name":"github.com/diegofps/pywup","commit":"f5238ce64d92ef90c7e71f8fdf48e34f7bf904d7"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-18T05:58:35.002Z","repository_id":57458601,"created_at":"2025-08-18T05:58:35.002Z","updated_at":"2025-08-18T05:58:35.002Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28414729,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster-computing","multiprocessing","plotting","regex"],"created_at":"2026-01-14T08:58:45.160Z","updated_at":"2026-01-14T08:58:45.828Z","avatar_url":"https://github.com/diegofps.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pywup\nA small set of tools.\n\nTODO: Expand this [markdown](https://guides.github.com/features/mastering-markdown/)\n\n# How to install / update\n\n```bash\npip3 install --upgrade pywup --user --no-cache-dir\n```\n\nIf you cannot run the wup command after installation, check if the path to the folder .local/bin is in your PATH. If not, add the following to the end of your ~/.bashrc:\n\n```\nexport PATH=/home/\u003cYOUR_USERNAME\u003e/.local/bin:$PATH\n```\n\n# The library \n\n## configure\n\nThis is a python library for creating template-based configure scripts.\n\n```python\n#!/usr/bin/env python3\n\nfrom pywup.configure import *\n\nmf = TemplateBuilder()\n\nmf.compiler     = find_program_or_abort([\"clang++\", \"g++\"], \"Compiler\", \"clang\")\nmf.python       = find_header_or_abort(\"m/Python.h\", \"Python header\", \"python3-dev\")\nmf.highgui      = find_header_or_abort(\"/highgui.hpp\", \"Opencv's highgui\", \"opencv-dev\")\nmf.imgproc      = find_header_or_abort(\"/imgproc.hpp\", \"Opencv's imgproc\", \"opencv-dev\")\nmf.valgrind_py  = find_file_or_abort(\"python3-devel/valgrind-python.supp\", \"valgrind suppression file for python3\", \"python3-dev\")\nmf.libimgcodecs = find_lib(\"libopencv_imgcodecs.so\", \"libopencv_imgcodecs\")\n\nmf.libs         = \"-lopencv_core -lopencv_highgui -lopencv_imgproc -I../../wup/cpp/include\"\nmf.headers      = \"-DPYTHON_H=$(PYTHON_H) -DHIGHGUI_H=$(HIGHGUI_H) -DIMGPROC_H=$(IMGPROC_H)\"\n\nif mf.libimgcodecs:\n    mf.libs     += \" -lopencv_imgcodecs\"\n\nmf.build(\"Makefile\", \"\"\"\\\nCC={compiler}\nHIGHGUI_H={highgui}\nPYTHON_H={python}\nIMGPROC_H={imgproc}\nVALGRIND_PYTHON={valgrind_py}\n\nLIBS = {libs}\nHEADERS= {headers}\n\nall:\n\t$(CC) -fPIC -shared wup_wrapper.cpp -o libwup.so -Wall -O3 -std=c++11 $(LIBS) $(HEADERS)\n\nrun:\n\t$(CC) main.cpp -o main -Wall -O3 -std=c++11 $(LIBS) $(HEADERS)\n\t./main\n\ndebug:\n\t$(CC) main.cpp -o main -Wall -g -std=c++11 $(LIBS) $(HEADERS)\n\tgdb main\n\nvalgrind:\n\t$(CC) -fPIC -shared wup_wrapper.cpp -o libwup.so -Wall -O1 -g -std=c++11 $(LIBS) $(HEADERS)\n\tvalgrind --leak-check=full --show-leak-kinds=all --track-origins=yes --verbose --suppressions=$(VALGRIND_PYTHON) python3 main3.py --model wisard --dataset mnist 2\u003e valgrind.out\n\"\"\")\n```\n\n# wup\n\nThis is a command line utility for running experiments, plotting and executing repetitive tasks.\n\n## collect\n\ncollect runs a program multiple times and collects outputs.\n\nExample 1 - Permutate variables and collect outputs\n\n* This will permutate the variables THREADS (arithmetic: 1, 3, 5, 7, ...) and JOBS (geometric: 1, 1.6, ...)\n* For each configuration, the program will run 10 times\n* For each execution, the following attributes will be collected: TRAIN_TIME, TEST_TIME and ACC\n* The output will be saved in CSV format at wespa.csv\n* The permutations values for each variable are passed to the program in the same order they are declared. Each \"{}\" is replaced by the variable value.\n\n```bash\nwup collect \\\n    --p TRAIN_TIME \"Info: Time to train was ([-0-9\\.e\\+]+) μs\" \\\n    --p TEST_TIME \"Info: Time to test was ([-0-9\\.e\\+]+) μs\" \\\n    --p ACC \"Test Acc = ([0-9\\.]+)\" \\\n    --runs 10 \\\n    --va THREADS 1 17 2 \\\n    --vg JOBS 1 80000 1.6 \\\n    --o wespa.csv \\\n    --c \"./wisard -createModel RamWisard -ramBits 28 -decoder classic -ramType prime \\\n            -train mnist ../data/emnist/byclass/emnist-byclass-train \\\n            -test mnist ../data/emnist/byclass/emnist-byclass-test \\\n            -times -numThreads {} -hashSize 18041 -jobsPerThread {} -bleaching Y -pPredict 2\"\n```\n\nExample 2 - Collect multiple rows from the same command execution\n\n* The parameter --n defines a line break, allowing us to collect multiple rows for the same execution.\n* The parameter --log defines an output file to write extra details during the process.\n\n```bash\nwup collect \\\n    --n \"[a-zA-Z0-9]+ : AvgIntersectionScore = [-0-9\\.e\\+]+  AvgCenterDistanceScore = [-0-9\\.e\\+]+\" \\\n    --p DS_NAME \"([a-zA-Z0-9]+) : AvgIntersectionScore = [-0-9\\.e\\+]+  AvgCenterDistanceScore = [-0-9\\.e\\+]+\" \\\n    --p INTERSECTION \"[a-zA-Z0-9]+ : AvgIntersectionScore = ([-0-9\\.e\\+]+)  AvgCenterDistanceScore = [-0-9\\.e\\+]+\" \\\n    --p CENTER_DISTANCE \"[a-zA-Z0-9]+ : AvgIntersectionScore = [-0-9\\.e\\+]+  AvgCenterDistanceScore = ([-0-9\\.e\\+]+)\" \\\n    --runs 10 \\\n    --o \"./collect.csv\" \\\n    --log \"./collect.log\" \\\n    --c \"./run_all.sh\"\n```\n\nExample 3 - Multiple commands and parallelization\n\n* Specify multiple --c to call multiple commands. When multiple commands are presented they will receive the same variables.\n* Their output will also be concatenated, you probably want to use --n to separate them in distinct lines.\n* The program will parse all their outputs as if they were continuous, one after the other.\n* Use --jobs to define the number of parallel processes.\n* Parallelization is be applied across parameter permutation and commands. If you have 16 permutations and 2 commands we have 32 parallel tasks.\n* You may also use --jobs without multiple commands.\n* You may also use --c with just one --jobs.\n* Use parallelism to tune parameters and single process to measure times. When you are measuring execution time, multiple processes may slightly slow each other as they compete for resources.\n\n```bash\nwup collect \\\n    --n \"[a-zA-Z0-9]+ : AvgIntersectionScore = [-0-9\\.e\\+]+  AvgCenterDistanceScore = [-0-9\\.e\\+]+\" \\\n    --p DS_NAME \"([a-zA-Z0-9]+) : AvgIntersectionScore = [-0-9\\.e\\+]+  AvgCenterDistanceScore = [-0-9\\.e\\+]+\" \\\n    --p INTERSECTION \"[a-zA-Z0-9]+ : AvgIntersectionScore = ([-0-9\\.e\\+]+)  AvgCenterDistanceScore = [-0-9\\.e\\+]+\" \\\n    --p CENTER_DISTANCE \"[a-zA-Z0-9]+ : AvgIntersectionScore = [-0-9\\.e\\+]+  AvgCenterDistanceScore = ([-0-9\\.e\\+]+)\" \\\n    --jobs 4 \\\n    --runs 10 \\\n    --o \"./collect.csv\" \\\n    --log \"./collect.log\" \\\n    --c \"./run.sh 'Basketball'\" \\\n    --c \"./run.sh 'Biker'\" \\\n    --c \"./run.sh 'Bird1'\" \\\n    --c \"./run.sh 'Bird2'\"\n```\n\n## heatmap\n\nheatmap receives a csv file and generates a heatmap.\n\n```bash\nwup heatmap \\\n    --data ./wespa.csv \\\n    --y \"THREADS\" \\\n    --x \"JOBS\" \\\n    --z \"TEST_TIME\" \\\n    --tz \"data[0,z] / data[i,z]\" \\\n    --tx \"\\\"%d\\\" % int(float(data[i,x]))\" \\\n    --ty \"\\\"%d\\\" % int(float(data[i,y]))\" \\\n    --tzz \"\\\"%.2f\\\" % data[i,j]\" \\\n    --title \"Speedup (threads / blockSize)\" \\\n    --size 10 4 \\\n    --o heatmap_wespa.png\n```\n\n## bars\n\nGenerates a bar graphic using one or more csv files.\n\n```bash\nwup bars \\\n    --load ./note.csv \\\n    --line THREADS TEST_TIME \"note test\" \\\n    --line THREADS TRAIN_TIME \"note train\" \\\n \\\n    --load ./out.csv \\\n    --line THREADS TEST_TIME \"wespa test\" \\\n    --line THREADS TRAIN_TIME \"wespa train\"\\\n \\\n    --title \"Predict speedups (threads / speedup)\" \\\n    --ty \"y[0,0] / y[i,j]\" \\\n    --ts \"y[0,0]*s[i,j] / (y[i,j]**2)\" \\\n    --tyy \"\\\"%.2f\\\" % ty[i,j] if ty[i,j] else ''\" \\\n    --tx \"int(float(x[i]))\" \\\n    --xlabel \"Threads\" \\\n    --ylabel \"Speedup\" \\\n    --barwidth 0.9 \\\n    --size 10 4 \\\n    --verbose \\\n    --o bars_parallelPredictSpeedup.png\n```\n\n## backup\n\nExample ***~/Dropbox/backups/system/wup.bak*** file\n\n```\nfile;~/.local/bin/macro_play;./local_bin/\nfile;~/.local/bin/macro_rec_start;./local_bin/\nfile;~/.local/bin/macro_rec_stop;./local_bin/\nfolder;~/.config/compton;./compton\nfile;~/.vimrc;./vimrc\nfolder;~/.config/i3;./i3\n```\n\nInvoke backup / restore\n\n```bash\n# This will copy files from system (left) to backup folder (right), overwriting any change or previous file in the backup folder.\nwup backup create ~/Dropbox/backups/system/wup.bak\n\n# This will copy files from the backup folder (right) to the system (left), overwriting any change or previous file in the system.\nwup backup restore ~/Dropbox/backups/system/wup.bak\n\n# This will sync all files and backup folders, copying the most recent one to the right direction. \nwup backup sync ~/Dropbox/backups/system/wup.bak\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiegofps%2Fpywup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiegofps%2Fpywup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiegofps%2Fpywup/lists"}