{"id":17655156,"url":"https://github.com/ickc/htcondor_ex","last_synced_at":"2025-03-30T09:26:51.694Z","repository":{"id":178648015,"uuid":"662177051","full_name":"ickc/htcondor_ex","owner":"ickc","description":null,"archived":false,"fork":false,"pushed_at":"2024-01-26T15:46:01.000Z","size":96,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-05T11:36:41.981Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ickc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-04T14:16:27.000Z","updated_at":"2023-07-08T19:50:41.000Z","dependencies_parsed_at":"2024-01-09T00:25:31.177Z","dependency_job_id":"f7d5deed-74a5-4ca3-884f-b7a809c151a5","html_url":"https://github.com/ickc/htcondor_ex","commit_stats":null,"previous_names":["ickc/htcondor_ex"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ickc%2Fhtcondor_ex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ickc%2Fhtcondor_ex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ickc%2Fhtcondor_ex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ickc%2Fhtcondor_ex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ickc","download_url":"https://codeload.github.com/ickc/htcondor_ex/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246299387,"owners_count":20755152,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-23T12:40:59.070Z","updated_at":"2025-03-30T09:26:51.669Z","avatar_url":"https://github.com/ickc.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# htcondor_ex\n\nSome examples to run jobs using HTCondor. It is mostly generic, but it is tested on [the Blackett Computing Facility – at the University of Manchester](https://www.blackett.manchester.ac.uk/) with HTCondor 9.0.17.\n\n# Examples\n\nUnder the directory `examples/`, you will find subdirectories of examples.\n\n- Conventions\n\n    - Makefiles: Inside each subdirectory, there's at least 2 make targets, `clean` and `submit`. `make submit` would submit the job, and `make clean` would cleanup the outputs from a job. Study the `submit` target will let you know which file is the job configuration file.\n\n    - Job configuration files: Most job configuration files here are `.ini` for syntax parser, and some are of different extensions to preserve the original code after copy \u0026 paste.\n\n- Compilation: binaries are put inside `bin/`\n    - Pre-compiled binaries: `make download` and it will download precompiled binaries compiled using `module load mpi/openmpi3-x86_64`.\n    - To compile it yourself, the only way to do it as of writing is to request an interactive node first, inside that, run `module load mpi/openmpi3-x86_64` then `make`. How you transfer it back to the login node is up to you. If you want to use rsync, setup your ssh keys beforehand.\n- Wrapper script: Running MPI via HTCondor requires a wrapper script.\n    - `openmpiscript`: HTCondor provided wrappers for OpenMPI and MPICH. The wrapper for OpenMPI, `openmpiscript`, does not work out of the box. A modified version is provided at `src/`.\n    - `cbatch_openmpi.sh`: a refactored version of `openmpiscript` to provide user more control over the environment and the mpirun command to use. This is more similar to `sbatch` from SLURM for example.\n    \n        - usage: `cbatch_openmpi.sh \u003cenv.sh\u003e \u003cmpirun.sh\u003e`\n        - the `\u003cenv.sh\u003e` is a script that setup the environment, including OpenMPI.\n        \n            Example:\n            \n            ```sh\n            module load mpi/openmpi3-x86_64\n            ```\n\n        - the `\u003cmpirun.sh\u003e` is a script that runs mpirun with the desired arguments,\n        - this script can either setup the host on their own, or use the 2 convenience functions provided:\n            - `set_OMPI_HOST_one_slot_per_condor_proc` setup one slot per condor process, useful for hybrid-MPI\n            - `set_OMPI_HOST_one_slot_per_CPU` setup one slot per CPU\n        \n            Example:\n\n            ```sh\n            set_OMPI_HOST_one_slot_per_condor_proc\n            mpirun -host \"$OMPI_HOST\" ...\n            ```\n\nDescriptions:\n\n`eg01` and `eg02`\n: from [Users’ Quick Start Guide — HTCondor Manual 10.6.0 documentation](https://htcondor.readthedocs.io/en/latest/users-manual/quick-start-guide.html).\n\n`interactive`\n: a minimal example on submitting interactive job. Note that from HTCondor documentation,\n\n    \u003e The interactive job is a vanilla universe job.\n\n    And therefore you cannot submit interactive job in the parallel universe (i.e. multi-nodes jobs) for example.\n\n`parallel` and `parallel-cpu`\n: minimal examples to submit jobs to the parallel universe, modified from the HTCondor manual.\n\n`mpi-hello-world-vanilla`\n: Running MPI executables in vanilla universe. This is simple to do.\n\n`mpi-hello-world`\n: Running MPI executables in parallel universe. Note the use of our own `openmpiscript` here.\n\n`cbatch-*`\n: Variants of MPI jobs that uses our own `cbatch_openmpi.sh` instead of `openmpiscript`.\n\n`debug*`\n: These are various jobs to test the system. Feel free to study and use them.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fickc%2Fhtcondor_ex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fickc%2Fhtcondor_ex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fickc%2Fhtcondor_ex/lists"}