{"id":18886534,"url":"https://github.com/facultyai/faculty-xval","last_synced_at":"2026-04-13T17:03:21.289Z","repository":{"id":57428186,"uuid":"180628552","full_name":"facultyai/faculty-xval","owner":"facultyai","description":"Cross-validation of Keras and scikit-learn models with the Faculty platform","archived":false,"fork":false,"pushed_at":"2019-05-31T15:00:27.000Z","size":81,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-02-01T09:16:11.737Z","etag":null,"topics":["cross-validation","faculty-platform","keras","machine-learning","python","scikit-learn"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/faculty-xval/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facultyai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-2.0.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-10T17:13:31.000Z","updated_at":"2023-05-24T08:01:46.000Z","dependencies_parsed_at":"2022-09-19T04:20:27.280Z","dependency_job_id":null,"html_url":"https://github.com/facultyai/faculty-xval","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facultyai%2Ffaculty-xval","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facultyai%2Ffaculty-xval/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facultyai%2Ffaculty-xval/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facultyai%2Ffaculty-xval/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facultyai","download_url":"https://codeload.github.com/facultyai/faculty-xval/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239859567,"owners_count":19708861,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cross-validation","faculty-platform","keras","machine-learning","python","scikit-learn"],"created_at":"2024-11-08T07:28:15.611Z","updated_at":"2026-04-13T17:03:16.256Z","avatar_url":"https://github.com/facultyai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![img|small](img/logo.png)\n\n# faculty-xval\n\nCross validation of machine-learning models on Faculty platform. At present, the\npackage mostly offers a way to cross validate models in parallel by means of\nFaculty jobs. To access the functionality one makes use of the class:\n\n```python\nfaculty_xval.validation.JobsCrossValidator\n```\n\nAdditional information is found in the example notebooks provided. Please have a\nlook at the section `Try out the examples` below.\n\nThe package supports `keras` and `sklearn` models. Whilst one can write custom\nmodels that are compatible with `faculty-xval`, no guarantee is given that the\npackage handles these situations correctly, in particular because of issues\nconcerning the randomisation of weights.\n\nTwo sets of installation instructions are provided below:\n\n- If you would like to simply use `faculty-xval`, please follow the\n  `User installation instructions`.\n- If you would like to develop `faculty-xval` further, please follow the\n  `Developer installation instructions`.\n\n## User installation instructions\n\n### Create an environment\n\nIn your project on Faculty platform, create an environment named `faculty_xval`.\nIn the `PYTHON` section, select `Python 3` and `pip` from the dropdown menus.\nThen, type `faculty-xval` in the text box, and click on the `ADD` button.\n\nThe environment installs the package `faculty-xval`, and should be applied on\nevery server that you create; this includes both interactive servers and job\nservers, as explained next.\n\n### Create a job definition\n\nCreate a new job definition named `cross_validation`. In the `COMMAND` section,\npaste the following:\n\n`faculty_xval_jobs_xval $in_paths`\n\nThen, add a `PARAMETER` with the name `in_paths`, and ensure that the\n`Make field mandatory` box is checked.\n\nFinally, under `SERVER SETTINGS`, add `faculty_xval` to the `ENVIRONMENTS`\nsection.\n\nFor cross-validation jobs that are computationally intensive, we recommend using\ndedicated servers as opposed to running on shared infrastructure. To achieve\nthis, click on `Large and GPU servers` under `SERVER RESOURCES`, and select an\nappropriate server type from the dropdown menu.\n\nRemember to click `SAVE` when you are finished.\n\n## Developer installation instructions\n\n### Select a username\n\nBefore beginning the installation process, pick an appropriate username, such as\n`foo`. This does not necessarily need to match your Faculty platform username.\nIn the following instructions, your selected username will be referred to as\n`\u003cUSER_NAME\u003e`.\n\n### Clone the repository\n\nCreate the folder `/project/\u003cUSER_NAME\u003e`. Then, run the commands:\n\n```bash\ncd /project/\u003cUSER_NAME\u003e\ngit clone https://github.com/facultyai/faculty-xval.git\n```\n\n### Create an environment\n\nNext, create an environment in your project named `faculty_xval_\u003cUSER_NAME\u003e`.\n\nIn this environment, under `SCRIPTS`, paste in the following code to the `BASH`\nsection, remembering to change the `USER_NAME` definition on the second line to\nyour selected `\u003cUSER_NAME\u003e`:\n\n```bash\n# Remember to change username!\nUSER_NAME=\u003cUSER_NAME\u003e\n\n# Install faculty-xval from local repository.\npip install /project/$USER_NAME/faculty-xval/\n\n# Turn USER_NAME into an environment variable.\necho \"export USER_NAME=$USER_NAME\" \u003e /etc/faculty_environment.d/app.sh\nif [[ -d /etc/service/jupyter ]] ; then\n  sudo sv restart jupyter\nfi\n```\n\nThis environment should be applied on every server that you create; this\nincludes both 'normal' interactive servers and job servers, as explained next.\n\n### Create a job definition\n\nNext, create a new job definition named `cross_validation_\u003cUSER_NAME\u003e`. In the\n`COMMAND` section, paste the following:\n\n`faculty_xval_jobs_xval $in_paths`\n\nThen, add a `PARAMETER` with the name `in_paths`, and ensure that the\n`Make field mandatory` box is checked.\n\nFinally, under `SERVER SETTINGS`, add `faculty_xval_\u003cUSER_NAME\u003e` to the\n`ENVIRONMENTS` section.\n\nFor cross-validation jobs that are computationally intensive, we recommend using\ndedicated servers as opposed to running in the cluster. To achieve this, click\non `Large and GPU servers` under `SERVER RESOURCES`, and select an appropriate\nserver type from the dropdown menu.\n\nRemember to click `SAVE` when you are finished.\n\n## Try out the examples\n\nPlease clone this repository. Examples of cross validation with `faculty-xval`\nfor the different types of model are provided in the directories\n`examples/keras` and `examples/sklearn`. Usage instructions are then divided in\ntwo notebooks:\n\n- `jobs_cross_validator_run.ipynb` loads the data, instantiates the model, and\n  starts a Faculty job that carries out the cross validation.\n- `jobs_cross_validator_analyse.ipynb` gathers the results from the cross\n  validation, reloads the target data, and calculates the model accuracy over\n  multiple train-test splits.\n\nNote that the example notebooks must be run in the order just defined.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacultyai%2Ffaculty-xval","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacultyai%2Ffaculty-xval","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacultyai%2Ffaculty-xval/lists"}