{"id":23865044,"url":"https://github.com/divya063/testingframework","last_synced_at":"2026-06-18T02:31:49.509Z","repository":{"id":107266559,"uuid":"186783992","full_name":"Divya063/TestingFramework","owner":"Divya063","description":"Testing framework for Jupyter notebooks","archived":false,"fork":false,"pushed_at":"2019-08-24T07:09:02.000Z","size":410,"stargazers_count":3,"open_issues_count":3,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-22T13:39:32.311Z","etag":null,"topics":["cern","cvmfs","eos","jupyter-notebooks","jupyterhub-api","python3","sqlite-database","storage","swan","testing-framework"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Divya063.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-15T08:32:46.000Z","updated_at":"2024-12-25T13:49:57.000Z","dependencies_parsed_at":"2023-07-15T07:16:04.315Z","dependency_job_id":null,"html_url":"https://github.com/Divya063/TestingFramework","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Divya063/TestingFramework","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Divya063%2FTestingFramework","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Divya063%2FTestingFramework/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Divya063%2FTestingFramework/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Divya063%2FTestingFramework/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Divya063","download_url":"https://codeload.github.com/Divya063/TestingFramework/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Divya063%2FTestingFramework/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34474063,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-18T02:00:06.871Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cern","cvmfs","eos","jupyter-notebooks","jupyterhub-api","python3","sqlite-database","storage","swan","testing-framework"],"created_at":"2025-01-03T08:47:20.698Z","updated_at":"2026-06-18T02:31:49.490Z","avatar_url":"https://github.com/Divya063.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"**[Installation](#setup-instructions)** |\n**[How to use](#how-to-use)** |\n**[How to write tests](write_test.md)**|\n**[Code](https://github.com/Divya063/TestingFramework)**\n\n# Google Summer of Code 2019\n\n# [Testing Framework for Jupyter notebooks](https://summerofcode.withgoogle.com/projects/#5216539194687488)\n\n\u003ccenter\u003e\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ca href=\"https://summerofcode.withgoogle.com/projects/#5216539194687488\"\u003e\u003cimg src=\"https://user-images.githubusercontent.com/6822941/29750351-e95e7b1c-8b5b-11e7-9f6b-b25b69f7353a.png\"/\u003e\u003c/a\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ca href=\"http://hepsoftwarefoundation.org/\"\u003e\u003cimg src=\"https://user-images.githubusercontent.com/6822941/29750350-e956b512-8b5b-11e7-9e34-4e3a5be9d37f.png\"/\u003e\u003c/a\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ca href=\"https://swan.web.cern.ch/\"\u003e\u003cimg src=\"https://avatars0.githubusercontent.com/u/38285709?s=200\u0026v=4\"/\u003e\u003c/a\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\u003c/center\u003e\n\n## Mentors\n- Enrico Bocchi\n- Diogo Castro\n- João Vicente\n- Jakub Moscicki\n- Enric Tejedor\n\n## Introduction\n\nSWAN (Service for Web-based ANalysis) is a cloud data analysis service developed and powered by CERN that provides Jupyter notebooks on demand. It is based on Jupyter upstream technology but it is deeply integrated with CERN-specific services, e.g., EOS which provides storage to SWAN , CVMFS which is used to retrieve software on the fly. Jupyter notebooks, despite being easily accessible from an intuitive web-based interface, are a complex environment, especially when used together with JupyterHub, custom extensions, external storage backends and computational clusters. This project aims at creating a testing framework for both upstream Jupyter components and SWAN-specific components which will allow the addition of new tests to cover new features of the SWAN service and will be able to run synthetic tests. The testing framework is self contained and  includes functional tests as well as performance tests.\n\nThis testing framework covers the following components:\n\nUpstream Components :\n1. configurable-http-proxy\n2. [JupyterHub API](#jupyterhub-api)\n3. [SQLite database managed by JupyterHub](#sqlite-database)\n4. [SWAN docker containers](#container)\n\nCERN-specific components :\n1. [The EOS filesystem](#storage)\n2. [The CVMFS repositories](#cvmfs)\n\n\n## Setup Instructions\n\n- This project assumes a [SWAN](https://swan.web.cern.ch/) or [ScienceBox](https://github.com/cernbox/uboxed) setup. ScienceBox contains SWAN and all the other CERN-specific services (i.e., EOS, CERNBox, CVMFS). To install required dependencies:\n\n```bash\ngit clone https://github.com/Divya063/TestingFramework.git\ncd TestingFramework\npip3 install -r requirements.txt\n```\n\n## How to use\n\nThere are two testing modes:\n - From the host machine (default)\n - From containers (sciencebox) - To run the test from user container use the flag `-u` and pass the \n session name as `--session {name}`, e.g- `--session user2`\n \n \nTo run tests from host machine use the command - `python3 run.py --configfile [path of yaml file]`, while running this command it is assumed that all the parameters (required to run the tests) are provided in the [test.yaml]() file or in some cases([JupyterHub API](#warning)) necessary configuration has already been done. \u003cbr\u003e\n\n### Storage\n\n- To run the test from user container use the command given below (there is no need to provide path argument as it will\nbe already set in test.yaml file.)\u003cbr\u003e\n`python3 run.py -u --session user2 --test storage --configfile test.yaml`\n\n**Parameters**\n```yaml\n    mount_sanity:\n      timeout: 8\n      mountpoints: ['user/u/user2/']\n    mount:\n      user_mode : 'sciencebox'\n    write:\n      fileSize: 1M\n      filepath: \"eos/user/u/user2/0.txt\"\n    delete:\n      filepath: \"eos/user/u/user2/0.txt\"\n    exists:\n      filepath: \"eos/user/u/user2/0.txt\"\n    throughput: \n      fileNumber: 10\n      fileSize: 1M\n      filepath: \"eos/user/u/user2/\"\n    checksum:\n      fileNumber: 10\n      fileSize: 1M\n      filepath: \"eos/user/u/user2/\"\n```\n1. mount_sanity:\n    - Test file : test_mount_sanity.py\n    - Use case : Given a list of mount points checks if a mount point is hanging\n    - To run this test explicitly use -  `python3 test_mount_sanity.py --timeout 5 --mount_points user user/u`\n\n2. mount:\n    - Test file : test_mount.py\n    - Use case : Checks if eos is mounted on host and sciencebox\n    - There are two testing modes:\n      - \"host\"\n      - \"sciencebox\"\n    -  To run this test explicitly use - `python3 test_mount.py --mode host`\n    \n3. throughput:\n    - Test file : test_throughput.py\n    - Use case : Benchmark read-write performance and compute the read and write throughput.\n    - To run this test explicity use - `python3 test_throughput.py --num 10 --file-size 1M --dest eos/user/u/user2`\n    \n4. checksum : \n    - Test file : test_checksum.py\n    - Use case : Calculates the checksum\n    - To run this test explicity use - `python3 test_checksum.py --num 10 --file-size 1M --dest eos/user/u/user2`\n\n### CVMFS\n\n- To run the test from user container use the command given below.\u003cbr\u003e\n`python3 run.py -u --session user2 --test CVMFS --configfile test.yaml`\n\n**Parameters**\n\n```yaml\ncvmfs:\n    mount:\n      repoName: 'sft.cern.ch'\n      repoPath: 'cvmfs/sft.cern.ch/'\n    ttfb:\n      repoPath: 'cvmfs/sft.cern.ch/'\n      filePath: 'cvmfs/sft.cern.ch/lcg/lastUpdate'\n    throughput:\n      num: 2 #Number of packages you want to read\n      repoPath: 'cvmfs/sft.cern.ch/'\n      filePath: 'cvmfs/sft.cern.ch/lcg/releases/'\n```\n1. mount : \n    - Test file : test_mount.py\n    - Use case : Checks if cvmfs folder is mounted or not\n    - To run this test explicity use - `python3 test_mount.py --repo sft.cern.ch --path cvmfs/sft.cern.ch/`\n2. ttfb:\n    - Test file : test_ttfb.py\n    - Use case : Evaluates the time needed to get the first byte (TTFB) of a file known to exist (lastUpdate).\n    - To run this test explicity use - `python3 test_ttfb.py ---repo sft.cern.ch --path cvmfs/sft.cern.ch/lcg/lastUpdate`\n    \n3. throughput:\n    - Test file : test_throughput.py\n    - Use case : Benchmark performance when reading from the repository and compute the read throughput.\n    - To run this test explicity use - `python3 test_throughput.py --num 2 --repo_path cvmfs/sft.cern.ch --path cvmfs/sft.cern.ch/lcg/releases/`\n\n### Jupyterhub API\n\n- To run the tests from jupyterhub container use `python3 run.py -u --session {session-name} --test jupyterhub-api --configfile test.yaml`.\ne.g - `python3 run.py -u --session user2 --test jupyterhub-api --configfile test.yaml`\n\n1. Parameters\n\nParameters for the test is present in test.yaml which is as follows:\n\n```yaml\n     jupyterhub_api:\n    check_api:\n      hostname: 'localhost'\n      port: '443'\n      token: \"\"\n      base_path: \"\"\n      verify : False\n    token:\n      hostname: 'localhost'\n      port: '443'\n      token: \"\"\n      users: ['user2']\n      base_path: \"\"\n      verify: False\n    create_session:\n      hostname: 'localhost'\n      port: '443'\n      token: 'b39639d589c44a2294b3dd1164607287'\n      users: ['user2']\n      base_path: \"\"\n      params:\n        TLS: False\n        LCG-rel: \"LCG_95a\"\n        platform: \"x86_64-centos7-gcc7-opt\"\n        scriptenv: \"none\"\n        ncores: 2\n        memory: 8589934592\n        spark-cluster: \"none\"\n      delay: 30 #Max\n      verify: False\n    check_session:\n      hostname: 'localhost'\n      port: '443'\n      token: \"\"\n      users: ['user2']\n      base_path: \"\"\n      verify: False\n    stop_session:\n      hostname: 'localhost'\n      port: '443'\n      token: ''\n      users: ['user2']\n      base_path: \"\"\n      verify: False\n```\n1. **check_api** \n    - Test file : test_check_api.py\"\n    - Use case :  Checks hub's sanity by making a GET request to \"https://localhost:443/hub/api/\".\nOn successful execution response code should be *200*.\n    - To run this test explicity use - `python3 test_check_api.py --token \" \" --port 443 --base_path \"\"`\n\n2. **token**\n    - Test file : test_token.py\"\n    - Use case :  Checks token validity by making a GET request to \"https://localhost:443/hub/api/users/user{}\".\nOn successful execution response code should be *200*.\n    - To run this test explicity use - `python3 test_token.py --port 443 --token \" \" --users user1 --base_path \"\"`\n\n2. **create_session** \n   - Test file : test_create_session.py\n   - Use case : Checks if session can be created successfully or not.\n   - parameters : \n     - params : This data needs to be passed to create a user container\n     - timedelay : In the process of creating sessions, first the required user's server is requested which is  validated by the response code *202*, a server is created consequently, which needs to repond within 30s, otherwise the server\n      will be obliterated. The wait time is maximum 30s, if the server didn't respond within stipulated time, response code *500* will be received.\u003cbr\u003e\n      Example :    \n      `TimeoutError: Server at http://172.18.0.15:8888/user/user0/ didn't respond in 30 seconds`\n    - To run the test explicitly use following command:\n       `python3 test_create_session.py --port 443 --token \" \" --users user2  --json '{\"LCG-rel\": \"LCG_95a\", \"platform\": \"x86_64-centos7-gcc7-opt\", \"scriptenv\": \"none\", \"ncores\": 2, \"memory\": 8589934592, \"spark-cluster\": \"none\"}' --delay 30 --base_path \"\"`\n\n   ---\n ### warning\n   Before creating servers/sessions make sure -\n   \n   - You generate the token by running `jupyterhub token dummy_admin`, add the token inside jupyterhub_config.py\n   file as `c.JupyterHub.service_tokens = {'dummy_admin' : '\u003ctoken_value\u003e'}` and restart the jupyterhub process with `supervisorctl restart jupyterhub`.\n   Also add the token to \"token\" field present in the yaml file.\n   \n   - And you have created users as each server is created for one particular user. \u003cbr\u003e\n   \n       - Using curl :\n       `curl -XPOST -v -k https://localhost:443/hub/api/users/user2 -H \"Authorization: token {token from yaml file}\"`\n       - Python code :\n       ```python\n           def check_create_users(self):\n            \"\"\"\n            Inside container\n            port = 443\n            Ouside container\n            port = 8443\n            \"\"\"\n            self.log.write(\"info\", \"creating users..\")\n            print(self.users)\n            for user in self.users:\n                global r\n                try:\n                    r = self.session.create_users(user)\n                except Exception as err:\n                    self.exit |= 1\n                    self.log.write(\"error\", str(err))\n                    self.log.write(\"error\", str(r))\n                else:\n                    if r.status_code == 201:\n                        self.log.write(\"info\", user + \" successfully created\")\n                    else:\n                        self.log.write(\"error\", (r.content).decode('utf-8'))\n                        self.exit |= 1\n            self.log.write(\"info\", \"Exit code \" + str(self.exit))\n            return self.exit\n     ```\n       ---\n3. **check_session**\n    - Test file : test_check_session.py\n    - Use case : Checks if a session is running or not\n    - To run this test explicitly use -\n        - For single user - `python3 test_check_session.py --port 443 --token \" \" --users user1 --base_path \"\"`\n        - For multiple users - `python3 test_check_session.py --port 443 --token \" \" --users user0 user1 user2 --base_path \"\"`\n\n4. **stop_session**\n\n     - Test file : test_stop_session.py\n     - Use case : Checks if a session  can be stopped or not\n     - To run this test explicitly use -\n        - For single user - `python3 test_stop_session.py --port 443 --token \" \" --users user1 --base_path \"\"`\n        - For multiple users - `python3 test_stop_session.py --port 443 --token \" \" --users user0 user1 user2 --base_path \"\"`\n\n### sqlite database\n\n- This test checks the consistency of the sqlite database present inside `srv/jupyterhub/` as jupyterhub.sqlite.\n- To run this test use \n`python3 run.py -u --session user2 --test database --configfile test.yaml`\nParameters:\n\n```yaml\ndatabase:\n    token:\n      path: \"jupyterhub.sqlite\"\n      user: \"user2\"\n      mode: \"active\" #active mode\n      table_name: \"api_tokens\"\n    servers:\n      path: \"jupyterhub.sqlite\"\n      user: \"user2\"\n      mode: \"active\" #active mode\n      table_name: \"servers\"\n    spawners:\n      path: \"jupyterhub.sqlite\"\n      user: \"user2\"\n      mode: \"active\" #active mode\n      table_name: \"spawners\"\n```\nTwo modes are there:\n- active mode - When the server is active\n- delete mode - When the server is removed or deleted\n\n1. **token**\n     - Test file : test_token.py\n     - Use case : Checks the status of \"token\" table when a session is created or removed.\n     - To run this test explicitly use - \n       `python3 test_token.py --path jupyterhub.sqlite -d --user user2 --table api_tokens`\n\n2. **servers**\n\n     - Test file : test_servers.py\n     - Use case : Checks the status of \"servers\" table when a session is created or removed.\n     - To run this test explicitly use - \n       `python3 test_servers.py --path jupyterhub.sqlite -d --user user2 --table servers`\n       \n3. **spawners**\n\n    - Test file : test_spawners.py\n    - Use case : Checks the status of \"spawners\" table when a session is created or removed.\n    - To run this test explicitly use - \n       `python3 test_spawners.py --path jupyterhub.sqlite -d --user user2 --table spawners`     \n\n       \n### Container\n\n- To run this test use `python3 run.py --configfile test.yaml` (this test is not meant to be run from the containers)\n\nParameters:\n\n```yaml\nuser_docker:\n    docker:\n      container_name: 'jupyter-user2'\n      timeout: 5\n```\nChecks if a container is healthy or not.  \n\n## Future Work\n   - Pending Work\n      + [Filed issues](https://github.com/Divya063/TestingFramework/issues)\n      + Implement healthchecking mechanism similar to liveliness-probes and readiness-probes\n      \n### Student\n\n***Divya Rani***\n\n- **Email ID**: [ranidivya063@gmail.com](mailto:ranidivya063@gmail.com)\n- **Linkedin Profile**: [https://linkedin.com/in/divya-rani-23924481/](https://linkedin.com/in/divya-rani-23924481/)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdivya063%2Ftestingframework","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdivya063%2Ftestingframework","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdivya063%2Ftestingframework/lists"}