{"id":18889758,"url":"https://github.com/scottlogic/quantifying-ai-productivity","last_synced_at":"2025-08-20T14:13:14.317Z","repository":{"id":216082432,"uuid":"659680532","full_name":"ScottLogic/quantifying-ai-productivity","owner":"ScottLogic","description":"An experiment in quantifying the productivity benefits of Generative AI tooling","archived":false,"fork":false,"pushed_at":"2024-01-08T09:27:30.000Z","size":4296,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-06-05T18:07:52.023Z","etag":null,"topics":["generative-ai","productivity"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ScottLogic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-28T10:31:50.000Z","updated_at":"2024-01-08T09:28:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"d3be31e7-a78c-4b6e-ac24-06f226e0d60c","html_url":"https://github.com/ScottLogic/quantifying-ai-productivity","commit_stats":null,"previous_names":["scottlogic/quantifying-ai-productivity"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ScottLogic/quantifying-ai-productivity","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScottLogic%2Fquantifying-ai-productivity","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScottLogic%2Fquantifying-ai-productivity/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScottLogic%2Fquantifying-ai-productivity/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScottLogic%2Fquantifying-ai-productivity/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ScottLogic","download_url":"https://codeload.github.com/ScottLogic/quantifying-ai-productivity/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ScottLogic%2Fquantifying-ai-productivity/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271330312,"owners_count":24740819,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-20T02:00:09.606Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generative-ai","productivity"],"created_at":"2024-11-08T07:50:48.323Z","updated_at":"2025-08-20T14:13:14.297Z","avatar_url":"https://github.com/ScottLogic.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# quantifying-ai\n\nThe aim of this experiment is to gain an understanding of the productivity gains that are possible when using generative AI technologies to aid with development and testing work.\n\n### A note on repository branches\n\nA cloned repository will default to the `main` branch. This branch contains a development solution in three languages, C#, Java and JavaScript, a suite of Cypress API tests and a suite of Postman API tests. \n\nIf you wish to develop the web server you should checkout the `start-here-dev` branch. This contains a skeleton web service implementation in three languages, C#, Java and JavaScript, a suite of Cypress API tests and a suite of Postman API tests. If you wish to do the code race version of the development task then you need the `start-here-code-race` branch.\n\nIf you wish to implement the API tests you should checkout the `start-here-test` branch. This contains a development solution in three languages, C#, Java and JavaScript, a skeleton suite of Cypress API tests and a suite of Postman API tests.\n```\ngit checkout \u003cstart-here-dev|start-here-test|start-here-code-race\u003e\n```\n\nYou should create your own branch to work on, commit your code and push it to the repository on your branch. We may wish to analyse the code that the generative AI helped produce as part of the outcomes of this experiment.\n```\ngit checkout -b \u003cnew-branch-name\u003e\n```\n\n## Developer task - Development of a simple web service\n\nThe development task is to implement a web service that provides endpoints to manage a to do list. The choice of implementation language is at your discretion. Skeleton implementations have been provided in C#, Java and JavaScript on the `start-here-dev` branch under the `src` directory. It is also valid to start completely from scratch if you wish.\n\nNo guidance is provided as to the use of generative AI tools in completing the implementation of the web server. It is valid to implement a solution using generative AI and consider the experience and productivity gains as a standalone activity. It is also valid to implement a solution without using generative AI and then revert to the skeleton code and re-implement a solution using generative AI to compare the experience and the time taken for each implementation.\n\nWe are interested in all aspects of the development experience, not just the amount of time saved by using code generated by the AI tool.\n\n### Static data\n\nThe server data will be initialised with a list containing three tasks:\n```\n[\n    {\n        \"uuid\": \"f360ba09-4682-448b-b32f-0a9e538502fa\",\n        \"name\": \"Walk the dog\",\n        \"description\": \"Walk the dog for forty five minutes\",\n        \"created\": \"2023-06-23T09:30:00Z\",\n        \"completed\": null,\n        \"complete\": false\n    },\n    {\n        \"uuid\": \"fd5ff9df-f194-4c6e-966a-71b38f95e14f\",\n        \"name\": \"Mow the lawn\",\n        \"description\": \"Mow the lawn in the back garden\",\n        \"created\": \"2023-06-23T09:00:00Z\",\n        \"completed\": null,\n        \"complete\": false\n    },\n    {\n        \"uuid\": \"5c3ec8bc-6099-4cd5-b6da-8e2956db3a34\",\n        \"name\": \"Test generative AI\",\n        \"description\": \"Use generative AI technology to write a simple web service\",\n        \"created\": \"2023-06-23T09:00:00Z\",\n        \"completed\": null,\n        \"complete\": false\n    }\n]\n```\n\nThe intial GET endpoint returns the list of tasks held by the server. If you are starting from scratch you need to implement this endpoint before continuing.\n\nThe data above is stored in the file `quantifying-ai/static_data/ToDoList.json` within the project. Feel free to copy and paste it into the implementation, or load the data from the file.\n\n### Changes required\n\n#### Change the GET endpoint.\n\nChange the existing GET endpoint to accept an optional boolean parameter named `complete`. The returned list of tasks should be filtered based on the value given for the parameter, if supplied.\n\nhttp://localhost:8080/todo{?complete=true}\n\nA GET endpoint that takes an optional boolean parameter `complete`. If the parameter is given then the endpoint returns a list of tasks that have been filtered based on the value supplied for the parameter:\n\n| URI                                       | Required behaviour                                                   |\n| ----------------------------------------- | -------------------------------------------------------------------- |\n| http://localhost:8080/todo                | Return a list of all tasks with HTTP status 200.                     |\n| http://localhost:8080/todo?complete=true  | Return a list only containing completed tasks with HTTP status 200.  |\n| http://localhost:8080/todo?complete=false | Return a list only containing incomplete tasks with HTTP status 200. |\n\n#### Add a new GET endpoint to obtain a task by uuid.\n\nAdd a new GET endpoint that uses a `uuid` as a path parameter to return a specific task from the list of tasks.\n\nhttp://localhost:8080/todo/{uuid}\n\nA GET endpoint that uses a `uuid` as a path parameter to return the task with the supplied uuid from the list of tasks. The endpoint returns the task with the given uuid if it exists, otherwise a fixed `UNKNOWN_TASK` is returned. If an invalid uuid is supplied the endpoint will return a bad request error.\n\n| URI                               | Required behaviour                                       |\n| --------------------------------- | -------------------------------------------------------- |\n| http://localhost:8080/todo/{uuid} | Return the task with supplied uuid with HTTP status 200. |\n\nGiven the static data data above:\n\nhttp://localhost:8080/todo/5c3ec8bc-6099-4cd5-b6da-8e2956db3a34 returns\n```\n{\n    \"uuid\": \"5c3ec8bc-6099-4cd5-b6da-8e2956db3a34\",\n    \"name\": \"Test generative AI\",\n    \"description\": \"Use generative AI technology to write a simple web service\",\n    \"created\": \"2023-06-23T09:00:00Z\",\n    \"completed\": null,\n    \"complete\": false\n}\n```\n\nwith HTTP status 200.\n\nhttp://localhost:8080/todo/5c3ec8bc-6099-1a2b-b6da-8e2956db3a34 returns\n```\n{\n    \"uuid\": \"00000000-0000-0000-0000-000000000000\",\n    \"name\": \"Unknown Task\",\n    \"description\": \"Unknown Task\",\n    \"created\": \"1970-01-01T00:00:00.000Z\",\n    \"completed\": null,\n    \"complete\": false\n}\n```\n\nwith HTTP status 200.\n\nhttp://localhost:8080/todo/invalid-uuid returns HTTP status 400 Bad Request. A meaningful response may also be returned, e.g.\n```\n{\n    \"timestamp\": \"2023-06-27T12:32:05.590Z\",\n    \"status\": 400,\n    \"error\": \"Bad Request\",\n    \"path\": \"/todo/invalid-uuid\"\n}\n```\n\n#### Add a new PUT endpoint to mark a task as complete.\n\nAdd a new PUT endpoint that uses a `uuid` as a path parameter to mark a specific task from the list of tasks as complete. To mark the task as complete the `completed` field should be set to the current time and the `complete` boolean should be set to true. The body of the response should indicate if the request was successful and should contain a boolean `success` flag and a string `message`.\n```\n{\n    \"success\": true,\n    \"message\": \"This task has now been completed.\"\n}\n```\n\nIf the task is already marked as completed, or the task is not found, then no change is made and the problem should be indicated in the response. If an invalid uuid is supplied the endpoint will return a bad request error.\n\n| URI                                         | Required behaviour                                                                                                                                                                                                                              |\n| ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| http://localhost:8080/todo/completed/{uuid} | Mark the task with supplied uuid as complete and return a meaningful response with HTTP status 200. To mark a task as complete the `completed` fields should be set to the current time and the `complete` boolean value should be set to true. |\n\nGiven the static data above:\n\nhttp://localhost:8080/todo/completed/5c3ec8bc-6099-4cd5-b6da-8e2956db3a34 returns\n```\n{\n    \"success\": true,\n    \"message\": \"This task has now been completed.\"\n}\n```\n\nwith HTTP status 200.\n\nA further call to http://localhost:8080/todo/completed/5c3ec8bc-6099-4cd5-b6da-8e2956db3a34 returns\n```\n{\n    \"success\": false,\n    \"message\": \"Task already marked complete.\"\n}\n```\n\nwith HTTP status 200.\n\nhttp://localhost:8080/todo/completed/5c3ec8bc-6099-1a2b-b6da-8e2956db3a34 returns\n```\n{\n    \"success\": false,\n    \"message\": \"Task not found.\"\n}\n```\n\nwith HTTP status 200.\n\nhttp://localhost:8080/todo/completed/invalid-uuid returns HTTP status 400 Bad Request. A meaningful response may also be returned, e.g.\n```\n{\n    \"timestamp\": \"2023-06-27T12:32:05.590Z\",\n    \"status\": 400,\n    \"error\": \"Bad Request\",\n    \"path\": \"/todo/completed/invalid-uuid\"\n}\n```\n\n#### Add a new POST endpoint to create a new task and add it to the list of tasks.\n\nAdd a new POST endpoint that takes two parameters, task `name` and task `description`, that creates a new task item with the given name and description. The `uuid` of the new task will be assigned by the server as a random uuid and the `created` timestamp should be set to the current time. The new item will have no value for the `completed` timestamp and a value of false for the `complete` flag. The body of the response should include the uuid of the new task and a string message.\n```\n{\n    \"taskId\": \"13f8e57c-49dc-4301-afe9-0bcf2e840056\",\n    \"message\": \"Task {task name} added successfully.\"\n}\n```\n\nThe status of the response should be 201 (CREATED) for a successful operation. If both name and description parameters are not supplied the endpoint will return a bad request error.\n\n| URI                                                                    | Required behaviour                                                                                             |\n| ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |\n| http://localhost:8080/todo/addTask{?name=TaskName\u0026description=Description} | Create a new task with the given name and description, add it to the list of tasks and return HTTP status 201. |\n\nhttp://localhost:8080/todo/addTask?name=TaskName\u0026description=Description returns:\n```\n{\n    \"taskId\": \"13f8e57c-49dc-4301-afe9-0bcf2e840056\",\n    \"message\": \"Task TaskName added successfully.\"\n}\n```\n\nwith HTTP status 201 (CREATED). Note that the uuid will be randomly generated.\n\nhttp://localhost:8080/todo/addTask?name=Name returns HTTP status 400 Bad Request. A meaningful response may also be returned, e.g.\n```\n{\n    \"timestamp\": \"2023-06-27T12:32:05.590Z\",\n    \"status\": 400,\n    \"error\": \"Bad Request\",\n    \"path\": \"/todo/addTask?name=Name\"\n}\n```\n\n### Testing the web service\n\nYou can test your web service implementation using Postman and Cypress test suites.\n\n#### Postman\n\nTo test using the Postman collection, first [download](https://www.postman.com/downloads/) and install Postman. After opening Postman, sign-in or register an account. You can then import the Postman collection by clicking `Import`. The `postmanCollection.json` file can be found under the `test\\postman` folder.\n\nThe Postman suite is intended as an aid for development and testing.\n\n#### Cypress\n\nThe Cypress test suite contains fifteen tests that should all pass when the web service implementation is complete. To use Cypress first [download](https://nodejs.org/en/download) and install nodejs. Once installed, open a new terminal, change directory into the `test` directory and run:\n```\nnpm install\nnpx cypress open\n```\n\nOnce Cypress opens, click on `E2E Testing` and select a browser (eg. Chrome). On the next screen you can run the tests.\n\nThe tests can be also run from the command line:\n```\nnpx cypress run\n```\n\n## Tester task - Development of API tests\n\nThe testing task is to implement API tests using any generative AI model, such as ChatGPT, Google Bard, etc. A skeleton framework has been provided in Cypress on the `start-here-test` branch under the `test/cypress/e2e/api-tests` directory. It is also valid to start completely from scratch if you wish. See the above for details of the four web service API endpoints.\n\nStart the web service in the language of your choice and make sure it is running on localhost:8080. Instructions about starting each of the web service implementations is in the `README` files in the respective source folders, e.g. `quantifying-ai/src/javascript/README.md`.\n\nTo use Cypress first [download](https://nodejs.org/en/download) and install nodejs. To start Cypress, open a new terminal, change directory into the `quantifying-ai/test` directory, and type:\n```\nnpm install\nnpx cypress open\n```\n\nThis will open a Cypress browser. Select 'E2E testing', choose a browser (e.g. Chrome) and 'Start E2E Testing...'\n\nThe tests found in `test/cypress/e2e` will appear. To run, click on your test suite, e.g. `todoAppAPI.cy.js`\n\nYou can add tests to this file, or create a new test suite.\n\nYou can also run tests headlessly:\n```\nnpx cypress run\n```\n\n### More information\n\nFor more detailed information check the [Quantifying Generative AI](https://scottlogic.atlassian.net/l/cp/C5VDmLC5) pages on Confluence.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscottlogic%2Fquantifying-ai-productivity","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscottlogic%2Fquantifying-ai-productivity","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscottlogic%2Fquantifying-ai-productivity/lists"}