{"id":25922230,"url":"https://github.com/jonathanalgar/customcode-analyzer-generator","last_synced_at":"2026-03-10T14:04:49.218Z","repository":{"id":279323355,"uuid":"938443693","full_name":"jonathanalgar/CustomCode-Analyzer-Generator","owner":"jonathanalgar","description":"A command-line tool to generate validated OutSytems Developer Cloud (ODC) External Library projects complete with unit tests.","archived":false,"fork":false,"pushed_at":"2025-03-06T07:18:37.000Z","size":13261,"stargazers_count":3,"open_issues_count":11,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-06T07:45:09.417Z","etag":null,"topics":["csharp","odc","outsystems","roslyn","static-analysis"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonathanalgar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-25T00:57:37.000Z","updated_at":"2025-03-03T17:18:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"df8e3768-47fa-4ff7-be5f-7ca0afc4977f","html_url":"https://github.com/jonathanalgar/CustomCode-Analyzer-Generator","commit_stats":null,"previous_names":["jonathanalgar/customcode-analyzer-generator"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/jonathanalgar/CustomCode-Analyzer-Generator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanalgar%2FCustomCode-Analyzer-Generator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanalgar%2FCustomCode-Analyzer-Generator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanalgar%2FCustomCode-Analyzer-Generator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanalgar%2FCustomCode-Analyzer-Generator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonathanalgar","download_url":"https://codeload.github.com/jonathanalgar/CustomCode-Analyzer-Generator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonathanalgar%2FCustomCode-Analyzer-Generator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30336112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T12:41:07.687Z","status":"ssl_error","status_checked_at":"2026-03-10T12:41:06.728Z","response_time":106,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csharp","odc","outsystems","roslyn","static-analysis"],"created_at":"2025-03-03T16:17:57.098Z","updated_at":"2026-03-10T14:04:49.189Z","avatar_url":"https://github.com/jonathanalgar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OutSystems Developer Cloud (ODC) Custom Code Analyzer: Generator\n\n[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer-Generator\u0026metric=ncloc)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer-Generator) [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](https://makeapullrequest.com)\n\n[![Reliability Rating](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer-Generator\u0026metric=reliability_rating)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer-Generator) [![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer-Generator\u0026metric=sqale_rating)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer-Generator)\n [![Vulnerabilities](https://sonarcloud.io/api/project_badges/measure?project=jonathanalgar_CustomCode-Analyzer-Generator\u0026metric=vulnerabilities)](https://sonarcloud.io/summary/new_code?id=jonathanalgar_CustomCode-Analyzer-Generator)\n\n\u003e :information_source: This component is unofficial and unsupported by OutSystems.\n\n![](./README_resources/header.png)\n\n⚡ **`CustomCode-Analyzer-Generator` is a command-line tool to generate validated OutSytems Developer Cloud (ODC) External Library projects complete with unit tests.**. See examples of generated solutions [here](./README_resources/example_generations/RedisConnector/) (_**\u003e** Connect to a redis database set up on my secure gateway (port 6379). password and key will be input params. return the key value_) and [here](./README_resources/example_generations/PythonRunner/) (_**\u003e** Take a python program in a string as input and return the stdout in the output_). ⚡\n\n## Overview\n\nThe BSD-3 licensed [`CustomCode-Analyzer`](https://github.com/jonathanalgar/CustomCode-Analyzer) component was released to give you real-time feedback your C# code as you build an external library in your IDE of choice. But more generally `CustomCode-Analyzer` can be thought of as a way of locally validating an external library project at the point of build. This makes it a powerful component in an end-to-end Large Language Model (LLM) pipeline for generating an ODC external library:\n\n![](./README_resources/diagram.png)\n\nThe `CustomCode-Analyzer-Generator` is an orchestrator  around the `CustomCode-Analyzer` and other local components and external APIs. As seen in the above diagram the `CustomCode-Analyzer-Generator` takes user input for an external library functionality and:\n\n* Calls the NuGet API and LLMs to generate the candidate implementation and unit test code.\n* Creates implementation and unit test C# projects, installs packages and incorporates candidate code.\n* Uses `CustomCode-Analyzer` as part of build to validate implementation code.\n* Uses generated unit tests to further validate implementation code.\n* Manages a reflection mechanism for self-correction and improvement in case of build failure or unit test failure.\n\nIn practice, that looks something like this (no speedup applied on a slow laptop!):\n\n![](./README_resources/recording.gif)\n\n## How to use\n\n\u003e :information_source: For safety, `CustomCode-Analyzer-Generator` runs inside a sandboxed Docker container. Therefore, you must first install [Rancher Desktop](https://rancherdesktop.io/) (recommended) or [Docker Desktop](https://www.docker.com/products/docker-desktop/) to run it.\n\n### Windows\n\n1. Open Windows PowerShell.\n1. Run the [quick start script](./windows_setup.ps1):\n\n        irm https://raw.githubusercontent.com/jonathanalgar/CustomCode-Analyzer-Generator/refs/heads/main/windows_setup.ps1 | iex\n\n1. Follow the instructions in the script.\n\n### Mac\n_(Quick start script for Mac coming soon)._\n\n1. Open the [`Terminal` app](https://support.apple.com/en-sg/guide/terminal/apd5265185d-f365-44cb-8b09-71a064a42125/mac).\n1. Create the required directories in the home directory or elsewhere:\n\n        mkdir -p ~/CustomCode-Analyzer-Generator/generated-solutions \u0026\u0026 chmod 700 ~/CustomCode-Analyzer-Generator/generated-solutions \u0026\u0026 cd ~/CustomCode-Analyzer-Generator\n\n1. Create a `.env` file in `~/CustomCode-Analyzer-Generator` and edit with your text editor of choice. Use [`.env_example`](./.env_example) as template.\n1. Run the command:\n\n        docker run --pull always -it --rm --env-file .env -v \"$(pwd)/generated_solutions:/app/output\" ghcr.io/jonathanalgar/customcode-analyzer-generator:latest\n\n1. The generated solution libraries will be created in `~/CustomCode-Analyzer-Generator/generated_solutions`.\n\n##  Technical notes on evaluation\n\nTo validate that the LLM pipeline works effectively and generates external libraries aligned with user intent there is a supplementary evaluation pipeline complete with [eight diverse test cases](./agents/evaluation/ground_truth/). The objective is [to add more](https://github.com/jonathanalgar/CustomCode-Analyzer-Generator/issues/6).\n\nThis means a range of models and prompts can be programatically benchmarked. The central piece of pipeline is the Roslyn-based [`CCGATestGenerator`](./agents/evaluation/CCAGTestGenerator/), which—given the LLM-generated implementation code and human created ground truth file—will deterministically create a unit test.\n\nTest cases covered include [binary data](./agents/evaluation/ground_truth/pdf.yml) and [network access through the secure gateway](./agents/evaluation/ground_truth/redis.yml). The test set is currently limited to single action (but with support for multi-parameter using an LLM to do name mapping) and single type output.\n\nSummary of consolidated no-repetition results for 2x OpenAI models with one-shot prompt (current default prompt):\n\n![](README_resources/runs.png)\n\nSee full runs [here](https://htmlpreview.github.io/?https://github.com/jonathanalgar/CustomCode-Analyzer-Generator/blob/main/benchmark_results/runs.html) or use [this notebook](./agents/evaluation/benchmark_analysis.ipynb).\n\nYou can replicate a run using:\n\n```\ndocker compose -f run_full_benchmark.yml up --abort-on-container-exit\n```\n\n\u003e :bulb: API access to o3-mini is [still in preview](https://help.openai.com/en/articles/10362446-api-access-to-o1-and-o3-mini).\n\n_(More documentation and analysis coming soon.)_\n\n## TODO\n\nSee [here](https://github.com/jonathanalgar/CustomCode-Analyzer/issues?q=is%3Aopen+is%3Aissue+label%3Aenhancement).\n\n## Feedback and contributions\n\nPlease report bugs and feature requests [here](https://github.com/jonathanalgar/CustomCode-Analyzer/issues/new/choose).\n\nPRs are welcome. In particular, code quality improvements, new unit/integration tests, new ground truths, and documentation improvements are all welcome 🤗 All changes to code should pass all existing tests (which are lacking and need to be expanded!). Please format any new code with [Mypy and Flake8 for Python](https://github.com/jonathanalgar/CustomCode-Analyzer-Generator/blob/d3cd06bd8f3fb962a1bb5459a15f098f9fc4b0c1/Makefile#L10-L12) and [Csharpier for C#](https://github.com/jonathanalgar/CustomCode-Analyzer-Generator/blob/d3cd06bd8f3fb962a1bb5459a15f098f9fc4b0c1/Makefile#L14-L15).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonathanalgar%2Fcustomcode-analyzer-generator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonathanalgar%2Fcustomcode-analyzer-generator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonathanalgar%2Fcustomcode-analyzer-generator/lists"}