{"id":21473407,"url":"https://github.com/cedarscript/cedarscript-integration-aider","last_synced_at":"2025-07-15T08:32:19.217Z","repository":{"id":258874328,"uuid":"875832090","full_name":"CEDARScript/cedarscript-integration-aider","owner":"CEDARScript","description":"Allows Aider to use CEDARScript as an edit format","archived":false,"fork":false,"pushed_at":"2024-11-18T03:50:16.000Z","size":273,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-18T04:38:15.151Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CEDARScript.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-20T23:42:19.000Z","updated_at":"2024-11-18T03:50:20.000Z","dependencies_parsed_at":"2024-11-15T16:16:48.737Z","dependency_job_id":null,"html_url":"https://github.com/CEDARScript/cedarscript-integration-aider","commit_stats":null,"previous_names":["cedarscript/cedarscript-integration-aider"],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CEDARScript%2Fcedarscript-integration-aider","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CEDARScript%2Fcedarscript-integration-aider/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CEDARScript%2Fcedarscript-integration-aider/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CEDARScript%2Fcedarscript-integration-aider/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CEDARScript","download_url":"https://codeload.github.com/CEDARScript/cedarscript-integration-aider/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226027819,"owners_count":17562139,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T10:17:04.582Z","updated_at":"2025-07-15T08:32:19.205Z","avatar_url":"https://github.com/CEDARScript.png","language":"Python","readme":"# CEDARScript Integration: Aider\n\n[![PyPI version](https://badge.fury.io/py/cedarscript-integration-aider.svg)](https://pypi.org/project/cedarscript-integration-aider/)\n[![Python Versions](https://img.shields.io/pypi/pyversions/cedarscript-integration-aider.svg)](https://pypi.org/project/cedarscript-integration-aider/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)\n\n`CEDARScript Integration: Aider` enables [`Aider`](https://aider.chat/) to use \n[**CEDARScript**](https://github.com/CEDARScript/cedarscript-grammar#readme)\nas an [_edit format_](https://aider.chat/docs/benchmarks.html#edit-formats).\n\n## Table of Contents\n- [What is CEDARScript?](#what-is-cedarscript)\n- [Installation](#installation)\n- [Running a Benchmark](#running-a-benchmark)\n- [Why Use CEDARScript?](#why-use-cedarscript)\n- [Performance Comparison](#performance-comparison)\n   - [Notable Achievements](#notable-achievements)\n   - [The Gemini 1.5 Flash benchmark highlights](#the-gemini-15-flash-benchmark-highlights)\n- [Individual Test Analysis](#individual-test-analysis)\n- [Detailed Analysis](#detailed-analysis)\n- [Contributing](#contributing)\n- [License](#license)\n\n## What is CEDARScript?\n\n[CEDARScript](https://bit.ly/cedarscript) (_Concise Examination, Development, And Refactoring Script_)\nis a domain-specific language designed to improve how AI coding assistants interact with codebases and communicate their code modification intentions.\nIt provides a standardized way to express complex code modification and analysis operations, making it easier for\nAI-assisted development tools to understand and execute these tasks.\n\n## Installation\n\n1. Install **Aider with _CEDARScript and CedarTL support_** via this command below:\n```shell\npython -m ensurepip --upgrade\npip install --upgrade --force-reinstall \\\ngit+https://github.com/elifarley/aider@cedarscript \\\naider-chat\n```\n2. Now, simply use the [`--edit-format` switch](https://aider.chat/docs/more/edit-formats.html) and select `cedarscript`:\n```shell\naider --edit-format cedarscript\n```\n\n## Running a Benchmark\n\n### One-Time Actions\nFirst, [install Aider with CEDARScript](#installation);\n\nThen, follow the [**benchmark setup instructions**](https://github.com/Aider-AI/aider/blob/main/benchmark/README.md#setup-for-benchmarking) once.\n\nThen, install the [`refactor-benchmark`](https://github.com/Aider-AI/refactor-benchmark/tree/main#benchmark-details),\nwhich will perform refactorings on a _non-trivial_ amount of code found in fairly **large** files:\n```shell\n( cd tmp.benchmarks \u0026\u0026 git clone https://github.com/Aider-AI/refactor-benchmark.git )\n```\n\n## For Every Benchmark Run\nFinally, for every new benchmark you want to run:\n```shell\n# Launch the docker container\nbenchmark/docker.sh\n\n# Inside the container, install aider as a development build.\n# This way you're running the code that you cloned above, including any local changes.\npip install -e .\n\n### \nbenchmark/benchmark.py gemini-flash-cedarscript-version-refactor \\\n--model gemini/gemini-1.5-flash-latest \\\n--edit-format cedarscript \\\n--exercises-dir refactor-benchmark \\\n--threads 1 #### Must be only 1 ####\n```\n\n## Why use CEDARScript?\n\n`TL;DR`: You can get higher success rates when refactoring large files, comparing to other edit formats.\n\n1. **Higher Success Rates**: Significantly improves the performance of AI models in code refactoring tasks.\n2. **Cost-Effective Performance**: Enables more affordable models to compete with top-tier options.\n3. **Standardized Communication**: Provides a consistent format for AI-code interaction in coding tasks.\n4. **Enhanced Accuracy**: Reduces errors and improves the quality of AI-generated code modifications.\n\n## Performance Comparison\n\nCEDARScript has shown remarkable improvements in AI model performance for code refactoring:\n\n| Model             | Format      | Pass Rate | Well-Formed Cases | Syntax Errors | Indentation Errors | Cost | Avg. Time per case |\n|-------------------|-------------|-----------|-------------------|---------------|--------------------|------|--------------------|\n| Gemini 1.5 PRO    | CEDARScript | 77.5%     | 86.5%             | 4             | 3                  | 26.2 | 29                 |\n| Gemini 1.5 Flash  | CEDARScript | 76.4%     | 94.4%             | 3             | 5                  | 0.68 | 14.7               |\n| Claude 3.5 Sonnet | diff        | 64.0%     | 76.4%             | n/a           | n/a                | n/a  | n/a                |\n| Gemini 1.5 PRO    | diff-fenced | 49.4%     | 7.9%              | 21            | 93                 | 28.3 | 110.1              |\n\n### Notable Achievements:\n- **Gemini 1.5 _PRO_** with **CEDARScript** outperformed both its diff-fenced format and **Claude 3.5 Sonnet**.\n- Most remarkably, the more cost-effective **Gemini 1.5 _Flash_** model, using **CEDARScript**, outperformed **Claude 3.5 Sonnet**.\n  - It goes to show that even a more affordable model can surpass top-tier competitors when equipped with the _right_ tools.\n\nThis suggests that **CEDARScript** can level the playing field, enabling more accessible AI models\nto compete with and even _exceed_ the capabilities of more expensive options in certain coding tasks.\n\n### The Gemini 1.5 Flash benchmark highlights\n\n- 48% of tests (43 total) showed improvements\n- 103% increase in Pass 1 success rate (75 tests)\n- Test duration reduced by 93% (from 5:17:26 to 0:25:17)\n- Token efficiency greatly improved:\n- Sent tokens: -37% (7.59M)\n- Received tokens: -96% (180K)\n- Error reduction:\n- Error outputs: -94% (35 total)\n- Malformed outputs: -94% (6 cases)\n- Syntax errors: -85% (3 cases)\n- Indent errors eliminated (100% reduction)\n\n\u003cdetails\u003e\n\u003csummary\u003eDelta...\u003c/summary\u003e\n  \n![image](https://github.com/user-attachments/assets/86683a1b-2b64-49c9-89ff-eb18d3511ae7)\n\u003c/details\u003e\n\n### Individual Test Analysis\n\n\u003cdetails\u003e\n\u003csummary\u003eIndividual Test Diff\u003c/summary\u003e\n\n```diff\n--- 2024-10-22-05-21-41--gemini-1.5-flash-refactoring-whole\n+++ 2024-10-22-05-13-37--gemini-1.5-flash-refactoring-cedarscript-i0.0.9-e0.3.3\n# ============= Failed Attempts per Test =============\n# N \u003e= 0: it eventually passed after N failed attempts\n# N \u003c 0 : All attempts failed and limit was reached\n\n@@ Improved, now PASSED (36) @@\n++analyzer_cli_DebugAnalyzer__make_source_table: -4 -\u003e 0\n++autodetector_MigrationAutodetector__trim_to_apps: -4 -\u003e 0\n++backends_ModelBackend_with_perm: -4 -\u003e 0\n++builtin_BuiltinVariable_call_setattr: -4 -\u003e 0\n++checks_BaseModelAdminChecks__check_ordering_item: -4 -\u003e 0\n++checks_BaseModelAdminChecks__check_raw_id_fields_item: -4 -\u003e 0\n++checks_ModelAdminChecks__check_action_permission_methods: -4 -\u003e 0\n++checks_ModelAdminChecks__check_inlines_item: -4 -\u003e 0\n++checks_ModelAdminChecks__check_list_display_item: -4 -\u003e 0\n++clustering_ops_KMeans__mini_batch_training_op: -4 -\u003e 0\n++codeeditor_CodeEditor___get_brackets: -4 -\u003e 0\n++config_AppConfig__path_from_module: -4 -\u003e 0\n++config_ConfigCLI__get_settings_vars: -4 -\u003e 0\n++coordinator_HERETransitDataUpdateCoordinator__parse_transit_response: -4 -\u003e 0\n++cuda_cpp_scheduling_CUDACPPScheduling__can_fuse_epilogue_impl: -4 -\u003e 0\n++dataframeeditor_DataFrameView_next_index_name: -4 -\u003e 0\n++diffsettings_Command_output_hash: -4 -\u003e 0\n++dim2_Dim2CompatTests_test_reductions_2d_axis0: -4 -\u003e 3\n++distribution_DistributionFiles_parse_distribution_file_SUSE: -4 -\u003e 0\n++doc_DocCLI_get_role_man_text: -4 -\u003e 0\n++figure_FigureBase_colorbar: -4 -\u003e 0\n++functional_Functional__conform_to_reference_input: -4 -\u003e 0\n++galaxy_GalaxyCLI_execute_list_collection: -4 -\u003e 0\n++kernel_SpyderKernel_get_fault_text: -4 -\u003e 1\n++main_widget_PylintWidget_parse_output: -4 -\u003e 0\n++methods_BaseMethodsTests_test_where_series: -4 -\u003e 0\n++ogrinspect_Command_add_arguments: -4 -\u003e 0\n++onnxfunction_dispatcher_OnnxFunctionDispatcher__get_aten_name: -4 -\u003e 1\n++operations_DatabaseOperations_last_executed_query: -4 -\u003e 0\n++polar_RadialTick__determine_anchor: -4 -\u003e 0\n++profile_analyzer_cli_ProfileAnalyzer__get_list_profile_lines: -4 -\u003e 0\n++split_cat_SplitCatSimplifier_replace_cat: -4 -\u003e 0\n++split_cat_SplitCatSimplifier_replace_split: -4 -\u003e 0\n++text_CountVectorizer__limit_features: -4 -\u003e 0\n++triton_TritonScheduling_define_kernel: -4 -\u003e 0\n++triton_TritonScheduling_generate_node_schedule: -4 -\u003e 0\n\n@@ Improved, minor (3) @@\n+ checks_BaseModelAdminChecks__check_autocomplete_fields_item: 3 -\u003e 0\n+ dataloader_DataLoader__is_role: 2 -\u003e 0\n+ operations_OracleOperations_convert_extent: 1 -\u003e 0\n\n@@ Worsened, now FAILED (7) @@\n--base_BaseHandler_adapt_method_mode: 0 -\u003e -4\n--feedgenerator_Atom1Feed_add_item_elements: 0 -\u003e -4\n--generic_bsd_GenericBsdIfconfigNetwork_parse_inet_line: 0 -\u003e -4\n--graph_drawer_FxGraphDrawer__stringify_tensor_meta: 0 -\u003e -4\n--group_batch_fusion_GroupLinearFusion_fuse: 0 -\u003e -4\n--inspectdb_Command_normalize_col_name: 0 -\u003e -4\n--introspection_DatabaseIntrospection__parse_column_or_constraint_definition: 0 -\u003e -4\n\n@@ Stable: PASSED (30) @@\n=+autosave_AutosaveForPlugin_get_files_to_recover: 0\n=+base_BaseHandler_check_response: 0\n=+baseconv_BaseConverter_convert: 0\n=+compile_utils_MetricsContainer__get_metric_object: 0\n=+concat__Concatenator__clean_keys_and_objs: 0\n=+config_NetworkConfig_parse: 0\n=+csrf_CsrfViewMiddleware__set_csrf_cookie: 0\n=+dumpdata_Command_add_arguments: 0\n=+finders_FileSystemFinder_check: 0\n=+gateway_Gateway_get_and_delete_all_sms: 0\n=+getitem_BaseGetitemTests_test_get: 0\n=+grad_scaler_GradScaler__unscale_grads_: 0\n=+gradient_checker_GradientChecker__assertInferTensorChecks: 0\n=+graph_MigrationGraph_iterative_dfs: 0\n=+grpc_debug_server_EventListenerBaseServicer__process_tensor_event_in_chunks: 0\n=+i18n_JavaScriptCatalog_get_paths: 0\n=+inspectdb_Command_get_field_type: 0\n=+inspectdb_Command_get_meta: 0\n=+introspection_DatabaseIntrospection__get_column_collations: 0\n=+load_v1_in_v2__EagerSavedModelLoader__extract_signatures: 0\n=+makemessages_Command_add_arguments: 0\n=+makemigrations_Command_add_arguments: 0\n=+migrate_Command_add_arguments: 0\n=+operations_DatabaseOperations_bulk_insert_sql: 0\n=+operations_DatabaseOperations_check_expression_support: 0\n=+reshaping_BaseReshapingTests_test_concat_mixed_dtypes: 0\n=+schema_DatabaseSchemaEditor_quote_value: 0\n=+shell_Command_python: 0\n=+special_RunSQL__run_sql: 0\n=+weather_NWSWeather__forecast: 0\n\n@@ Stable: FAILED (13) @@\n=-autodetector_MigrationAutodetector_check_dependency: -4 -\u003e -4\n=-checks_ModelAdminChecks__check_list_editable_item: -4 -\u003e -4\n=-common_methods_invocations_foreach_inputs_sample_func__sample_rightmost_arg: -4 -\u003e -4\n=-common_utils_TestCase_genSparseTensor: -4 -\u003e -4\n=-doc_DocCLI_display_plugin_list: -4 -\u003e -4\n=-generator_GenOpTestCase_out_variant_op_test_case_generator: -4 -\u003e -4\n=-options_ModelAdmin_message_user: -4 -\u003e -4\n=-patches__Curve__get_arrow_wedge: -4 -\u003e -4\n=-quiver_Barbs__make_barbs: -4 -\u003e -4\n=-reshaping_BaseReshapingTests_test_unstack: -4 -\u003e -4\n=-sharding_policies_MaxShardSizePolicy__add_partition: -4 -\u003e -4\n=-split_cat_SplitCatSimplifier_get_transform_params: -4 -\u003e -4\n=-symbolic_shapes_ShapeEnv_bind_symbols: -4 -\u003e -4\n\n--- 2024-10-22-05-21-41--gemini-1.5-flash-refactoring-whole                                                                                                                    \n+++ 2024-10-26-23-32-08--gemini-1.5-pro-refactoring-cedarscript-i0.0.18-e0.3.13                                                                                                \n@@ ============= TEST STATUS CHANGES ============ @@                                                                                                                           \n                                                                                                                                                                               \n\u003c REMOVED      :  71 ( 80% of total)                                                                                                                                           \n\u003c+      PASSED :  35 ( 39% of total)                                                                                                                                           \n\u003c-      FAILED :  36 ( 40% of total)                                                                                                                                           \n                                                                                                                                                                               \n+ IMPROVED     :  11 ( 12% of total)                                                                                                                                           \n++  Now PASSES :  10 ( 11% of total)                                                                                                                                           \n+        Minor :   1 (  1% of total)                                                                                                                                           \n                                                                                                                                                                               \n- WORSENED     :   3 (  3% of total)                                                                                                                                           \n--  Now FAILED :   3 (  3% of total)                                                                                                                                           \n-        Minor :   0 (  0% of total)                                                                                                                                           \n                                                                                                                                                                               \n= STABLE       :   4 (  4% of total)                                                                                                                                           \n=+      PASSED :   1 (  1% of total)                                                                                                                                           \n#-      FAILED :   3 (  3% of total)                                                                                                                                           \n                                                                                                                                                                               \n@@ ============= PERFORMANCE METRICS ============ @@                                                                                                                           \n# TOTAL TEST COUNT :         18 (-71)                                                                                                                                          \n# DURATION hh:mm:ss:    0:10:29 (-  5:32:14,  -97%) ---------                                                                                                                  \n# COST ($)         :       9.40 (     +7.02, +294%) ++++++++++++++++++++                                                                                                       \n# TOKENS SENT      :  2,683,000 (-9,312,700,  -78%) -------                                                                                                                    \n# TOKENS RECEIVED  :     12,382 (-4,970,531, -100%) ---------                                                                                                                  \n# TIMEOUTS         :          0 N/A                                                                                                                                            \n# ERROR OUTPUTS    :         62 N/A                                                                                                                                            \n# USER ASKS        :         37 (      -521,  -93%) ---------                                                                                                                  \n# CONTEXT EXHAUSTS :          0 N/A                                                                                                                                            \n# MALFORMED        :          4 N/A                                                                                                                                            \n# SYNTAX ERRORS    :          1 (      -105,  -99%) ---------                                                                                                                  \n# INDENT ERRORS    :          9 (       -11,  -55%) -----                                                                                                                      \n# LAZY COMMENTS    :          0 (       -28, -100%) ---------- \n```\n\u003c/details\u003e\n\n#### Detailed Analysis\n\nThis overview suggests that the CEDARScript edit format has had a significant positive impact on the task of method extraction,\nwith improvements in nearly half of the tests and only a small percentage of tests worsening.\n\n**Improvements:**\n\n36 tests that previously failed now pass. This is a substantial improvement, indicating that CEDARScript is more \neffective in correctly extracting methods from classes across a wide range of codebases.\n\n**Notable improvements include:**\n- `analyzer_cli_DebugAnalyzer__make_source_table`: Suggests better handling of debug-related code refactoring.\n- `autodetector_MigrationAutodetector__trim_to_apps`: Indicates improved capability in refactoring Django migration-related code.\n- `cuda_cpp_scheduling_CUDACPPScheduling__can_fuse_epilogue_impl`: Shows better performance in handling complex CUDA-related code.\n- `triton_TritonScheduling_define_kernel` and triton_TritonScheduling_generate_node_schedule: Demonstrates improved capability in refactoring GPU computing-related code.\n\n**Minor Improvements:**\n\n3 tests showed minor improvements, such as `checks_BaseModelAdminChecks__check_autocomplete_fields_item`, suggesting \nslight enhancements in handling Django admin-related code.\n\n**Regressions:**\n\n7 tests that previously passed now fail. While concerning, it's a relatively small number compared to the improvements.\n\n**Notable regressions include:**\n- `feedgenerator_Atom1Feed_add_item_elements`: Suggests potential issues with refactoring feed generation code.\n- `generic_bsd_GenericBsdIfconfigNetwork_parse_inet_line`: Indicates challenges in refactoring network-related parsing code.\n- `introspection_DatabaseIntrospection__parse_column_or_constraint_definition`: Shows difficulties in handling database schema introspection code.\n\n**Stability:**\n\n30 tests remained stable and passing, indicating that CEDARScript maintained performance in many areas, including various \nDjango commands, database operations, and utility functions.\n13 tests remained stable but failing, suggesting that some challenging areas were not addressed by either format.\nThese include complex operations like `symbolic_shapes_ShapeEnv_bind_symbols` and `reshaping_BaseReshapingTests_test_unstack`.\n\n**Analysis by Domain:**\n\n- **Web Frameworks (e.g., Django)**: Generally improved, with better handling of model admin checks, configuration, and \ndatabase operations.\n- **Data Science and ML**: Mixed results. Improvements in areas like clustering operations (`clustering_ops_KMeans__mini_batch_training_op`)\nand data frame handling, but persistent issues in some reshaping operations.\n- **System-level Operations**: Some improvements (e.g., `distribution_DistributionFiles_parse_distribution_file_SUSE`)\nbut also regressions (e.g., `generic_bsd_GenericBsdIfconfigNetwork_parse_inet_line`).\n- **GPU and High-Performance Computing**: Significant improvements, especially in `CUDA` and `Triton`-related code.\n\n**Interpretation:**\n\nCEDARScript appears more effective in handling complex code structures, especially in areas related to web frameworks, \ndata processing, and high-performance computing.\nIt shows improved capability in understanding class contexts and correctly extracting methods across various domains.\nHowever, it may introduce new challenges in certain specific areas, possibly due to its different approach to code manipulation.\n\n**Areas for Further Investigation:**\n\nUnderstanding why certain tests regressed (e.g., `feedgenerator_Atom1Feed_add_item_elements`) could provide insights for improvement.\n\nAnalyzing the stable failing tests (e.g., `symbolic_shapes_ShapeEnv_bind_symbols`) to see if CEDARScript can be enhanced \nto address these persistent issues.\n\n**Conclusion:**\n- The introduction of CEDARScript appears to be a significant improvement for the task of extracting methods from classes.\n- It shows particular strength in handling complex codebases, especially those related to web frameworks, data processing, and high-performance computing.\n- However, care should be taken to address the areas where regressions occurred, particularly in system-level operations and certain parsing tasks.\n- The consistent performance across various domains suggests that CEDARScript offers a more robust and versatile approach to code refactoring.\n\nThis analysis indicates that CEDARScript is a promising enhancement to Aider, offering more accurate and comprehensive \nmethod extraction capabilities across a wide range of codebases.\nHowever, it also highlights the need for continued refinement, especially in areas where regressions were observed.\n\u003c/details\u003e\n\n### Benchmark Metrics\n\n\u003cdetails\u003e\n\u003csummary\u003eSonnet 3.5 + diff\u003c/summary\u003e\n\n```yaml\n- dirname: refac-claude-3.5-sonnet-diff-not-lazy\n  model: claude-3.5-sonnet (diff)\n  edit_format: diff\n  pass_rate_1: 64.0\n  percent_cases_well_formed: 76.4\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eGemini 1.5 PRO + diff-fenced (leaderboard site)\u003c/summary\u003e\n\n```yaml\n- dirname: refac-gemini\n  model: gemini/gemini-1.5-pro-latest\n  edit_format: diff-fenced\n  pass_rate_1: 49.4\n  percent_cases_well_formed: 7.9\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eGemini 1.5 PRO + diff-fenced (own tests)\u003c/summary\u003e\n\n```yaml\n- dirname: 2024-10-05-00-43-21--diff-fenced-Gemini-Refactoring\n  test_cases: 89\n  model: gemini/gemini-1.5-pro-latest\n  edit_format: diff-fenced\n  commit_hash: 772710b-dirty\n  pass_rate_1: 18.0\n  pass_rate_2: 21.3\n  pass_rate_3: 24.7\n  percent_cases_well_formed: 34.8\n  error_outputs: 180\n  num_malformed_responses: 180\n  num_with_malformed_responses: 58\n  user_asks: 128\n  lazy_comments: 2\n  syntax_errors: 21\n  indentation_errors: 93\n  exhausted_context_windows: 0\n  test_timeouts: 0\n  command: aider --model gemini/gemini-1.5-pro-latest\n  date: 2024-10-05\n  versions: 0.57.2.dev\n  seconds_per_case: 110.1\n  total_cost: 28.2515\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eGemini 1.5 PRO + CEDARScript\u003c/summary\u003e\n\n```yaml\n- dirname: 2024-10-19-22-48-07--cedarscript-0.3.1-refactoring-gemini1.5pro\n  test_cases: 89\n  model: gemini/gemini-1.5-pro-latest\n  edit_format: cedarscript-g\n  commit_hash: 4da1e9b-dirty\n  pass_rate_1: 77.5\n  percent_cases_well_formed: 86.5\n  error_outputs: 337\n  num_malformed_responses: 19\n  num_with_malformed_responses: 12\n  user_asks: 12\n  lazy_comments: 0\n  syntax_errors: 4\n  indentation_errors: 3\n  exhausted_context_windows: 0\n  test_timeouts: 0\n  command: aider --model gemini/gemini-1.5-pro-latest\n  date: 2024-10-19\n  versions: 0.59.2.dev\n  seconds_per_case: 29.0\n  total_cost: 26.2374\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eGemini 1.5 Flash + whole\u003c/summary\u003e\n\n```yaml\n- dirname: 2024-10-22-05-21-41--gemini-1.5-flash-refactoring-whole\n  test_cases: 89\n  model: gemini/gemini-1.5-flash-002\n  edit_format: whole\n  commit_hash: feb1c38\n  pass_rate_1: 41.6\n  pass_rate_2: 42.7\n  pass_rate_3: 43.8\n  pass_rate_4: 44.9\n  percent_cases_well_formed: 100.0\n  error_outputs: 0\n  num_malformed_responses: 0\n  num_with_malformed_responses: 0\n  user_asks: 558\n  lazy_comments: 28\n  syntax_errors: 106\n  indentation_errors: 20\n  exhausted_context_windows: 0\n  test_timeouts: 0\n  command: aider --model gemini/gemini-1.5-flash-002\n  date: 2024-10-22\n  versions: 0.59.2.dev\n  seconds_per_case: 231.1\n  total_cost: 2.3894\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eGemini 1.5 Flash + CEDARScript\u003c/summary\u003e\n\n```yaml\n- dirname: 2024-10-20-00-33-27--cedarscript-0.3.1-refactoring-gemini-1.5-flash\n  test_cases: 89\n  model: gemini/gemini-1.5-flash-latest\n  edit_format: cedarscript-g\n  commit_hash: 4da1e9b-dirty\n  pass_rate_1: 76.4\n  percent_cases_well_formed: 94.4\n  error_outputs: 403\n  num_malformed_responses: 13\n  num_with_malformed_responses: 5\n  user_asks: 21\n  lazy_comments: 0\n  syntax_errors: 3\n  indentation_errors: 5\n  exhausted_context_windows: 0\n  test_timeouts: 0\n  command: aider --model gemini/gemini-1.5-flash-latest\n  date: 2024-10-20\n  versions: 0.59.2.dev\n  seconds_per_case: 14.7\n  total_cost: 0.6757\n```\n\u003c/details\u003e\n\n#### functional_Functional__conform_to_reference_input\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ediff-fenced\u003c/summary\u003e\n\n```yaml\n    \"cost\": 0.33188854999999995,\n    \"duration\": 27.793912172317505,\n    \"test_timeouts\": 0,\n    \"commit_hash\": \"772710b-dirty\",\n    \"num_error_outputs\": 2,\n    \"num_user_asks\": 3,\n    \"num_exhausted_context_windows\": 0,\n    \"num_malformed_responses\": 2,\n    \"syntax_errors\": 0,\n    \"indentation_errors\": 3,\n    \"lazy_comments\": 0,\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003ecedarscript\u003c/summary\u003e\n\n```yaml\n    \"cost\": 0.18178265,\n    \"duration\": 11.176445960998535,\n    \"test_timeouts\": 0,\n    \"commit_hash\": \"772710b-dirty\",\n    \"num_error_outputs\": 0,\n    \"num_user_asks\": 1,\n    \"num_exhausted_context_windows\": 0,\n    \"num_malformed_responses\": 0,\n    \"syntax_errors\": 0,\n    \"indentation_errors\": 0,\n    \"lazy_comments\": 0,\n```\n\n\u003c/details\u003e\n\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the MIT License.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedarscript%2Fcedarscript-integration-aider","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcedarscript%2Fcedarscript-integration-aider","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedarscript%2Fcedarscript-integration-aider/lists"}