{"id":34984175,"url":"https://github.com/erdc/ampl","last_synced_at":"2025-12-27T01:23:54.153Z","repository":{"id":242737058,"uuid":"706836273","full_name":"erdc/ampl","owner":"erdc","description":"Automated Machine-learning PipeLine","archived":false,"fork":false,"pushed_at":"2025-06-27T17:05:46.000Z","size":43479,"stargazers_count":5,"open_issues_count":1,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-06-27T17:38:18.879Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/erdc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-10-18T17:57:31.000Z","updated_at":"2025-06-27T17:05:50.000Z","dependencies_parsed_at":"2025-06-27T17:37:59.980Z","dependency_job_id":null,"html_url":"https://github.com/erdc/ampl","commit_stats":null,"previous_names":["erdc/ampl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/erdc/ampl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdc%2Fampl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdc%2Fampl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdc%2Fampl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdc%2Fampl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/erdc","download_url":"https://codeload.github.com/erdc/ampl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erdc%2Fampl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28067561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-26T02:00:06.189Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-27T01:23:53.583Z","updated_at":"2025-12-27T01:23:54.140Z","avatar_url":"https://github.com/erdc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# An Automated Machine Learning Pipeline (AMPL)\n\n* Why are we looking at NNs and ML models\n* A set of steps are defined for the process of training a NN and ML models\n* Name the steps and mention breifly what those steps are\n* Describe how the results are displayed with the percentage of points under a 20,10,5,2.5,0.5\n* With the steps defined, improving the workflow to allow for both non-technical and\n  technical users.\n* After improvements were made, the next goal was to allow for automating the process of training a network.\n\nMachine Learning techniques have risen in popularity as ML has shown to be useful in providing an expert level response to predicting values, recognizing patterns, and identifying objects in images. While working through applying ML to [\"SYNTHETIC CFD ESTIMATION FOR BLACKHAWK AIRFOIL DRAG COEFFICIENT\"](https://doi.org/10.2514/6.2024-1230), and [\"ESTIMATING KINETIC ENERGY REDUCTION FOR TERMINAL BALLISTICS\"](https://link.springer.com/article/10.1007/s00521-023-09382-3), it was noted that the steps for applying ML was similiar enough to where a single workflow could be designed to handle both of these problems along with previously unexplorered problem spaces. The steps that were taken for both the rotorcraft and ballistics problem were Feature Importance, Hyperparamter optimization which is searching for superior model parameters, training the best models returned from hyperparamter optimization, and evaluating the performance of the best models. This documentation will describe the details of each of the previously mentioned steps. Details will also be provided on how to utilize each of the steps individually or as an automated workflow. In this documentation we will describe an automated machine learning pipeline and present results of applying the pipeline to sample problems.\n\nThe workflow steps used to design the models used for predicting values for the Ballistics and Rotorcraft work were the same steps. Since the Ballistics work and the rotorcraft work use very different data, but the steps in the workflow were the same, a general workflow that could design ML models for many different problems was desired. Having a general method would reduce effort in the beginning stages of working on a new problem or dataset and allow for exploration of methods and techniques to create better models. The general method would also remove the need to implement each step from ground up and would improve the timeline from receiving the data to having a reasonably performing model. The method for the general workflow is called \"An Automated Machine Learning Pipeline\" and the method will fit the following criteria:\n\n* Easy to get started for novice machine learning users\n* Have sufficient options for expert users\n* The tools used in each step must be able to be modifiable or replaceable to allow improvements to the pipeline\n* Able to run each step in the workflow as a solo component\n* Eliminate the need to manually design ML model to allow more time to be focused on adding new or enhancing existing Steps in the automated machine learning pipeline\n\n## Steps to run AMPL\n\n1. [Anaconda (Conda) setup](#anaconda-conda-setup)\n2. [Install AMPL](#install-ampl)\n   * [GitLab setup](./README_git.md?ref_type=heads#gitlab-setup)\n      * [Option 1: Get SSH key](./README_git.md?ref_type=heads#option-1-get-ssh-key-recommended)\n      * [Option 2: Get HTTPS Access Token](./README_git.md?ref_type=heads#option-2-get-https-access-token-needed-to-clone-the-repo-as-well-as-pullpush)\n   * [Clone AMPL repository](#clone-ampl-repository)\n   * [Install AMPL in your conda environment](#install-ampl)\n      * [Initial Steps for Running AMPL](#initial-steps-for-running-ampl)\n3. [AMPL setup and Configuration file](#ampl-setup-and-configuration-file)\n    * [Suggested directory structure for organizing AMPL](#suggested-directory-structure-for-organizing-AMPL)\n4. [Two Methods for Interfering with AMPL](#Two-Methods-for-Interfering-with-AMPL---API,-CLI)\n    * [AMPL - API](#ampl-api)\n    * [AMPL - CLI](#ampl-cli)\n\n## Anaconda (Conda) setup\n\nPlease download and install Anaconda python if it has not previously been installed. Installatiion instructions can be found within [README_anaconda](./README_anaconda.md?ref_type=heads#install-anaconda) if needed by the user.\n\n### Anaconda Setup\n\nIt is important to maintain an up-to-date version of Anaconda. Even if a user already has Anaconda, please follow the steps for Updating Conda and Anaconda. Not having an updated version of conda is the most commonly experienced error by new users.\n\n1. Update Conda and Anaconda (Recommended for all users)\n\nUpdate the conda package manager to the latest version in your base environment\n```shell\nconda update -n base conda -y\n```\n\nUse conda to update Anaconda to the latest version in your base environment\n```shell\nconda update -n base anaconda -y\n```\n\n```shell\nconda activate    \n```\nYou should then see (base).\n\n2. Create a new conda environment named `ampl` and install the libraries and modules required for AMPL.\n\nNote: If the user experiences an error during these installs, the most common source for that error is the need to update conda and anaconda as seen in the previous step.\n\n```shell\nconda create -n ampl python=3.11 pandas numpy yaml scikit-learn jupyter recommonmark scikit-learn-intelex plotly::plotly anaconda::sphinx -y\n```\n\n```shell\nconda install -n ampl -c conda-forge optuna matplotlib imbalanced-learn sphinx-gallery cloud_sptheme -y\n```\n\n```shell\nconda activate ampl\n```\n\n```shell\npython -m pip install --upgrade pip myst-parser joblib\n```\n\n```shell\npip install shap pillow requests xgboost jinja2 more_itertools optuna-integration tensorflow\n```\n\n## Setting up gitlab\n\nThe user will need to setup their SSH key to git in order to pull the source code for AMPL. This step can be skipped by users who have already setup their git SSH key. This process for setting up the git SSH key will be the same whether a user is running locally or on HPC as a user will need to store an SSH key for each machine they use to access git. Instructions for setting up git are found here within the [README_git] (./README_git.md?ref_type=heads#GitLab-setup)\n\n### Creating AMPL Code and Run/working Directory - for both API and CLI users\n\nThe directory structure for working with AMPL is the same regardless of using the API or CLI method to interface with AMPL. This recommeneded directory structure will help with organization and determining where to put the AMPL repository, as well as provide a convienient way to organize the directory structure so that ML models and information (plots, statistics, etc..) are easily accessed by the user. A user can use any directory structure they would like, but will have to customize their yaml input file to account for any differences between their chosen directory structure and the recommended directory structure.\n\n#### Recommended directory structure\n\nThe user may use multiple methods for creating a directory structure, however, the example below assumes the user is working from commandline. For the example working with AMPL, we will use an open-source concrete dataset. When a user is working with their own data they should rename any references to 'concrete' with their own dataset name.\n\n1. Create a parent folder for AMPL Code\n\nex: $ mkdir \u003campl_dir\u003e \n```shell\nmkdir AMPL\n```\n\n2. Move into the \u003campl_dir\u003e\n\nex: cd \u003campl_dir\u003e \n```shell\ncd AMPL \n```\n\n3. Create AMPL repository code directory:\n\nwhatever you would like to name your designated AMPL repository code directory \nex: mkdir \u003campl_code_dir\u003e\n```shell\nmkdir code\n```\n\n4. Create directory for AMPL runs:\n\nwhatever you would like to name your directory to hold all of the AMPL runs.\nex: mkdir \u003call_run_dir\u003e \n```shell\nmkdir all_run_dir\n```\n\n5. Move into the \u003call_run_dir\u003e\n\nex: cd \u003call_run_dir\u003e \n```shell\ncd all_run_dir\n```\n\n6. Create directory for a specific run of AMPL:\n\nwhatever you would like to name your designated AMPL working directory based on the data being used.\nhere we create concrete_run_dir for holding information for the concrete data example. Please name\nconcrete to something different based on the data you are working with.\nex: mkdir \u003cdataset#_run_dir\u003e\n\n```shell\nmkdir concrete_run_dir\n```\n\nNote: Repeat step six to create the structure to maintain the data for different data sets and runs. Example directory structure is provided below.\n\n#### Example Directory Structure\n\nThe following is an example directory structure of what AMPL will look like when the code is downloaded and a user is working with multiple datasets. This example shows where the ampl code is located, and provides two examples of where the run data is stored. The first example the user has a csv file and uses one yaml file for specifying how AMPL will run. The second example the user has a sqlite data file and two different yaml files for specifying how AMPL with run when working with the same dataset. AMPL will accept both csv and sqlite data, however, the user will need to modify the yaml file based on which type of data is being used. Examples for working with csv or sqlite are provided below.\n\nNote: When working with AMPL, the normal use case is to create a Folder named 'AMPL' to store everything. This folder is the root folder and should be the folder a user opens when working with AMPL whether through the [API](#ampl-api) or through the [command-line option](ampl-cli).\n\n```shell\n├── \u003campl_dir\u003e\n│   ├── \u003campl_code_dir\u003e\n│   │   ├── \u003campl\u003e\n│   │   │   ├──\u003cdocs\u003e\n│   │   │   ├──\u003cexamples\u003e\n│   │   │   ├──\u003csrc\u003e\n│   │   │   ├──\u003ctests\u003e \n│   ├── \u003call_run_dir\u003e\n│   │   ├── \u003cdataset1_run_dir\u003e\n│   │   │   ├── \u003cdata_dir\u003e\n│   │   │   │   ├── \u003cdata1.csv\u003e\n│   │   │   ├── \u003cconfig_file.yml\u003e\n│   │   ├── \u003cdataset2_run_dir\u003e\n│   │   │   ├── \u003cdata_dir\u003e\n│   │   │   │   ├── \u003cdata2.sqlite\u003e\n│   │   │   ├── \u003cconfig_file1.yml\u003e\n│   │   │   ├── \u003cconfig_file2.yml\u003e\n```\n\n### Clone AMPL repository\n\n1. Navigate to your AMPL repository code directory \u003campl_code_dir\u003e created in the previous step [Recommended directory structure](#recommended-directory-structure):\n\nex: cd \u003campl_dir\u003e/\u003campl_code_dir\u003e\n```shell\ncd code\n```\n\n\nThe following is an example of the user's directory structure \n```shell\ncd ../code\n  ├── ampl_dir\n-\u003e│   ├── code\n  │   ├── all_run_dir\n```\n\n\n4. Go to the AMPL Git repo page in your browser, select Code and then there will be two options to `Clone`. Copy the URL based on whether you are using the SSH Key or HTTPS option (SSH is recommended). Use the following command to clone the repo into your AMPL working directory:\n\n\nexample using ssh:\nex: git clone \u003ccopied URL\u003e\n```shell\ngit clone git@github.com:erdc/ampl.git\n```\n\nNote: If you don't have access or are getting a permission error from Git, please refer to [Option 1: Get SSH key](./README_git.md?ref_type=heads#option-1-get-ssh-key-recommended)\n\n#### Install AMPL in your conda environment\n\n Note for Windows users: Please open Anaconda power shell as an administrator for the commands to work.\n\n1. Activate the conda environement you created previously while installing Anaconda.\n\n  ex: conda activate \u003cenv_name\u003e\n  ```shell\n  conda activate ampl \n  ```\n\n2. Install the needed packages.\n\n  ```shell\n  # Navigate to the inside the code folder. \n    ├── AMPL\n  -\u003e│   ├── code\n  ```\n  \n  ex: pip install -e \u003campl_code_dir\u003e\n  ```shell\n  pip install -e ampl\n  ```\n\n3. If the install was successful then you will see soemthing similiar to:\n\n  ```shell\n     Successfully built ampl\n     Installing collected packages: ampl\n     Successfully installed ampl-0.0.3\n  ```\n\n# Test AMPL in your env by running the following:\n\n\n1: Make sure you are in the right conda environment, i.e. the ampl environment\n  ex: conda activate \u003cenv_name\u003e\n  ```shell\n  conda activate ampl\n  ```\n2: Run some tests that are part of the ampl code by first navigating to the tests directory\n  ex: cd \u003campl_code_dir\u003e/tests\n\n  ```shell\n  cd ampl/tests\n  ```\n \n3: Test the pipeline connections\n  ```shell\n  python -m unittest test_pipeline_nn\n  ```\n\n  The command above that tests the pipeline connection does a small test to confirm if the connections are set up properly.  It starts with using Optuna to find the best trial and then runs 10 epochs to train that best trial.  After it is done running, it displays the reuslts of the test in a table that includes the layer type, output shape, and number of parameters. Below the table are more detials about the parameters that are used in the test run.\n\n### AMPL setup and Configuration file\n\n#### Initial Steps for Running AMPL\n\nAMPL utilizes configuration file(s) to setup a study/run.\n\nWhen AMPL is executed, a .yaml file must be passed in as an argument for running in either CLI or API mode. The .yaml file contains a path to the location of the input data, which is used for training, validation, testing, as well as the path to the results directory. The location of these directories are relative to the aforementioned path provided in the .yaml in addition to many other configuration settings.\n\n[Default Config File](./src/ampl/default_config.yml)\n\n#### Suggested directory structure for organizing AMPL\n\nYou will want to create a directory for your specific AMPL run to keep your runs organized, especially if you will be running AMPL on multiple datasets. This is where all your project related folders should be created and where your ampl_config.yml should reside. We recommend creating a separate folder to store data within it and refer to it in the config file.\n\n1. Navigate to your AMPL running directory.  The run directory is typically located in the home directory \u003campl_run_dir\u003e, previously refered to as AMPL/all_run_dir in examples:\n\n\n  ex: cd \u003campl_dir\u003e/\u003campl_run_dir\u003e\n  from the AMPL root directory\n  ```shell\n  cd all_run_dir\n  ```\n   \n  ```shell\n  The folder structure will look like this\n    ├── ampl_dir\n    │   ├── code\n  -\u003e│   ├── all_run_dir\n  ```\n  \n2. Create a directory for your run and navigate to this directory.  Use a name that indentifies the dataset you will be using. Since the example that we will be using is based on a concrete dataset, we will name the directory appropriately:\n\n  ex: mkdir \u003cdataset_run_dir\u003e\n  ```shell\n  mkdir concrete_run_dir\n  ```\n\n  ```shell\n  The folder structure will look like this\n    ├── ampl_dir\n    │   ├── code\n    │   ├── all_run_dir\n  -\u003e│   │   ├── concrete_run_dir\n  ```\n\n3. Move into the \u003cconcrete_run_dir\u003e directory:\n\n# ex: cd \u003cdataset_run_dir\u003e \n```shell\ncd concrete_run_dir\n```\n```shell\n# The current folder structure\n  ├── ampl_dir\n  │   ├── code\n  │   ├── all_run_dir\n-\u003e│   │   ├── concrete_run_dir\n```\n\n4. Create a directory to hold your data:\n\n# ex: mdir \u003cdata_dir\u003e\n```shell\nmkdir concrete_data\n```\n\n```shell\n# The current folder structure\n  ├── AMPL\n  │   ├── code\n  │   ├── all_run_dir\n  │   │   ├── concrete_run_dir\n-\u003e│   │   │   ├── concrete_data\n```\n\n5. Copy your dataset to the \u003cdata_dir\u003e directory. Please have the data in a SQLite or a CSV file type.  Most datasets can easily be converted into CSV format, but be sure that the index column is not included.\n\nAs an example data set moving forward, we will be utilizing a public concrete data set. Copy the \"concrete.csv\" file from \"AMPL\\code\\ampl\\tests\\data\\concrete.csv\"  to the concrete_data folder created in the previous step. The below folder sructre shows where the concrete.csv data file shoud go.\n\n```shell\n    # the current folder structure  \n    ├── AMPL\n    │   ├── code\n    │   ├── all_run_dir\n    │   │   ├── concrete_run_dir\n    │   │   │   ├── concrete_data\n  -\u003e│   │   │   │   ├── concrete.csv\n```\n\nA direct link can be found to the example concrete dataset at the following link:\n  \u003chttps://archive.ics.uci.edu/dataset/165/concrete+compressive+strength\u003e\n\n  Yeh,I-Cheng. (2007). Concrete Compressive Strength. UCI Machine Learning Repository. \u003chttps://doi.org/10.24432/C5PK67\u003e.\n\n6. Create a default configuration file:\n\n  Run the following code from within the \u003cconcrete_run_dir\u003e directory in the terminal. The following command is using the CLI method of interfacing with ampl.\n\n  ```shell\n  python -m ampl ampl_config.yml -c\n  ```\n\n  ```shell\n  The folder structure will look like this\n    ├── ampl_dir\n    │   ├── code\n    │   ├── all_run_dir\n  -\u003e│   │   ├── concrete_run_dir \u003c--- from inside this folder\n    │   │   │   ├── concrete_data\n    │   │   │   │   ├── concrete.csv\n    │   │   │   ├── ampl_config.yml \u003c---this file will be created\n  ```\n\n7. Edit the `ampl_config.yml` file by filling in all the required fields.\n\nThe following block of code provides a description of the variables that require user modification in the ampl_config.yml. A table is provided later in this step that shows the edits to the yaml file for the example using the concrete dataset. When using a different dataset, please refer back to the table in this section as a quick reference for the variables you will need to modify when applying ampl to a different dataset.\n\nThe following block provides the variables a user will need to modify in order to run AMPL along with a description of the variables. Look over the next block but not change is needed yet.\n```shell\n# A name that describes what the study is about and needs to be unique.\nstudy_name: 'your_study_name'  \n# The column that you are trying to predict\ntarget_variable: 'your_target_variable'\n# Name of the data     \ndataset_name: 'your_dataset_name'\n# The path to the SQLITE data file, set to NULL if using a CSV file. \ndata_file: null \n# If the data file is SQLite, the data_table_name needs to be the name of the table you wish to use for the study from the SQLite data file.  This is null if using CSV data.\ndata_table_name: null \n# The path to the CSV file, set to null for SQLite.\ncsv_file: 'path_to_CSV'\n# The path where it will save the normalized version of the CSV file.\n# It is not mandatory to normalize the data, but it is recommended. \n# If using a sqlite input file this line should be set to null\ncsv_normalized_file : 'path_to_CSV' + '_normalized'\n# If you have any columns with categorical data, this enumerates them to numerical data.\n# Set to null if there is no categorical data\ncols_to_enum:  null \n# Include all of the possible features (columns) to use.  When listing out columns, make sure to not add the column from the target_variable(s).\nfeature_list: 'col_1'\n              'col_2'\n              'col_3'\n              'col_4'\n              'col_5'\n              'col_6'\n    \n# The number of columns used as features in the dataset. Do not include the include the target variable. \nnumber_of_features: 6 \n```\n\nThe following table shows all of the variables in the `ampl_config.yml` file that require modification, their default values, and an example modification to work with the concrete data set example. \nDo note that even though 'data_file' and 'data_table_name' are listed in the table below, they are not modified for the concrete example. 'data_file' and 'data_table_name' are for working with sqlite which is not used in this example. \n\n| Variable           | Default Variables         | Concrete Variables                                                   |\n| :----------------- | :------------------------ | :----- |\n| study_name         | 'your_study_name'         | 'compress_strength' |\n| target_variable    | 'your_target_variable'    | 'Concrete compressive strength' |\n| dataset_name       | 'your_dataset_name'       | 'strength_estimate' |\n| data_file          | null                      | null |\n| data_table_name    | null                      | null |\n| csv_file           | 'path_to_CSV'             | 'concrete_data/concrete.csv' |\n| cols_to_enum       | null                      | null |\n| feature_list       | null                      | - 'Cement' - 'Blast Furnace Slag' - 'Fly Ash' - 'Water' - 'Superplasticizer' - 'Coarse Aggregate' - 'Fine Aggregate' - 'Age' |\n| number_of_features | 6                         | 8 |\n\nMost of the fields in the table above are a copy and paste into the ampl_config.yml file. However, the feature list requires a little editing. The following is how the feature_list using the concrete example should look in the yaml file\n\n```shell\n# The following is how the feature_list should look in the yml file\nfeature_importance:\n  feature_list:  # REQUIRED USER MODIFICATION # List of columns to include in study/run, don't include any target columns\n    - 'Cement' \n    - 'Blast Furnace Slag' \n    - 'Fly Ash'\n    - 'Water'\n    - 'Superplasticizer'\n    - 'Coarse Aggregate' \n    - 'Fine Aggregate'\n    - 'Age'\n```\n\nThe following command will run the concrete example from the concrete_run_dir. The directory structure of where the user should be running the command is now provided followed by the commands the user needs to run:\n\n```shell\n    ├── ampl_dir\n    │   ├── code\n    │   ├── all_run_dir\n  -\u003e│   │   ├── concrete_run_dir \u003c--- run the command from inside this directory\n    │   │   │   ├── concrete_data\n    │   │   │   │   ├── concrete.csv\n    │   │   │   ├── ampl_config.yml\n```\n\n```shell\ncd concrete_run_dir #cd \u003cdataset1_run_dir\u003e\npython -m ampl ampl_config.yml\n```\n\n\nThis is what the directory structure looks like after using the examples provided in each of the previous steps in this guide and after running the concrete example code. You will find plots, graphs, and other information in the 'results_compress_strength' folder:\n\n```shell\n├── AMPL\n│   ├── code\n│   ├── all_run_dir\n│   │   ├── concrete_run_dir\n│   │   │   ├── concrete_data\n│   │   │   │   ├── concrete.csv\n│   │   │   ├── ampl_config.yml\n│   │   │   ├── results_compress_strength\n│   │   │   │   ├── plots\n│   │   │   │   ├── saved_models\n```\n\n### Two Methods for Interfering with AMPL - API, CLI\n\nAMPL provides two different ways to interface with it. One is through the AMPL API (Application Programming Interface) and the other is via AMPL CLI (Command Line Interface). The API will be helpful to users wishing to use scripts to run AMPL, and the CLI will be helpful for those wishing to run AMPL without writing scripts. An example of using the CLI would be running on HPC. Both the AMPL API and the AMPL CLI will be described in detail in their respective sections below.\n\n#### AMPL API\n\nThe AMPL API will provide the user with the means to interface with AMPL through scripts. This type of interface is useful for creating scripts to run AMPL as part of a workflow. In the next section we will show an example of using a script that uses the AMPL API.\n\n##### Getting started API\n\nThe following script will use AMPL to create a fully dense neural network using the previously created [folder structure](#Suggested-directory-structure-for-organizing-AMPL), and example downloaded concrete.csv dataset. This script is available in the examples folder under [examples/concrete_example.py](\"./examples/concrete_example.py\")\n\n```python\nfrom ampl import *\nfrom ampl.util import Util\nimport os\n# Change the directory \nfilename = 'ampl_config.yml'\n# If using an ide, open the AMPL root folder. See the \n# comments at the top of this file for an example directory structure\nwork_dir = './all_run_dir/concrete_run_dir/'\n# This will reset the running directory to concrete_run_dir\nos.chdir(work_dir)\n# Print the files located at the relative path. This will help with finding the yml \n# file if a user receives file not found while trying to work with relative paths.\n#Util.relativePathHelper(work_dir)\nconfig = Configuration(filename)\npipeline = config.create_pipeline_nn()\npipeline.run_all()\n```\n\nThe following is an example using python to create a yaml file. You will still need to edit the yaml file with the information for the run. The lines within the yaml file that require modification are listed in an example within the [Suggested directory structure for organizing AMPL](Suggested-directory-structure-for-organizing-AMPL) section. This script is available in the examples folder under [examples/create_config_api_example.py](\"./examples/create_config_api_example.py\")\n\n```python\nfrom ampl import *\nfrom ampl.util import Util\nimport os\n# Project should be run \n# change the directory \nfilename = 'ampl_config.yml'\n# Open the root AMPL project in your ide for running this code\n# An example of the folder structure can be located within the readme \n# README.md -\u003e AMPL Interface -\u003e Creating AMPL Code and Run/working Directory - for both API and CLI users -\u003e Example Directory Structure\n# use the concrete_run_dir from the root folder of AMPL\nwork_dir = './all_run_dir/concrete_run_dir/'\n# this will reset the running directory to concrete_run_dir\nos.chdir(work_dir)\n# Create default config file\nUtil.create_default_config_file(filename)\n```\n\nThe next example shown here is for both Neural networks and Decision Trees. The following code is an example of what to do if your dataset contains columns that are used to create a target column. In the below example we use the starting_velcoity cubed minus the ending_velocity cubed to populate the values within the target column for every row within the dataframe. The commented lines\n\n```python\nimport ampl\n# Ignore this step if the dataset contains the target column already.\n# Here the 'starting_Velocity(m/s)' and 'ending_Velocity(m/s)' are two columns that are already part of the dataset and are used\n# to create the target column \ntarget_func = lambda df_: df_['starting_Velocity(m/s)'].astype('float') ** 3 - df_['ending_Velocity(m/s)'].astype('float') ** 3\n                            \nconfig = ampl.Configuration('data/pipeline_config.yml', target_col_function=target_func)\n# Creating Neural Network AMPL\npipeline_nn = config.create_pipeline_nn()\npipeline_nn.optuna.run()\n# Getting Best Trial and Top Trials from Optuna Study\nbest_trial_df, top_trials_df = pipeline_nn.optuna.load_trials_df() \nprint(best_trial_df)\nprint(top_trials_df)\npipeline_nn.build.run()\npipeline_nn.eval.run()\n# only a few methods for ensembling are implented at this time. \n#pipeline_nn.ensemble.run()\n \n# Creating Decision Tree AMPL - If the user would like to try decision trees instead of neural networks, uncomment the lines below\n# pipeline_dt = config.create_pipeline_dt()\n# pipeline_dt.optuna.run()\n# pipeline_dt.build.run()\n# pipeline_dt.eval.run()\n# pipeline_dt.ensemble.run()\n```\n\n#### AMPL CLI\n\nThe AMPL CLI will provide an easy to use means for calling AMPL functionality through the command line. The CLI is helpful when the user doesn't want to write scripts to call AMPL, but wants to call the functions of AMPL. An example application of using the CLI would be to run AMPL on HPC.\n##### Getting started CLI\nOnce AMPL is installed in your conda env, use a terminal to run AMPL.\nAMPL CLI help - To display the AMPL CLI help use the following command\n```shell\npython -m ampl -h\n```\nIt will display similar text as shown below, this text may differ as the application matures\n```text\nusage: FAIT AMPL package [-h] [-d] [-c] [-dt] [-o] [-b] [-ev] [-en]\n                         config_file\nProcess AMPL Pipeline Configuration YAML file.\npositional arguments:\n  config_file           Path to YAML Configuration file\noptional arguments:\n  -h, --help            show this help message and exit\n  -d, --debug           Print debug info\n  -c, --create_config   Creates a new default YAML Configuration file\n  -dt, --decision_tree  Use Decision Tree instead of Neural Networks\n  -o, --optuna          Run Optuna Model step\n  -b, --build           Run Build Model step\n  -ev, --evaluate       Run Evaluate Model step\n  -en, --ensemble       Run Ensemble Model step\n```\n##### Run AMPL using CLI\nFrom the terminal run the following commands. The examples assume that steps were taken to create the following directory structure:\n├── AMPL\n│   ├── code\n│   ├── all_run_dir\n│   │   ├── concrete_run_dir\n│   │   │   ├── concrete_data\n│   │   │   │   ├── concrete.csv\n│   │   │   ├── ampl_config.yml\n```shell\nconda activate ampl\ncd concrete_run_dir #cd \u003cdataset1_run_dir\u003e\npython -m ampl ampl_config.yml \n```\nHowever, if you would like to run a single step within the pipeline, you can use the following commands instead. For example, to just run the build and evaluate steps, you would use the following AMPL-CLI command with the 'build' and 'evaluate' command line parameters. Additional command line parameters are located in the [Getting started CLI](#Getting-started-CLI) section:\n```shell\ncd concrete_run_dir #cd \u003cdataset1_run_dir\u003e\npython -m ampl ampl_config.yml -b -ev\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferdc%2Fampl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ferdc%2Fampl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferdc%2Fampl/lists"}