{"id":20777251,"url":"https://github.com/sharadcodes/iiswc-ai-ml-training","last_synced_at":"2025-08-18T12:10:32.578Z","repository":{"id":198826556,"uuid":"701170916","full_name":"sharadcodes/iiswc-ai-ml-training","owner":"sharadcodes","description":"IISWC AI/ML TRAINING FILES","archived":false,"fork":false,"pushed_at":"2023-10-07T07:20:21.000Z","size":5652,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-11T21:45:38.853Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://github.com/sharadcodes/iiswc-ai-ml-training/archive/refs/heads/main.zip","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sharadcodes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-06T04:41:47.000Z","updated_at":"2023-10-07T05:43:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"70ebb4be-8034-4707-ada0-12f38ddc0ff7","html_url":"https://github.com/sharadcodes/iiswc-ai-ml-training","commit_stats":null,"previous_names":["sharadcodes/iiswc-ai-ml-training"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sharadcodes/iiswc-ai-ml-training","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharadcodes%2Fiiswc-ai-ml-training","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharadcodes%2Fiiswc-ai-ml-training/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharadcodes%2Fiiswc-ai-ml-training/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharadcodes%2Fiiswc-ai-ml-training/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sharadcodes","download_url":"https://codeload.github.com/sharadcodes/iiswc-ai-ml-training/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharadcodes%2Fiiswc-ai-ml-training/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270989168,"owners_count":24680688,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T13:14:37.877Z","updated_at":"2025-08-18T12:10:32.550Z","avatar_url":"https://github.com/sharadcodes.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# iiswc-ai-ml-training\n\n# Downloads\nDownload all the datasets \u0026 code files\n## [Click here to download zip file](https://github.com/sharadcodes/iiswc-ai-ml-training/archive/refs/heads/main.zip)\n\n## ANN Code with explanation\n\n\u003eClick on the copy icon after hovering mouse over the code on top right corner of the code sections below and paste them in collab notebooks\n\n```python\n# Import necessary libraries\nimport pandas as pd  # Import the pandas library for data manipulation\nimport numpy as np  # Import the numpy library for numerical operations\nimport seaborn as sns  # Import seaborn for data visualization\nimport matplotlib.pyplot as plt  # Import matplotlib for plotting\nfrom sklearn import preprocessing  # Import preprocessing from scikit-learn for data scaling\nfrom sklearn.model_selection import train_test_split  # Import train_test_split for data splitting\nimport keras  # Import Keras for building and training neural networks\n```\n\nThese lines import the required Python libraries and modules for data handling, visualization, and neural network modeling.\n\n```python\n# Load the dataset\ndata_frame = pd.read_excel('./ann_dataset.xlsx')\n```\n\nThis line loads an Excel dataset located in the current directory and stores it in a pandas DataFrame named `data_frame`.\n\nThis line creates a temporary copy of the DataFrame for later reference.\n\n```python\n# Convert non-numeric values to NaN and drop rows with NaN values\ndata_frame = data_frame.applymap(lambda x: pd.to_numeric(x, errors='coerce')).dropna()\n```\n\nHere:\n- `applymap` applies the lambda function to each element of the DataFrame, converting non-numeric values to NaN.\n- `dropna` removes rows containing NaN values.\n\n```python\n# Generate a correlation matrix for the dataset\ncorr_matrix = data_frame.corr()\n```\n\nThis line calculates the correlation matrix for the numeric columns in the DataFrame, showing the pairwise correlations between features.\n\n```python\n# Create a heatmap to visualize the correlations between features\nplt.figure(figsize=(12, 8))\nsns.heatmap(corr_matrix, annot=True, cmap='coolwarm', linewidths=0.5)\nplt.title(\"Correlation Heatmap\")\nplt.show()\n```\n\nThese lines create a heatmap using Seaborn to visualize the correlations between features in the dataset.\n\n```python\n# Separate input (X) and output (y) variables\nX = data_frame.drop(['Evp'], axis=1)  # X contains input features\ny = data_frame['Evp']  # y contains the target variable\n```\n\nThis code separates the input features (`X`) and the target variable (`y`) from the DataFrame. `X` contains all columns except the 'Evp' column, while `y` contains only the 'Evp' column.\n\nThis line creates a temporary copy of the input features (`X`) for later reference.\n\n```python\n# Perform feature scaling on the input features\nscaler = preprocessing.StandardScaler()\nX = scaler.fit_transform(X)  # Standardize the input features\n```\n\nHere:\n- `StandardScaler` is used to standardize (scale) the input features.\n- `fit_transform` scales and standardizes the features, making them have a mean of 0 and a standard deviation of 1.\n\n```python\n# Split the data into training and testing sets\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n```\n\nThese lines split the data into training and testing sets using the `train_test_split` function:\n- `X_train` and `y_train` contain the training data and labels.\n- `X_test` and `y_test` contain the testing data and labels.\n- `test_size` specifies the proportion of data to be used for testing (20% in this case).\n- `random_state` ensures reproducibility by providing a fixed random seed.\n\n```python\n# Create a neural network model using Keras\nmodel = keras.Sequential([\n    keras.layers.Dense(11, activation='relu', input_dim=X.shape[1]),  # Input layer with 11 neurons and ReLU activation\n    keras.layers.Dense(100, activation='relu'),  # Hidden layer with 100 neurons and ReLU activation\n    keras.layers.Dense(20, activation='relu'),   # Hidden layer with 20 neurons and ReLU activation\n    keras.layers.Dense(8, activation='relu'),    # Hidden layer with 8 neurons and ReLU activation\n    keras.layers.Dense(1)  # Output layer with 1 neuron (for regression)\n])\n```\n\nThis code defines a feedforward neural network model using Keras with multiple layers:\n- `Sequential` creates a sequential model where layers are added one after the other.\n- `Dense` layers represent fully connected layers.\n- `input_dim` is specified in the first layer since it's the input layer.\n- Activation functions ('relu' for ReLU) are applied to the neurons in each layer.\n\n```python\n# Compile the model\nmodel.compile(loss='mean_squared_error', optimizer=keras.optimizers.Adam(learning_rate=0.000001), metrics=['mae'])\n```\n\nHere:\n- `compile` compiles the model for training.\n- `loss` is set to 'mean_squared_error,' which is the mean squared error (MSE) loss for regression problems.\n- `optimizer` is set to Adam with a specific learning rate (0.000001).\n- `metrics` is set to 'mae' (Mean Absolute Error) for evaluation during training.\n\n```python\n# Train the model\nhistory = model.fit(X_train, y_train, batch_size=32, epochs=1500, validation_data=(X_test, y_test), callbacks=[keras.callbacks.EarlyStopping(patience=100)])\n```\n\nThis code trains the model:\n- `fit` trains the model on the training data.\n- `batch_size` determines the number of samples in each mini-batch.\n- `epochs` specifies the number of training epochs.\n- `validation_data` is used for validation during training.\n- `callbacks` include early stopping to prevent overfitting by stopping training if validation loss doesn't improve for a specified number of epochs.\n\n```python\n# Evaluate the model on the testing data\nmse, mae = model.evaluate(X_test, y_test)\nprint(f\"Mean Squared Error on Test Set: {mse}\")\nprint(f\"Mean Absolute Error on Test Set: {mae}\")\n```\n\nThis code evaluates the trained model on the testing data and prints the MSE and MAE.\n\n```python\n# Plot training and validation loss\nplt.figure(figsize=(12, 6))\nplt.subplot(1, 2, 1)\nplt.plot(history.history['loss'], label='Training Loss')\nplt.plot(history.history['val_loss'], label='Validation Loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.legend()\nplt.title('Loss vs. Epochs')\n\n# Plot training and validation MAE\nplt.subplot(1, 2, 2)\nplt.plot(history.history['mae'], label='Training MAE')\nplt.plot(history.history['val_mae'], label='Validation MAE')\nplt.xlabel('Epochs')\nplt.ylabel('MAE')\nplt.legend()\nplt.title('MAE vs. Epochs')\n\nplt.tight_layout()\nplt.show()\n```\n\nThese lines create two subplots to visualize the training and validation loss, as well as training and validation MAE (Mean Absolute Error) over epochs.\n\n```python\n# Save the trained model for future use\nmodel.save('my_model.h5')\n```\n\nThis line saves the trained Keras model to a file named 'my_model.h5' for future use.\n\n```python\n# Load the previously saved model\nold_model = keras.models.load_model('my_model.h5')\n```\n\nThis line loads a previously saved Keras model from the 'my_model.h5' file.\n\n\n---\n\n\n# Manual testing:\n\n```python\n# importing numpy\nimport numpy\nfrom sklearn import preprocessing\n\n# Create a StandardScaler object\nscaler = preprocessing.StandardScaler()\n\n# Manual testing by providing input variables\nsingle_row = [10.6, 10.6, 15.4, 13.6, 19.8, 8.9, 100, 82, 5, 8.7, 0]\n\n# Performing scaling as we did on our training data\nscaled_single_row = scaler.fit_transform(numpy.array(single_row).reshape(1, -1))\n```\n\n1. `import numpy`: Import the numpy library, a powerful library for numerical operations in Python.\n\n2. `from sklearn import preprocessing`: Import the preprocessing module from scikit-learn, which provides tools for data preprocessing, including feature scaling.\n\n3. `scaler = preprocessing.StandardScaler()`: Create a StandardScaler object named `scaler`. The `StandardScaler` is used to standardize (scale) input data, ensuring that it has a mean of 0 and a standard deviation of 1.\n\n4. `single_row`: Define a list named `single_row` that contains a set of input values. This list represents a single data point that you want to make predictions on.\n\n5. `scaled_single_row`: Perform feature scaling on the `single_row` data point. The `scaler.fit_transform()` method is used to scale the data. `numpy.array(single_row).reshape(1, -1)` converts the `single_row` list into a 2D NumPy array (required by the scaler), and then `fit_transform` scales it based on the mean and standard deviation learned during training.\n\nThe reshape function is used to convert the 1D array into a 2D array with a single row and an automatically calculated number of columns. The -1 argument in the reshape function is a placeholder that tells numpy to calculate the number of columns based on the size of the input array.\n\n```python\n# Predicting the output\npredictions = old_model.predict(scaled_single_row)\n```\n\n6. `predictions`: Use the loaded neural network model (`old_model`) to make predictions on the scaled input data (`scaled_single_row`). This line generates predictions for the target variable based on the provided input data.\n\n```python\n# Printing the predictions\npredictions[0][0]\n```\n\n7. `predictions[0][0]`: Access the predicted value from the predictions array. Since `predictions` may contain multiple predictions for different data points, `[0][0]` is used to retrieve the first (and in this case, the only) prediction. This value represents the model's prediction for the target variable based on the provided input.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharadcodes%2Fiiswc-ai-ml-training","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsharadcodes%2Fiiswc-ai-ml-training","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharadcodes%2Fiiswc-ai-ml-training/lists"}