{"id":14958637,"url":"https://github.com/anonym0uswork1221/jaraconverse-transformersbased","last_synced_at":"2026-02-27T12:16:24.163Z","repository":{"id":251404352,"uuid":"837343805","full_name":"Anonym0usWork1221/JaraConverse-TransformersBased","owner":"Anonym0usWork1221","description":"This JaraConverse model is a cutting-edge Transformer-based supervised Language Model (LLM) specifically designed for generating Python code snippets.","archived":false,"fork":false,"pushed_at":"2024-08-02T18:58:29.000Z","size":5389,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-26T03:13:36.874Z","etag":null,"topics":["ai","code-generator","conversational-ai","deep-neural-networks","keras","keras-nlp","large-language-model","llm","machine-learning","optimized","python","scratch","tensoflow","transformers","transformers-models"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Anonym0usWork1221.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-02T18:43:49.000Z","updated_at":"2024-10-31T22:55:44.000Z","dependencies_parsed_at":"2024-08-02T20:01:18.757Z","dependency_job_id":"d24f7f83-1715-4c66-a25f-51d7190490e6","html_url":"https://github.com/Anonym0usWork1221/JaraConverse-TransformersBased","commit_stats":null,"previous_names":["anonym0uswork1221/jaraconverse-transformersbased"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonym0usWork1221%2FJaraConverse-TransformersBased","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonym0usWork1221%2FJaraConverse-TransformersBased/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonym0usWork1221%2FJaraConverse-TransformersBased/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Anonym0usWork1221%2FJaraConverse-TransformersBased/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Anonym0usWork1221","download_url":"https://codeload.github.com/Anonym0usWork1221/JaraConverse-TransformersBased/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245579675,"owners_count":20638679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","code-generator","conversational-ai","deep-neural-networks","keras","keras-nlp","large-language-model","llm","machine-learning","optimized","python","scratch","tensoflow","transformers","transformers-models"],"created_at":"2024-09-24T13:17:38.465Z","updated_at":"2026-02-27T12:16:23.351Z","avatar_url":"https://github.com/Anonym0usWork1221.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# JaraConverse - A Transformer-Based Supervised LLM\r\n\r\nJaraConverse is a state-of-the-art Transformer-based supervised Language Model (LLM) designed for generating Python code snippets. The model is trained using a dataset stored in an SQLite3 database and is equipped with advanced configuration options to optimize training and inference. This README provides an extensive overview of the model, its training process, and configuration details.\r\n\r\n *  Date   : 2024/08/02\r\n *  Author : **__Abdul Moez__**\r\n *  Version : 0.1\r\n *  [Repository](https://github.com/Anonym0usWork1221/JaraConverse-TransformersBased)\r\n\r\n MIT License\r\n\r\n## Dependencies\r\n* Python 3.9+\r\n* Tensorflow \u003c=2.15\r\n* Datasets\r\n* Transformers\r\n* codecarbon\r\n* plotly\r\n\r\n## License\r\nThis project is licensed under the MIT License.\r\n\r\n## Table of Contents\r\n- [Installation](#installation)\r\n- [Input Dataset](#input-dataset)\r\n- [Training the Model](#training-the-model)\r\n- [Visualizing Training Progress](#visualizing-training-progress)\r\n- [Running the Demo](#running-the-demo)\r\n- [Configuration Details](#configuration-details)\r\n  - [GlobalVariables.py](#globalvariablespy)\r\n\r\n## Installation\r\nBefore training the model, ensure you have all necessary dependencies installed. You can do this by running:\r\n```bash\r\npip install -r requirements.txt\r\n```\r\n\r\n## Input Dataset\r\nThis model uses an SQLite3 database by default, requiring two columns:\r\n`title` and `code` inside the `snippets` table. You can change these default settings in the `DataBaseConfiguration` enum in `GlobalVariables.py` if you are using a different format.\r\nA sample image of the required dataset structure is attached.\r\n\r\n\u003ca href = \"Images/database_samples.png\"\u003e\r\n  \u003cimg src = \"Images/database_samples.png\"/\u003e\r\n\u003c/a\u003e\r\n\r\n## Training the Model\r\nTo train the JaraConverse model, execute the following command:\r\n\r\n```bash\r\npython JaraConverseTrainer.py\r\n```\r\n\r\nEnsure your input data is formatted correctly in the SQLite3 database with columns for `title` and `code`. You can adjust these default column names in the `GlobalVariables.py` file, which holds all the configurations for the model.\r\n\r\n## Visualizing Training Progress\r\nJaraConverse uses TensorBoard for monitoring the training process. After training, you can visualize the training progress and other metrics by running:\r\n\r\n```bash\r\npython JaraConverseVisualizer.py\r\n```\r\n\r\nThis will launch TensorBoard and allow you to view detailed graphs and metrics of the training process.\r\n\r\n## Running the Demo\r\nThe demo script loads the model from a checkpoint and generates code snippets based on the input data. Run the demo script with:\r\n\r\n```bash\r\npython JaraConverseDemo.py\r\n```\r\n\r\nBy default, `JaraConverseDemo.py` loads the model from a checkpoint. This is due to compatibility issues when training on Colab and using the model on another system. Ensure you use the same parameters for loading the checkpoint as those used during training.\r\n\r\n## Configuration Details\r\nThe `GlobalVariables.py` file contains all the configuration parameters for the JaraConverse model. Below is a detailed explanation of each configuration parameter to help developers understand and customize the model.\r\n\r\n### GlobalVariables.py\r\n#### VariableParameters\r\nThis enum class holds the general parameters for model training and setup.\r\n\r\n```python\r\nclass VariableParameters(Enum):\r\n    MODEL_NAME: str = \"JaraConverse\"\r\n    SET_LIMIT_ON_GPU: bool = False\r\n    MAX_GPU_UTILIZATION_ON_LIMIT: int = 5\r\n\r\n    SET_LIMIT_ON_CPU: bool = False\r\n    OMP_THREADS: int = 5\r\n    MKL_THREADS: int = 5\r\n    INTER_AND_INTRA_OP_PARALLELISM_THREADS: int = 0\r\n\r\n    SAVED_STATES_NAME: str = \"saved_states.pkl\"\r\n    SAVED_HISTORY_NAME: str = \"saved_history.pkl\"\r\n    SAVED_MODEL_NAME: str = \"JaraConverse.keras\"\r\n    SAVED_MODEL_WEIGHTS_NAME: str = \"saved_weights.h5\"\r\n    CHECKPOINT_NAME: str = \"cp.ckpt\"\r\n\r\n    BASE_PATH: str = Path(__file__).parent.__str__()\r\n    MODEL_BASE_PATH: str = path.join(BASE_PATH, f\"{MODEL_NAME}Model\").__str__()\r\n    CHECKPOINT_DIR: str = path.join(MODEL_BASE_PATH, \"model_checkpoints\").__str__()\r\n    TENSORBOARD_DIR: str = path.join(MODEL_BASE_PATH, \"tensorboard\").__str__()\r\n\r\n    SAVED_STATES_DIR: str = path.join(MODEL_BASE_PATH, \"model_saved_states\").__str__()\r\n    CLEANED_DATASET_DIR: str = path.join(MODEL_BASE_PATH, \"cleaned_dataset\").__str__()\r\n    SAVED_MODEL_DIR: str = path.join(MODEL_BASE_PATH, \"trained_model\").__str__()\r\n\r\n    SAVED_MODEL_WEIGHTS_DIR: str = path.join(MODEL_BASE_PATH, \"trained_weights\").__str__()\r\n    VISUALIZER_DIR: str = path.join(MODEL_BASE_PATH, \"training_visualization\").__str__()\r\n\r\n    SAVED_HISTORY_PATH: str = path.join(SAVED_STATES_DIR, SAVED_HISTORY_NAME).__str__()\r\n```\r\n\r\n#### DataBaseConfiguration\r\nThis enum class configures the database parameters for training.\r\n\r\n```python\r\nclass DataBaseConfiguration(Enum):\r\n    TRAINING_DATABASE_PATH: str = path.join(VariableParameters.BASE_PATH.value, \"python_code_snippets.db\").__str__()\r\n    DATABASE_TABLE_NAME: str = \"snippets\"\r\n    UNNECESSARY_COLUMNS_IN_DB: list[str] = None\r\n\r\n    INPUT_DATA_COLUMN_NAME: str = \"title\"\r\n    OUTPUT_DATA_COLUMN_NAME: str = \"code\"\r\n    SPLIT_DATASET: bool = True\r\n\r\n    SPLIT_PERCENTAGE: float = 0.2\r\n    SHUFFLE_DATASET: bool = True\r\n    FORCE_REPROCESS_DATASET: bool = False\r\n```\r\n\r\n#### TransformersTokenizerConfiguration\r\nThis enum class configures the tokenizer parameters for the model.\r\n\r\n```python\r\nclass TransformersTokenizerConfiguration(Enum):\r\n    TOKENIZER_PATH: str = path.join(VariableParameters.MODEL_BASE_PATH.value, \"JaraConverseTokenizer\").__str__()\r\n\r\n    TRAIN_TOKENIZER: bool = False\r\n    TRAINING_TOKENIZER_DATA_COLUMN: str = \"code\"\r\n    TRAINING_TOKENIZER_VOCAB_SIZE: int = 52000\r\n\r\n    TRAINING_SEED: int = 2050\r\n    TRAINING_BATCH_SIZE: int = 32\r\n    VALIDATION_BATCH_SIZE: int = 8\r\n```\r\n\r\n#### JaraConverseModelConfiguration\r\nThis enum class configures the model parameters.\r\n\r\n```python\r\nclass JaraConverseModelConfiguration(Enum):\r\n    MAX_MODEL_INPUT_SIZE: int = 512\r\n    MAX_MODEL_OUTPUT_SIZE: int = 512\r\n    MAX_POSITIONAL_ENCODING_LENGTH: int = MAX_MODEL_OUTPUT_SIZE + 50\r\n\r\n    NUMBER_OF_LAYERS: int = 6\r\n    DIMENSIONALITY_OF_MODEL_EMBEDDINGS: int = 212\r\n    FF_DIMENSION: int = 212\r\n\r\n    NUM_OF_HEADS: int = 8\r\n    LEARNING_DROPOUT_RATE: float = 0.001\r\n    IS_FIXED_LEARNING_RATE: bool = False\r\n    FIXED_LEARNING_RATE: float = 2.5e-5\r\n\r\n    MODEL_EPOCHS: int = 2\r\n    MODEL_EARLY_STOPPING_PATIENCE: int = 5\r\n\r\n    ADAM_SCHEDULER_WARMUP_STEPS: int = 4000\r\n    ADAM_OPTIMIZER_BETA_1: float = .9\r\n    ADAM_OPTIMIZER_BETA_2: float = .98\r\n\r\n    ADAM_OPTIMIZER_EPSILON: float = 1e-9\r\n\r\n    GRADIENT_ACCUMULATION_STEPS = 4\r\n```\r\n\r\n#### AutoCalculateModelParams\r\nThis class automatically calculates certain model parameters based on configurations.\r\n\r\n```python\r\nclass AutoCalculateModelParams(object):\r\n    STEP_PER_TRAINING_EPOC: int = TransformersTokenizerConfiguration.TRAINING_BATCH_SIZE.value\r\n    STEP_PER_VALIDATION_EPOC: int = TransformersTokenizerConfiguration.VALIDATION_BATCH_SIZE.value\r\n```\r\n\r\n\r\n# Contributor\r\n\r\n\u003ca href = \"https://github.com/Anonym0usWork1221/JaraConverse-TransformersBased/graphs/contributors\"\u003e\r\n  \u003cimg src = \"https://contrib.rocks/image?repo=Anonym0usWork1221/JaraConverse-TransformersBased\"/\u003e\r\n\u003c/a\u003e\r\n\r\n-----------\r\nSupport and Contact Information\r\n----------\r\n\u003e If you require any assistance or have questions, please feel free to reach out to me through the following channels:  \r\n* **Email**: `abdulmoez123456789@gmail.com`\r\n\r\n\u003e I have also established a dedicated Discord group for more interactive communication:  \r\n* **Discord Server**: `https://discord.gg/RMNcqzmt9f`\r\n\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanonym0uswork1221%2Fjaraconverse-transformersbased","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanonym0uswork1221%2Fjaraconverse-transformersbased","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanonym0uswork1221%2Fjaraconverse-transformersbased/lists"}