{"id":18907737,"url":"https://github.com/infraspecdev/bentoml-template","last_synced_at":"2025-04-15T04:32:05.978Z","repository":{"id":261254792,"uuid":"843310772","full_name":"infraspecdev/bentoml-template","owner":"infraspecdev","description":"A bentoml template to take a ML model to production","archived":false,"fork":false,"pushed_at":"2025-02-03T07:49:43.000Z","size":72,"stargazers_count":7,"open_issues_count":5,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-28T16:43:41.425Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/infraspecdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-16T08:31:01.000Z","updated_at":"2024-11-12T12:40:49.000Z","dependencies_parsed_at":"2024-11-05T15:41:54.985Z","dependency_job_id":null,"html_url":"https://github.com/infraspecdev/bentoml-template","commit_stats":null,"previous_names":["infraspecdev/bentoml-template"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infraspecdev%2Fbentoml-template","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infraspecdev%2Fbentoml-template/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infraspecdev%2Fbentoml-template/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/infraspecdev%2Fbentoml-template/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/infraspecdev","download_url":"https://codeload.github.com/infraspecdev/bentoml-template/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249006540,"owners_count":21197293,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T09:22:59.252Z","updated_at":"2025-04-15T04:32:04.598Z","avatar_url":"https://github.com/infraspecdev.png","language":"Python","readme":"# BentoML Template\n\nThis repository contains the BentoML template for model API deployments.\n\n## Prerequisites\n\nBefore you begin, ensure you have met the following requirements:\n\n1. [install python](https://www.python.org/downloads/)\n2. [install pip](https://pip.pypa.io/en/stable/installation/)\n3. [install aws](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and login to your account.\n4. A model stored in a S3 bucket in `.pickle` format.\n\n## Usage\n\nTo use this template, clone the repository and customize it according to your model's requirements. Below is a quick start guide:\n\n1. **Clone the repository:**\n\n   ```sh\n   git clone https://github.com/infraspecdev/bentoml-template.git\n   cd bentoml-template\n   ```\n2. **Customize the template:**\n\n   - Update `config.ini` to download the correct models from your S3 bucket.\n   - Update `validations.py` to change the input validation for your model.\n   - Update `service.py` to use your model.\n   - Update `bentofile.yaml` If you have changed the service name in `service.py`.\n3. **Run these commands to create a python environment:**\n\n   ```bash\n   python3 -m venv .venv\n   source .venv/bin/activate\n   ```\n4. **Run these commands to install all the dependencies:**\n\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n   **Note:** If you want to use iris model run the following command to store the model in `/models`\n   ```base\n   python train_and_save_model.py\n   ```\n\n5. **Run these commands to start the service:**\n\n   ```bash\n   bentoml serve .\n   ```\n\n   This will start a local server at [http://localhost:3000](http://localhost:3000/)\n\n## To run the model in docker or podman\n\n1. **Build the bento model:**\n\n   ```bash\n   bentoml build .\n   ```\n2. **Build the image:**\n\n   ```bash\n   bentoml containerize iris_classifier_service:\u003cBUILD_VERSION\u003e\n   ```\n3. **Run the container:**\n\n   ```bash\n   docker run --rm -p 8080:8080 iris_classifier_service:\u003cBUILD_VERSION\u003e\n   ```\n\nThe build version will be provided as the output of the `bentoml build` command. This will look something similar\nto `IrisClassifierService:nftm2tqyagzp4mtu`. In this example, `nftm2tqyagzp4mtu` is the build\nversion. For this quickstart example, the name is `IrisClassifierService`, but you need to replace it with the name of your service class.\n**Note:** Update the envs in the bentofile.yaml\n\n```yaml\nenvs:\n  - name: \u003cENV_VARIABLE_NAME\u003e\n    value: \u003cVALUE\u003e\n```\n\n## Environment Variables\n\n### To configure env variables\n\n1. Create a `.env` file with values similar to the given `.env.example` file.\n2. Change the values in the `.env` according to your requirement.\n\n### Details about the environment variables:\n\n- **BENTOML_PORT:** Port on which the BentoML service will run.\n- **JWT_SECRET:** Secret key used for signing JWT tokens. This should be a secure, randomly generated string.\n- **JWT_EXPIRATION_MINUTES:** Duration (in minutes) for which the JWT token remains valid.\n- **ENVIRONMENT:** Environment in which the service is running. Can be set to `development`, `staging`, or `production`.\n- **LOG_LEVEL:** Logging level for the application. Can be set to `DEBUG`, `INFO`, `WARNING`, `ERROR`, or `CRITICAL`.\n  Default is `WARNING`.\n\n## Download Models\n\nTo download the models from S3, update the bucket name and directory in the `configs/config.ini` file.\n\n1. Replace `YOUR_S3_BUCKET_NAME` with the actual s3 bucket name.\n2. Replace `YOUR_S3_BUCKET_DIRECTORY` with the directory path of the models in the s3.\n\nThe models can be downloaded by running:\n\n```bash\npython3 download_models.py\n```\n\nOn a local machine, this will require AWS secret and access keys to download from S3.\n\n## Updating `service.py` file\n\nTo deploy your specific model API using the provided BentoML template, you can follow the following points:\n\n1. **Download or Import the Required Model and Libraries:** Replace the current file name for the model If you want to run a custom model or use import to import the model\n\n```python\nwith open(\"./models/\u003cYOUR_MODEL_FILE_NAME\u003e\", \"rb\") as model_file:\n    model = pickle.load(model_file)\n```\n\n2. **Update the class name:** Change the class name to your model specific class name.\n\n```python\nclass WeatherPrediciton\n```\n\nYou will need to update the class name in the`bentofile.yaml` also,\n```yaml     \nservice: \"service:WeatherPrediciton\"\n```\n\n3. **Modify the API Route:** Update the API route to match the model's endpoint.\n\n```python\n@bentoml.api(route='/api/v1/analyze')\n```\n\nYou will need to update the routes in following files as well.\n\n```python\n# middleware/validate_jwt.py\nprotected_routes = ['/api/v1/analyze']\n\n# middlewares/request_response_handler.py\nroutes_to_log = [\"/api/v1/analyze\"]\n\n# middlewares/validation_handler.py\nroutes_to_validate = ['/api/v1/analyze']\n\n#utils/common/validations.py\nreturn {\"/api/v1/analyze\": WeatherPredicitonParams}\n```\n\n4. **Update the Request Validation Schema:** Update the input validation(Params) in `utils/common/validations.py\n\n```python\nclass IrisRequestParams(BaseModel):\n    sepal_length: float = Field(description=\"Sepal length in cm\", gt=0)\n    sepal_width: float = Field(description=\"Sepal width in cm\", gt=0)\n    petal_length: float = Field(description=\"Petal length in cm\", gt=0)\n    petal_width: float = Field(description=\"Petal width in cm\", gt=0)\n```\n\nThe import statements, class name, route, inference logic, and middleware will be based on your specific use-case for model API development. The examples provided above are for reference only to give you an idea of the components you need to change.\n\nIn this example, we used a iris classifier model. If you are using a different\nmodel or framework, these components might need to be completely different. You should code your class based on your specific requirements and the model you are using.\n\n## JWT Authentication\n\nIn the quickstart sample, the `/api/v1/predict` endpoint requires the JWT token in the request authorization headers\nto authenticate the request. If any other route which needs to be authenticated before serving the request, add the\nendpoint in the `protected_routes` list in `middlewares/validate_jwt.py`.\n\nTo add more protected routes, update the `protected_routes` list:\n\n```python\n# middlewares/validate_jwt.py\nprotected_routes = [\n\t'/api/v1/predict',\n\t'/api/v1/analyze'\n]\n```\n\n## Example curl request\n\n```bash\ncurl -X 'POST' \\\n  'http://localhost:\u003cBENTOML_PORT\u003e/api/v1/predict' \\\n  -H 'accept: application/json' \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: \u003cJWT_TOKEN\u003e'  \\\n  -d '{\n  \"sepal_length\": 1,\n  \"sepal_width\": 2,\n  \"petal_length\": 3,\n  \"petal_width\": 4\n}'\n```\n\nReplace ` \u003cBENTOML_PORT\u003e` and `\u003cJWT_TOKEN\u003e` with their values. Change `api/v1/predict` and the request body with the new service route and new body if updated.\n**To generate the JWT token:**\n\n```bash\n   python3 utils/jwt/generate_token.py\n```\n\nYou can change the token expiry and secret by changing the environment variables `JWT_EXPIRATION_MINUTES`\nand `JWT_SECRET` in the `.env` file.\n\nYou can run this in your terminal to generate a `JWT_SECRET`.\n\n```bash\ndate | base64 | base64\n```\n\n## Logging\n\n[Structure Logging](utils/structure_logging/README.md#structure-logging)\n\n## Monitoring\n\n[Prometheus Metrics](utils/monitoring/README.md#prometheus-metrics)\n\n## DynamoDB Setup\n\n[Local DynamoDB Setup with Python](utils/dynamodb/README.md)\n\n## CI/CD Setup\n\n1. [Build Container Image](docs/github_workflows/build.md)\n2. [Deploy to staging/production](docs/github_workflows/deploy.md)\n3. [Test](docs/github_workflows/test.md)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finfraspecdev%2Fbentoml-template","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finfraspecdev%2Fbentoml-template","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finfraspecdev%2Fbentoml-template/lists"}