{"id":47944230,"url":"https://github.com/dataiku/dss-plugin-snowpark-container-services-llm","last_synced_at":"2026-04-04T08:20:49.588Z","repository":{"id":229202741,"uuid":"733563790","full_name":"dataiku/dss-plugin-snowpark-container-services-llm","owner":"dataiku","description":null,"archived":false,"fork":false,"pushed_at":"2024-04-30T20:11:52.000Z","size":126,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-07-10T11:58:19.963Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dataiku.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-19T15:57:57.000Z","updated_at":"2024-04-30T20:11:55.000Z","dependencies_parsed_at":"2024-04-30T21:48:36.094Z","dependency_job_id":null,"html_url":"https://github.com/dataiku/dss-plugin-snowpark-container-services-llm","commit_stats":null,"previous_names":["dataiku/dss-plugin-snowpark-container-services-llm"],"tags_count":0,"template":false,"template_full_name":"dataiku/dss-plugin-template","purl":"pkg:github/dataiku/dss-plugin-snowpark-container-services-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataiku%2Fdss-plugin-snowpark-container-services-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataiku%2Fdss-plugin-snowpark-container-services-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataiku%2Fdss-plugin-snowpark-container-services-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataiku%2Fdss-plugin-snowpark-container-services-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dataiku","download_url":"https://codeload.github.com/dataiku/dss-plugin-snowpark-container-services-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataiku%2Fdss-plugin-snowpark-container-services-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31392740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T04:26:24.776Z","status":"ssl_error","status_checked_at":"2026-04-04T04:23:34.147Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-04T08:20:48.999Z","updated_at":"2026-04-04T08:20:49.566Z","avatar_url":"https://github.com/dataiku.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Snowflake Snowpark Container Services LLM Plugin\n\nWith this plugin, you can leverage LLMs hosted in Snowpark Container Services (SPCS) as part of the LLM Mesh.\n\n![Screenshot 2024-02-09 at 4 58 15 PM](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/155dc4dd-148f-484a-a6c5-6643a52b2b2c)\n\n# Capabilities\n\n- Custom LLM connection option\n- Connect to a chat completion endpoint hosted in SPCS to use in Dataiku Prompt Studios, LLM recipes, and via the LLM Mesh python APIs\n- Connect to an embedding endpoint hosted in SPCS to use in Dataiku Embed recipe for Retrieval Augmented Generation (RAG)\n\n# Limitations\n\n- Must use Dataiku \u003e= v12.5.2\n- Must use Snowflake Oauth to connect to SPCS\n- Must deploy and maintain one’s own LLMs on SPCS (we have sample code below to help)\n\n# Setup\n## Setup resources on the Snowflake side\n1. Ensure that Snowpark Container Services (including SPCS jobs) is activated in your Snowflake account (ask your Snowflake rep)\n\n2. (In Snowflake as ACCOUNTADMIN) Run the below code to:\n\ta. Create a role with access to SCPS and grant this role to users\n\tb. Create two SPCS compute pools (for the main LLM and embedding LLM)\n\tc. Create a warehouse to run simple admin queries (or you can grant access to an existing warehouse to the DATAIKU_SPCS_ROLE)\n\td. Create a database and schema to hold the LLM model registry\n\te. Create a Snowflake OAUTH security integration  \n\n``` sql\n-- Start of Snowflake SQL instructions\n-- We'll need the ACCOUNTADMIN role to create these roles and compute pools\nUSE ROLE ACCOUNTADMIN;\n\n-- First create a network rule and external access integration. We'll store them in a dedicated DB/Schema\nCREATE DATABASE NETWORK_DB;\nCREATE SCHEMA NETWORK_SCHEMA;\n\nCREATE OR REPLACE NETWORK RULE SNOWFLAKE_EGRESS_ACCESS\n  MODE = EGRESS\n  TYPE = HOST_PORT\n  VALUE_LIST = ('0.0.0.0:443','0.0.0.0:80');\n\nCREATE EXTERNAL ACCESS INTEGRATION SNOWFLAKE_EGRESS_ACCESS_INTEGRATION\n  ALLOWED_NETWORK_RULES = (SNOWFLAKE_EGRESS_ACCESS)\n  ENABLED = true;\n\n-- create a new oauth integration\nCREATE SECURITY INTEGRATION OAUTH_DATAIKU_SPCS\n  TYPE = OAUTH\n  ENABLED = TRUE\n  OAUTH_CLIENT = CUSTOM\n  OAUTH_CLIENT_TYPE = 'CONFIDENTIAL'\n  OAUTH_REDIRECT_URI = 'https://\u003cYOUR_DATAIKU_INSTANCE_URL\u003e/dip/api/oauth2-callback'\n  OAUTH_ISSUE_REFRESH_TOKENS = TRUE\n  OAUTH_REFRESH_TOKEN_VALIDITY = 7776000\n  BLOCKED_ROLES_LIST = ('SYSADMIN');\n\n-- get the OAUTH_CLIENT_ID and OAUTH_CLIENT_SECRET\nCALL SYSTEM$SHOW_OAUTH_CLIENT_SECRETS('OAUTH_DATAIKU_SPCS');\n\n-- Create a role for Dataiku users to use when calling LLMs on SPCS\nCREATE ROLE DATAIKU_SPCS_ROLE;\nGRANT ROLE DATAIKU_SPCS_ROLE TO USER \"\u003cSNOWFLAKE_USER_TO_DEPLOY_AND_USE_MODELS\u003e\";\n\n-- Grant access to the external access integration to the new role\nGRANT USAGE ON INTEGRATION SNOWFLAKE_EGRESS_ACCESS_INTEGRATION TO ROLE DATAIKU_SPCS_ROLE;\n\n-- Create a compute pool with GPU resources to host the LLM for chat completion. GPU_3 is the smallest type with GPUs\n-- Grant usage of this compute pool to the DATAIKU_SPCS_ROLE\nCREATE COMPUTE POOL DATAIKU_GPU_NV_S_MODEL_COMPUTE_POOL with instance_family=GPU_NV_S min_nodes=1 max_nodes=1;\nGRANT USAGE ON COMPUTE POOL DATAIKU_GPU_NV_S_MODEL_COMPUTE_POOL to role DATAIKU_SPCS_ROLE;\nGRANT MONITOR ON COMPUTE POOL DATAIKU_GPU_NV_S_MODEL_COMPUTE_POOL to role DATAIKU_SPCS_ROLE;\n\nGRANT BIND SERVICE ENDPOINT on ACCOUNT to DATAIKU_SPCS_ROLE;\n\n-- Create a compute pool with CPU resources to host the LLM for sentence embeddings. STANDARD_1 is the smallest type\n-- Grant usage of this compute pool to the DATAIKU_SPCS_ROLE\nCREATE COMPUTE POOL DATAIKU_CPU_X64_XS_EMBED_COMPUTE_POOL with instance_family=CPU_X64_XS min_nodes=1 max_nodes=1;\nGRANT USAGE ON COMPUTE POOL DATAIKU_CPU_X64_XS_EMBED_COMPUTE_POOL to role DATAIKU_SPCS_ROLE;\nGRANT MONITOR ON COMPUTE POOL DATAIKU_CPU_X64_XS_EMBED_COMPUTE_POOL to role DATAIKU_SPCS_ROLE;\n\nCREATE OR REPLACE WAREHOUSE DATAIKU_SPCS_WAREHOUSE WITH\n  WAREHOUSE_SIZE='X-SMALL'\n  AUTO_SUSPEND = 60\n  AUTO_RESUME = true\n  INITIALLY_SUSPENDED=false;\n  \nGRANT ALL ON WAREHOUSE DATAIKU_SPCS_WAREHOUSE TO ROLE DATAIKU_SPCS_ROLE;\n\nCREATE DATABASE DATAIKU_SPCS;\nGRANT OWNERSHIP ON DATABASE DATAIKU_SPCS TO ROLE DATAIKU_SPCS_ROLE;\n\nUSE ROLE DATAIKU_SPCS_ROLE;\nUSE DATABASE DATAIKU_SPCS;\nUSE WAREHOUSE DATAIKU_SPCS_WAREHOUSE;\nCREATE SCHEMA MODEL_REGISTRY;\nUSE SCHEMA MODEL_REGISTRY;\n-- End of Snowflake SQL instructions\n```\n\n## Setup the plugin in Dataiku\n1. Install the plugin - Go to Plugins -\u003e Add Plugin -\u003e Fetch from Git repository, then enter this repo URL:\n\n![add_plugin_from_git_repo](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/72b5ec9a-e037-4d83-a6b4-e8fbf16018d5)\n\n![clone_plugin](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/a33f9925-6baf-4476-8695-2954056cf5b4)\n\n3. In the plugin settings, add a “Snowflake login with SSO” preset:\n\n![spcs_plugin_oauth_params](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/938ba63f-6713-4820-9462-9474f8ec6708)\n\n4. In the Admin -\u003e Connections page, create a new Snowflake connection called “spcs-access-only”. Enter the Snowflake DB, Schema, Warehouse, and Oauth info from the Snowflake-side setup. The “Auth authorization endpoint” should look like https://\u003cYOUR_SNOWFLAKE_ACCOUNT\u003e.aws.snowflakecomputing.com/oauth/authorize and the “Auth token endpoint” should look like https://\u003cYOUR_SNOWFLAKE_ACCOUNT\u003e.aws.snowflakecomputing.com/oauth/token-request. Uncheck “allow write” on the right in order to prevent users from creating datasets in this connection. We’ll use it to deploy models to SPCS only. Change the credentials mode to “per user”\n\n![spcs_access_connection](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/4fd6fa52-9b02-4479-af03-9d95c5f285bb)\n\n![spcs_access_connection_2](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/b99d299b-fa7d-4612-948a-228d9fdb1010)\n\n5. Go to you user profile credentials, and go through the Oauth dance for both the “spcs-access-only” connection and “snowpark-container-services-llm” plugin\n\n![spcs_user_oauth](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/5e5dc06b-a4ab-49c0-b319-f4e283265d41)\n![spcs_user_oauth_2](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/c832f93b-d5dc-498f-9916-e875c94e1217)\n\n\n6. Create a python 3.8 code environment (I named it “py_38_snowpark_llms”) and add the following packages:\n```\nscikit-learn==1.2.1\nmlflow==1.30.0\nstatsmodels==0.12.2\nprotobuf==3.16.0\nxgboost==1.7.3\nlightgbm==3.3.5\nmatplotlib==3.7.4\nscipy==1.10.1\nsnowflake-snowpark-python==1.12.1\nsnowflake-snowpark-python[pandas]\nsnowflake-connector-python[pandas]\nMarkupSafe\u003c2.1.0\ncloudpickle==2.0.0\nflask\u003e=1.0,\u003c1.1\nitsdangerous\u003c2.1.0\nJinja2\u003e=2.11,\u003c2.12\nsnowflake-ml-python==1.3.0\ndash==2.15.0\ndash_bootstrap_components==1.5.0\ntransformers==4.37.2\nsentence-transformers==2.3.1\ndatasets==2.16.1\ntorch\nsentencepiece==0.1.99\npresidio-anonymizer==2.2.352\npresidio_analyzer==2.2.352\nspacy==3.7.3\nlangchain==0.0.347\nbitsandbytes\u003e0.37.0\n```\n\n6. Deploy LLM(s) to SPCS, then retrieve the resulting public endpoints for chat completion and embedding models. We have a sample notebook [here](Deploy_LLMs_to_Snowpark_Container_Services.ipynb) that shows how to deploy, for chat completion: Zephyr 7B-beta, Llama2, Phi2, and Falcon; and for text embeddings: MiniLM-L6-v2.\n\nYour URLs should look something like: https://{ENDPOINT_ID}-{SNOWFLAKE_ACCOUNT_NAME/ACCOUNT_ID}.snowflakecomputing.app\n\n7. Create a new “Custom LLM” connection, then choose the “Snowpark Container Services LLM” plugin.\n\n![create_spcs_llm_connection](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/26b9b008-c4ab-4909-afc3-5f7717e56dc1)\n\nAdd a model for the main LLM, give it a name, chose the \"Chat completion\" capability, \"Snowpark Container Services LLM\" type, choose your Oauth preset, then enter the generated LLM endpoint URL from earlier. Enter your Snowflake account URL, the compute pool credit cost, and Snowflake credit cost (talk to your SNowflake rep for these). Max parallelism of queries is up to you. Start with 1 or 2.\n\n![chat_completion_spcs_model_in_connection](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/ed61fbcb-cecd-4c88-b7ca-80922321eab7)\n\n8. Add another model (in this same connection) for the text embedding model. Give it a name, choose the \"Text embedding\" capability, \"Snowpark Container Services LLM\" type, choose your Oauth preset, then enter the other generated LLM endpoint URL from the text embedding model. This model will likely have a different compute pool cost, depending on what you set up on the Snowflake side.\n\n![text_embedding_spcs_model_in_connection](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/c0ba3669-e4a6-473b-8f76-f91f0af691e7)\n\n10. Your chat completion LLM is now ready to use in LLM mesh!\n\n![spcs_chat_completion_model_prompt_studio](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/075a49b9-d8ed-4814-96ef-23b04b9aec54)\n\n12. As is your embedding LLM to generate embeddings for Retrieval Augmented Generation (RAG)!\n\n![spcs_text_embedding_model_knowledge_bank](https://github.com/dataiku/dss-plugin-snowpark-container-services-llm/assets/22987725/631fe96c-616f-4ea3-8c41-ff2c9ee5b98a)\n\n14. Here is sample SQL code to run from Snowflake to DROP your LLM services and compute pools:\n\n```sql\n--must drop a service before dropping a compute pool\nSHOW SERVICES;\nDROP SERVICE \u003cSERVICE_ID_FROM_ABOVE_RESULTS\u003e;\n\nDROP COMPUTE POOL DATAIKU_GPU_NV_S_MODEL_COMPUTE_POOL;\nDROP COMPUTE POOL DATAIKU_CPU_X64_XS_EMBED_COMPUTE_POOL;\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdataiku%2Fdss-plugin-snowpark-container-services-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdataiku%2Fdss-plugin-snowpark-container-services-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdataiku%2Fdss-plugin-snowpark-container-services-llm/lists"}