{"id":31482534,"url":"https://github.com/snowflake-labs/icetire","last_synced_at":"2025-10-02T07:54:50.887Z","repository":{"id":45537753,"uuid":"425079667","full_name":"Snowflake-Labs/icetire","owner":"Snowflake-Labs","description":"Data Science Sandbox for Snowflake","archived":false,"fork":false,"pushed_at":"2023-01-27T01:07:10.000Z","size":47,"stargazers_count":11,"open_issues_count":2,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2023-11-02T19:24:09.350Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Snowflake-Labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-05T20:44:16.000Z","updated_at":"2023-11-01T19:53:05.000Z","dependencies_parsed_at":"2023-02-15T03:35:25.727Z","dependency_job_id":null,"html_url":"https://github.com/Snowflake-Labs/icetire","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"purl":"pkg:github/Snowflake-Labs/icetire","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snowflake-Labs%2Ficetire","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snowflake-Labs%2Ficetire/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snowflake-Labs%2Ficetire/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snowflake-Labs%2Ficetire/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Snowflake-Labs","download_url":"https://codeload.github.com/Snowflake-Labs/icetire/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Snowflake-Labs%2Ficetire/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277974435,"owners_count":25908396,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-02T02:00:08.890Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-02T07:54:46.591Z","updated_at":"2025-10-02T07:54:50.869Z","avatar_url":"https://github.com/Snowflake-Labs.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Introduction\n\nIcetire is a docker image which aims to provide Snowflake users with a turn key docker environment already set-up with Snowflake drivers of the version of your choice with a comprehensive data science environment including Jupyter Notebooks, Python, Spark, R to experiment the various Snowflake connectors available.\n\nIcetire supports Spark 3.3 with Scala 2.12, as well as adds support for the Snowpark for Scala, and Snowpark for Python API. The following drivers and connectors for Snowflake are provided:\n\n- ODBC\n- JDBC\n- Python Connector\n- Spark Connector\n- SnowSQL Client\n\n\nSQL Alchemy python package is also installed as part of this docker image.\n\nThe base docker image is [Jupyter Docker Stacks](https://github.com/jupyter/docker-stacks). More specifically, the image used is [jupyter/all-spark-notebook](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-all-spark-notebook) which provides a comprehensive jupyter environment including r, sci-py, pyspark and scala.\n\nPlease review the [licensing terms](https://raw.githubusercontent.com/jupyter/docker-stacks/master/LICENSE.md) of the above mentioned project.\n\n**NOTE: Icetire is not officially supported by Snowflake, and is provided as-is.**\n\n# Prerequisites\n\n- You need git on your Mac or Windows\n- You need to download and install [Docker Desktop for Mac](https://hub.docker.com/editions/community/docker-ce-desktop-mac) or [Docker Desktop for Windows](https://hub.docker.com/editions/community/docker-ce-desktop-windows). You may need to create an account on Docker to be able to download it.\n\n**NOTE FOR WINDOWS**\n\nOn Windows, a common issue encountered is the configuration of line endings, which adds default CRLF Windows line endings to the script deploy_snowflake.sh causing the script to fail. You can either configure [```core.autocrlf```](https://docs.github.com/en/free-pro-team@latest/github/using-git/configuring-git-to-handle-line-endings#refreshing-a-repository-after-changing-line-endings) to ```false```, or use an editor like Notepad+++ to open the deploy_snowflake.sh file and save it in UNIX mode which will convert CRLF line endings into LF before you build the Icetire docker image.\n\n\n# Instructions\n\n## Download the repository\n\nChange the Directory to a location where you are storing your Docker images:\n\n```\nmkdir DockerImages\ncd DockerImages\ngit clone https://github.com/Snowflake-Labs/icetire.git\ncd icetire/\n```\n\nIf you are just updating the repository to the latest version (always recommended before building a docker image). Run the following command from within your local clone (under Icetire directory):\n\n```\ngit pull\n\n```\n\n## Build Icetire docker image\n\nThere are two ways to build the image, either with the default levels, or by specifying actual driver levels while building.\n\n### Default drivers, connectors and scala kernel levels\n\nThe default levels are specified in the Docker file (lines starting with ARG). These levels are refreshed on a best level basis, and have been sanity tested. If you are satisfied with these levels, you can simply run the following command:\n\n```\ndocker build --pull -t icetire .\n```\n\n### Specify the driver levels while building\n\nFirst check the latest clients available in the official [Snowflake documentation](https://docs.snowflake.net/manuals/release-notes/client-change-log.html#client-changes-by-version)\n\nOnce you have chosen the versions, you can pass the different versions as arguments in the docker build command:\n\n```\ndocker build --pull -t icetire . \\\n--build-arg odbc_version=2.21.8 \\\n--build-arg jdbc_version=3.12.10 \\\n--build-arg spark_version=2.8.1-spark_2.4 \\\n--build-arg snowsql_version=1.2.9 \\\n--build-arg almond_version=0.10.0 \\\n--build-arg scala_version=2.12.11\n```\n\n**NOTE: SnowSQL CLI has the ability to [auto-upgrade](https://docs.snowflake.net/manuals/user-guide/snowsql-install-config.html#label-understanding-auto-upgrades) to the latest version available. So, you may not need to specify a higher version.**\n\n### Build completion\n\nYou should see the following message at the very end:\n```\nSuccessfully tagged icetire:latest\n```\n\n## Running the image\n```\ndocker run -p 8888:8888 --name spare-0 icetire:latest\n```\nIf the port 8888 is already taken on your laptop, and you want to use another port, you can simply change the port mapping. For example, for port 9999, it would be:\n```\ndocker run -p 9999:8888 --name spare-1 icetire:latest\n```\n\nYou should see a message like the following the very first time you bring up this image. Copy the token value in the URL:\n```\n[I 23:33:42.828 NotebookApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret\n[I 23:33:43.820 NotebookApp] JupyterLab extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlab\n[I 23:33:43.820 NotebookApp] JupyterLab application directory is /opt/conda/share/jupyter/lab\n[I 23:33:43.822 NotebookApp] Serving notebooks from local directory: /home/jovyan\n[I 23:33:43.822 NotebookApp] The Jupyter Notebook is running at:\n[I 23:33:43.822 NotebookApp] http://(a8e53cbad3a0 or 127.0.0.1):8888/?token=eb2222f1a8cd14046ecc5177d4b1b5965446e3c34b8f42ad\n[I 23:33:43.822 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).\n[C 23:33:43.826 NotebookApp]\n\n    To access the notebook, open this file in a browser:\n        file:///home/jovyan/.local/share/jupyter/runtime/nbserver-17-open.html\n    Or copy and paste one of these URLs:\n        http://(a8e53cbad3a0 or 127.0.0.1):8888/?token=eb2222f1a8cd14046ecc5177d4b1b5965446e3c34b8f42ad\n```\n\n**Note:** If you are restarting the image, and you need to retrieve the token you can retrieve it as following:\n\n  - Retrieve its value from the docker logs as following:\n\n  ```\n  docker logs spare-0 --tail 10\n  ```\n\n - Open a bash session on the docker container\n\n ```\n docker exec -it spare-0 /bin/bash\n ```\n\n Then run the following command:\n\n ```\n jupyter notebook list\n ```\n\n## Accessing the image\n\nOpen a web browser on: http://localhost:8888\n\nIt will prompt you for a Password or token. Enter the token you have in the previous message.\n\n## Working with the image\n\nIcetire come with 4 different small examples of python notebooks allowing to test various connectors including odbc, jdbc, spark. You will need to customize your Snowflake account name, your credentials (user/password), database name and warehouse.\n\nYou can always upload to the jupyter environment any demo notebook from the main interface. See the Upload button at the top right:\n\n![Image](https://github.com/zoharsan/snowflake-jupyter-extras/blob/master/Notebooks.png)\n\nThese notebooks can work with the tpch_sf1 database which is provided as a sample within any Snowflake environment.\n\nIf you plan to develop new notebooks within the Docker environment, in order to avoid losing any work due to a Docker container discarded accidentally or any other container corruption, it is recommended to always keep a local copy of your work once you are done. This can be done in the Jupyter menu: File-\u003eDownload as.\n\n### Stopping and starting the docker image\n\nOnce finished, you can stop the image with the following command:\n```\ndocker stop spare-0\n```\nIf you want to resume work, you can start the image with the following command:\n```\ndocker start spare-0\n```\n\n### Additional handy commands\n\n- To delete the container. WARNING: If you do this, you will lose any notebook, and any work you have saved or done within the container.\n```\ndocker rm spare-0\n```\n- To open a bash session on the docker container, which will be useful to use the snowsql interface:\n```\ndocker exec -it spare-0 /bin/bash\n```\n- To copy files in the docker container:\n```\ndocker cp \u003cabsolute-file-name\u003e spare-0:\u003cabsolute-path-name in the container\u003e\nExample: docker cp README.md spare-0:/\n```\n- To list all docker containers available:\n```\ndocker ps -a\n```\n- To list all docker images available:\n```\ndocker image ls\n```\n- To delete a docker image:\n```\ndocker image rm \u003cimage-id\u003e\n```\nYou can find out the image id in the previous list command.\n\n### Known Issues \u0026 Troubleshooting\n\n---\n#### Stability: Notebook hangs or crashes on large data sets ####\n\nMake sure you have enough memory allocated for your Docker Workstation, at least 4 GB. On Mac:\n\n- Stop all your docker images (See instructions above to stop/start docker images).\n- Click on the Docker Icon on the top right hand side of your Mac Menu bar.\n- Select Preferences\n- Select Resources\n- Set CPUs to minimum of 2.\n- Set Memory to 4 GB.\n- Click on Apply \u0026 Restart.\n\n---\n#### Failure to connect to Snowflake when Snowflake URI has underscores \"_\" ####\n\nDue to a [JDK Bug](https://bugs.openjdk.java.net/browse/JDK-8221675) in JDK 8, JDBC connection in Spark or Snowpark may fail with the following stack:\n\n```\nnet.snowflake.client.jdbc.SnowflakeSQLException: JDBC driver encountered communication error. Message: Exception encountered for HTTP request: Remote host terminated the handshake.\nnet.snowflake.client.jdbc.RestRequest.execute(RestRequest.java:284)\nnet.snowflake.client.core.HttpUtil.executeRequestInternal(HttpUtil.java:496)\nnet.snowflake.client.core.HttpUtil.executeRequest(HttpUtil.java:441)\nnet.snowflake.client.core.HttpUtil.executeGeneralRequest(HttpUtil.java:408)\nnet.snowflake.client.core.SessionUtil.newSession(SessionUtil.java:574)\nnet.snowflake.client.core.SessionUtil.openSession(SessionUtil.java:279)\nnet.snowflake.client.core.SFSession.open(SFSession.java:435)\nnet.snowflake.client.jdbc.DefaultSFConnectionHandler.initialize(DefaultSFConnectionHandler.java:103)\nnet.snowflake.client.jdbc.DefaultSFConnectionHandler.initializeConnection(DefaultSFConnectionHandler.java:79)\nnet.snowflake.client.jdbc.SnowflakeConnectionV1.initConnectionWithImpl(SnowflakeConnectionV1.java:116)\nnet.snowflake.client.jdbc.SnowflakeConnectionV1.\u003cinit\u003e(SnowflakeConnectionV1.java:96)\ncom.snowflake.snowpark.internal.ServerConnection.$anonfun$connection$1(ServerConnection.scala:132)\nscala.Option.getOrElse(Option.scala:189)\ncom.snowflake.snowpark.internal.ServerConnection.\u003cinit\u003e(ServerConnection.scala:130)\ncom.snowflake.snowpark.internal.ServerConnection$.apply(ServerConnection.scala:23)\ncom.snowflake.snowpark.Session$SessionBuilder.createInternal(Session.scala:896)\ncom.snowflake.snowpark.Session$SessionBuilder.create(Session.scala:884)\nammonite.$sess.cmd2$Helper.\u003cinit\u003e(cmd2.sc:6)\nammonite.$sess.cmd2$.\u003cinit\u003e(cmd2.sc:7)\nammonite.$sess.cmd2$.\u003cclinit\u003e(cmd2.sc:-1)\njavax.net.ssl.SSLHandshakeException: Remote host terminated the handshake\nsun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1310)\nsun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1151)\nsun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1054)\nsun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:394)\nnet.snowflake.client.jdbc.internal.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)\nnet.snowflake.client.jdbc.internal.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)\nnet.snowflake.client.jdbc.internal.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)\nnet.snowflake.client.jdbc.internal.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)\nnet.snowflake.client.jdbc.internal.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)\nnet.snowflake.client.jdbc.internal.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)\nnet.snowflake.client.jdbc.internal.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)\nnet.snowflake.client.jdbc.internal.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)\nnet.snowflake.client.jdbc.internal.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)\nnet.snowflake.client.jdbc.internal.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)\nnet.snowflake.client.jdbc.internal.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)\nnet.snowflake.client.jdbc.internal.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)\nnet.snowflake.client.jdbc.RestRequest.execute(RestRequest.java:160)\nnet.snowflake.client.core.HttpUtil.executeRequestInternal(HttpUtil.java:496)\nnet.snowflake.client.core.HttpUtil.executeRequest(HttpUtil.java:441)\nnet.snowflake.client.core.HttpUtil.executeGeneralRequest(HttpUtil.java:408)\nnet.snowflake.client.core.SessionUtil.newSession(SessionUtil.java:574)\nnet.snowflake.client.core.SessionUtil.openSession(SessionUtil.java:279)\nnet.snowflake.client.core.SFSession.open(SFSession.java:435)\n```\n\nWhen using Snowpark, please upgrade to the latest Docker container (Updated May 19 2021) which upgrades the JDK to JDK 11. However, Spark 2.4 version doesn't support JDK 11 and is using JDK 8 within the docker. In order to circumvent this issue, replace underscores with dashes as follows:\n\nIt may fail if your URL is:\n```https://ca_acme.ca-central-1.aws.snowflakecomputing.com/```\nYou can replace it with to make it work:\n```https://ca-acme.ca-central-1.aws.snowflakecomputing.com/```\n\nThis workaround is also valid with Snowpark.\n\n#### Spark Notebook fail when using JDK 11 ####\n\nThe May 19 update of Icetire installs JDK 11, which is incompatible with Spark 2.4. You may see the following stack when trying to run your spark notebooks:\n\n```\ny4JJavaError: An error occurred while calling o84.showString.\n: java.lang.IllegalArgumentException: Unsupported class file major version 55\n\tat org.apache.xbean.asm6.ClassReader.\u003cinit\u003e(ClassReader.java:166)\n\tat org.apache.xbean.asm6.ClassReader.\u003cinit\u003e(ClassReader.java:148)\n\tat org.apache.xbean.asm6.ClassReader.\u003cinit\u003e(ClassReader.java:136)\n\tat org.apache.xbean.asm6.ClassReader.\u003cinit\u003e(ClassReader.java:237)\n\tat org.apache.spark.util.ClosureCleaner$.getClassReader(ClosureCleaner.scala:49)\n\tat org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.scala:517)\n\tat org.apache.spark.util.FieldAccessFinder$$anon$3$$anonfun$visitMethodInsn$2.apply(ClosureCleaner.scala:500)\n\tat scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)\n\tat scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:134)\n\tat scala.collection.mutable.HashMap$$anon$1$$anonfun$foreach$2.apply(HashMap.scala:134)\n\tat scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)\n\tat scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)\n\tat scala.collection.mutable.HashMap$$anon$1.foreach(HashMap.scala:134)\n```\n\nIf you see this error, you need to switch the default JDK to JDK 8 using the following command after open a bash session on your container (See instructions above in 'Additional Handy Commands' section):\n\n```\nroot@91875ce97fb6:~/work/snowpark# sudo update-alternatives --config java\n  There are 2 choices for the alternative java (providing /usr/bin/java).\n\n    Selection    Path                                            Priority   Status\n  ------------------------------------------------------------\n    0            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1111      auto mode\n  * 1            /usr/lib/jvm/java-11-openjdk-amd64/bin/java      1111      manual mode\n    2            /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java   1081      manual mode\n\n  Press \u003center\u003e to keep the current choice[*], or type selection number: 2\n```\n\nAfter doing this, restart the kernel for your notebook to pick-up the JDK 8 and it should work.\n\n#### Python Kernel Dying ####\n\nIn case you have the Python kernel dying while running the notebook, and you want to troubleshoot the root cause, please add these lines as your first paragraph of your notebook and execute the paragraph:\n```\n# Debugging\n\nimport logging\nimport os\n\nfor logger_name in ['snowflake','botocore','azure']:\n    logger = logging.getLogger(logger_name)\n    logger.setLevel(logging.DEBUG)\n    ch = logging.FileHandler('python_connector.log')\n    ch.setLevel(logging.DEBUG)\n    ch.setFormatter(logging.Formatter('%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))\n    logger.addHandler(ch)\n```\nThis will generate a python_connector.log file where the notebook resides. Use the commands above to ssh into the image and examine the log.\n\n---\n#### Building the Docker Image fails on Windows ####\n\nBuilding the docker image fails on Windows with the following errors:\n\n```\nStep 14/24 : RUN odbc_version=2.21.8 jdbc_version=3.12.10 spark_version=2.8.1-spark_2.4 snowsql_version=1.2.9 /deploy_snowflake.sh\n ---\u003e Running in 0cfd230c3949\n: not foundwflake.sh: 11: /deploy_snowflake.sh:\n: not foundwflake.sh: 13: /deploy_snowflake.sh:\n/deploy_snowflake.sh: 24: cd: can't cd to /\n: not foundwflake.sh: 25: /deploy_snowflake.sh:\n ...loading odbc driver version 2.21.8\ncurl: (3) URL using bad/illegal format or missing URL\n ...loading jdbc driver version 3.12.10\n: not foundwflake.sh: 28: /deploy_snowflake.sh:\ncurl: (3) URL using bad/illegal format or missing URL\n ...loading spark driver version 2.8.1-spark_2.4\n: not foundwflake.sh: 31: /deploy_snowflake.sh:\ncurl: (3) URL using bad/illegal format or missing URL\n ...load SnowSQL client version 1.2.9\n...\n```\nThis is caused by Windows CRLF line ending special characters added to the deploy_snowflake.sh script causing the script to fail in the Linux Ubuntu container. Open the deploy_snowflake.sh with an Editor like Notepad++ and save the file in UNIX mode which will convert CRLF to LF line endings and rerun the docker build command.\n\n\n---\n#### Known Issues Log ####\n\nPlease check [known issues](known_issues.md) log for known issues with Icetire.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnowflake-labs%2Ficetire","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnowflake-labs%2Ficetire","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnowflake-labs%2Ficetire/lists"}