{"id":21045023,"url":"https://github.com/quansight/omnisci-datascience-installer","last_synced_at":"2025-06-25T08:05:21.937Z","repository":{"id":47051476,"uuid":"287605445","full_name":"Quansight/omnisci-datascience-installer","owner":"Quansight","description":"Installer configuration for OmniSci stack","archived":false,"fork":false,"pushed_at":"2021-09-27T12:30:09.000Z","size":97,"stargazers_count":3,"open_issues_count":3,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-06-22T23:52:27.693Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Quansight.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-14T19:05:53.000Z","updated_at":"2021-04-16T01:20:38.000Z","dependencies_parsed_at":"2022-08-23T17:31:30.472Z","dependency_job_id":null,"html_url":"https://github.com/Quansight/omnisci-datascience-installer","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/Quansight/omnisci-datascience-installer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quansight%2Fomnisci-datascience-installer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quansight%2Fomnisci-datascience-installer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quansight%2Fomnisci-datascience-installer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quansight%2Fomnisci-datascience-installer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Quansight","download_url":"https://codeload.github.com/Quansight/omnisci-datascience-installer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Quansight%2Fomnisci-datascience-installer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261832634,"owners_count":23216495,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T14:19:36.159Z","updated_at":"2025-06-25T08:05:21.918Z","avatar_url":"https://github.com/Quansight.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# omnisci-datascience-installer\nInstaller configuration for OmniSci stack\n\n## Setup\n\nThere are a couple of things to do to get started here.\n\n1. First install constructor:\n\n```sh\nconda install constructor\n```\n\nAlso, you can install the dependencies from the environment-dev.yml file:\n\n```sh\nconda env create -n omnisci-ds-installer --file environment-dev.yml\n```\n\nThis command creates an environment called `omnisci-ds-installer` (you can use\nanother name if you want) and installs the packages listed in `environment-dev.yml`.\n\n2. Second, grab a copy of conda-standalone that you want to use.\n   It is sometimes the case (especially on Windows) that the versions of\n   conda-standalone that come when you install constructor (from step 1)\n   are actually pretty broken. It is more stable to rely on the official\n   standalone releases. These can be found at https://repo.anaconda.com/pkgs/misc/conda-execs/\n   You'll want to grab a recent version of one of these and stash it somewhere.\n   For example:\n\n```sh\nwget https://repo.anaconda.com/pkgs/misc/conda-execs/conda-latest-osx-64.exe -O conda.exe\n```\n\n3. On Mac, building pkg installers can often result in using too many file handles.\n   To get around this, please run the following commands. Note that you will need\n   have root access for some of these:\n\n```sh\n# perform as regular user\nulimit -S -n 204800\n\n# perform as root\nsudo sysctl -w kern.maxfiles=204800\nsudo sysctl -w kern.maxfilesperproc=245670\n```\n\n\n## Running Constructor\n\nNow you may run constructor with the following command:\n\n```sh\n# constructor --conda-exe /path/to/conda.exe constructor/sh\nconstructor --conda-exe conda.exe constructor/sh\n```\n\nBe aware that, on Mac, whether you are creating a sh-based or pkg-based installer\ndepends on the contents of the `constructor/construct.yaml` file.  Simply change\nthe `installer_type` key to have a value of `pkg` or `sh` depending on what you want.\n\n## Gotchas\n\nThe `pkg` and `sh` installer seem to need to have `conda` itself listed as a spec.\nWithout that when it tries to run `conda init` it will fail.\n\n\n## Releasing\n\nFor releasing, after the PR is merged, create a tag with the last commit from master and\npush it to the repository, for example:\n\n```sh\ngit fetch upstream\ngit checkout upstream/master\nrever check\nrever 0.0.3\n```\n\n\n## Installation\n\nOmniSci Data Science package installs OmniSci stack that includes intake/intake-omnisci, omnisci-pytools, pymapd, ibis-framework, etc.\n\n\n### MacOS Instructions\n\nFollow the instructions below for your current setup:\n\n**If you have no conda installed on your machine**\nIMPORTANT: If you already have conda installed, this will overwrite the conda path in your bash profile! If you really want to run the following instructions, you probably want to create first a backup of your configuration files (for example, `$HOME/.bashrc`, `$HOME/.bash_profile`, `$HOME/.zshrc`, according to your environment). Otherwise, check the instructions in the sections below.\n\nThe OmniSci Data Science Installer can be installed using a shell script if you have no conda on your machine. The shell installers for each version of the package are available on the GitHub  [releases page](https://github.com/Quansight/omnisci-datascience-installer/releases/). For each release, you will find a shell (`sh`) installer.\n\n\nTo run the shell installer:\n\nFirst, download the shell package (`omnisci-datascience-\u003crelease_ver\u003e-MacOSX-x86_64.sh`) `omnisci-datascience-installer` [release page](https://github.com/Quansight/omnisci-datascience-installer/releases/) and add execution permission to it:\n\nDownload the installer and ensure that you have execution permission for the package, for example:\n```sh\nwget https://github.com/Quansight/omnisci-datascience-installer/releases/download/0.1.3/omnisci-datascience-0.1.3-MacOSX-x86_64.sh\nchmod +x omnisci-0.0.2-MacOSX-x86_64.sh\n```\n\nNow you can install it by executing the script. If you'd prefer to install it without manual intervention, you can execute it with the `-b` flag.\n```sh\n./omnisci-0.0.2-MacOSX-x86_64.sh -b\n```\nOtherwise, you can execute the shell script with no flags:\n\n```sh\n./omnisci-0.0.2-MacOSX-x86_64.sh\n```\n\n\nYou can scroll through the license agreement by hitting enter or spacebar. If you'd like to skip to the end, you can select `q`\n\nOnce you have agreed the the license agreement and the install location, the installer will unpack all the required packages into the the install location.\n\nYou will see that your `bash` profile has been modified to include the conda path. In order to utilize conda, you'll need to either close and reopen your terminal, or you can source this updated file.\n\n```sh\nsource .bash_profile\n```\n\nYou have successfully installed the OmniSci Datascience Installer on your Mac in the `base` conda environment. This environment has already been activated for you. You are now ready to run some examples!\n\n\n**If you already have conda installed:**\n\nIf you already have Conda installed on your Mac, you can create the OmniSci environment directly. These instructions provide the zip example, but you could also use git commands to download the repository.\n\nDownload the repository. A separate zip file is available for each version (e.g. we're downloading v0.1.0 here).\n\n```sh\nwget https://github.com/Quansight/omnisci-examples/archive/v0.1.0.zip\n```\nUnpack the zip file and navigate to the root of the package.\n```sh\nunzip v0.1.0.zip\ncd omnisci-examples-0.1.0\n```\n\nCreate a new conda environment. This will download all the necessary packages.\n```sh\nconda env create -f environment.yml -n omnisci\n```\nActivate the new enviroment (requires you to have run `conda init`)\n```sh\nconda activate omnisci\n```\nYou have successfully installed the OmniSci Datascience Installer on your Mac in the `omnisci` conda environment. You are now ready to run some examples!\n\n\n\n**If you have the Anaconda Navigator GUI**\n\nIf you have already Anaconda Navigator GUI, to install the OmniSci Data Science stack, we recommend you to import a new environment from this [example](https://github.com/Quansight/omnisci-examples). You can download all the repository or you can just download the environment.yml file. For now, we will download just the environment file, but feel free to download all the repository and play with the examples there:\n\n```sh\nwget https://raw.githubusercontent.com/Quansight/omnisci-examples/master/environment.yml -O /tmp/environment.yml\n```\n\nNow, open your Anaconda Navigator, and click on the `Environments` menu. To create a new environment from the file you just download, click on the `Import`. It will open a window requesting information about the new environment. Set the name for your new environment, select the environment file located at `/tmp/environment.yml` and click on the `Import` button.\n\nNow you can access the OmniSci Data Science stack using the new environment you just created!\n\n## Using the OmniSci Mac Preview\n\nIf you already have [Mac Preview](https://www.omnisci.com/mac-preview) and you want to connect to that from your jupyter-lab notebook, first open your jupyter-lab instance. From the command line, go to the folder you want as root for your notebooks (or use --notebook-dir with the path of your desired folder) and call the `jupyter-lab` command, for example:\n\n```sh\n# $HOME is a variable that points to the path of you user folder\nmkdir -p $HOME/notebooks\ncd $HOME/notebooks\njupyter-lab\n```\n\nTo create a new notebook for Python code, click on the Python 3 icon inside `Launcher` page. If `Launcher` page is not showing, click first on the `+` icon on the left corner near the menu bar:\n\n![jupyter-lab lancher page](https://user-images.githubusercontent.com/5209757/95106626-bd2d6880-0706-11eb-8d61-b76cbdf5b253.png)\n\nOmniSci Data Science installer installs automatically, among other packages, jupyterlab, ibis and pymapd. As default, OmniSci Mac Preview server listen to the port 16274. So, let's try to connect to the OmniSci Mac Preview and get the name of all tables available there!\n\nAdd the following code at the first cell of you notebook:\n\n```sh\n# import ibis-framework package\nimport ibis\n# connect to OmniSci Mac Preview\nclient = ibis.omniscidb.connect(\n    host=\"localhost\",\n    port=\"16274\",\n    database=\"omnisci\",\n    user=\"admin\",\n    password=\"HyperInteractive\",\n)\n# get the name of all tables available\nclient.list_tables()\n```\n\nNow, run that and check if it returns the follow result:\n\n```sh\n['omnisci_states',\n 'omnisci_counties',\n 'omnisci_countries',\n 'flights_2008_7M',\n 'us_counties_covid_cases_enriched']\n```\n\nIf it is not working, check if you did all the previous step according to the instructions. If the problem persists, we encourage you to open an [issue](https://github.com/Quansight/omnisci-datascience-installer/issues) with all the details about your problem.\n\nIf it worked, congratulations! Your environment is working properly and you can start to play with OmniSci Data Science tools by yourself! Have fun!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquansight%2Fomnisci-datascience-installer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquansight%2Fomnisci-datascience-installer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquansight%2Fomnisci-datascience-installer/lists"}