{"id":27370125,"url":"https://github.com/datastaxdevs/mini-demo-astradb-glean","last_synced_at":"2025-04-13T08:48:03.717Z","repository":{"id":257462924,"uuid":"858224328","full_name":"datastaxdevs/mini-demo-astradb-glean","owner":"datastaxdevs","description":"Demo showing how to index AstraDB data into Glean","archived":false,"fork":false,"pushed_at":"2024-09-16T20:59:49.000Z","size":91756,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-09-18T00:15:14.227Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datastaxdevs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-16T14:24:28.000Z","updated_at":"2024-09-16T20:59:52.000Z","dependencies_parsed_at":"2024-09-16T23:35:08.641Z","dependency_job_id":"6ed37c89-b9f7-455f-ba1d-7a06652163f1","html_url":"https://github.com/datastaxdevs/mini-demo-astradb-glean","commit_stats":null,"previous_names":["datastaxdevs/mini-demo-astradb-glean"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastaxdevs%2Fmini-demo-astradb-glean","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastaxdevs%2Fmini-demo-astradb-glean/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastaxdevs%2Fmini-demo-astradb-glean/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datastaxdevs%2Fmini-demo-astradb-glean/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datastaxdevs","download_url":"https://codeload.github.com/datastaxdevs/mini-demo-astradb-glean/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248688190,"owners_count":21145762,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-13T08:48:03.048Z","updated_at":"2025-04-13T08:48:03.711Z","avatar_url":"https://github.com/datastaxdevs.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mini-demo-astradb-glean\nDemo showing how to index AstraDB data into Glean\n\nYou can follow this tutorial fully in a google collab or follow the instructions below to run locally\n\n## Work in a Collab\n\n[![Open In Colab](https://img.shields.io/badge/Open%20in%20Colab-blue?logo=google-colab\u0026style=for-the-badge)](https://colab.research.google.com/github/datastaxdevs/mini-demo-astradb-glean/blob/main/AstraDB_Glean_Integration.ipynb)\n\n## Run Locally\n\n[![Run Locally](https://img.shields.io/badge/Run%20Locally-python3-blue?style=for-the-badge)](#)\n\n\n### 1.1 Setup AstraDB\n\nℹ️ [Astra Reference documentation](https://docs.datastax.com/en/astra-db-serverless/databases/create-database.html)\n\n\n`✅ 1.1.a`: Create an Astra ACCOUNT\n\nAccess [https://astra.datastax.com](https://astra.datastax.com) and register with `Google` or `Github` account.\n\n![](https://github.com/datastaxdevs/mini-demo-astradb-glean/blob/main/images/01-login.png?raw=true)\n\n\n`✅ 1.1.b`: Create an Astra Database\n\nGet to the databases dashboard (by clicking on Databases in the left-hand navigation bar, expanding it if necessary), and click the `[Create Database]` button on the right.\n\n![](https://github.com/datastaxdevs/mini-demo-astradb-glean/blob/main/images/02-create-db.png?raw=true)\n\n\n- **ℹ️ Fields Description**\n\n| Field                                      | Description                                                                                                                                                                                                                                   |\n|--------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| **Vector Database vs Serverless Database** | Choose `Vector Database` In june 2023, Cassandra introduced the support of vector search to enable Generative AI use cases.                                                                                                                   |\n| **Database name**                          | It does not need to be unique, is not used to initialize a connection, and is only a label (keep it between 2 and 50 characters). It is recommended to have a database for each of your applications. The free tier is limited to 5 databases. |\n| **Cloud Provider**                         | Choose whatever you like. Click a cloud provider logo, pick an Area in the list and finally pick a region. We recommend choosing a region that is closest to you to reduce latency. In free tier, there is very little difference.            |\n| **Cloud Region**                           | Pick region close to you available for selected cloud provider and your plan.      \n\nIf all fields are filled properly, clicking the \"Create Database\" button will start the process.\n\n![](https://github.com/datastaxdevs/mini-demo-astradb-glean/blob/main/images/03-pending-db.png?raw=true)\n\n\nIt should take a couple of minutes for your database to become `Active`.\n\n![](https://github.com/datastaxdevs/mini-demo-astradb-glean/blob/main/images/04-active-db.png?raw=true)\n\n`✅ 1.1.c`: Create an Astra TOKEN\n\nTo connect to your database, you need the API Endpoint and a token. The api endpoint is available on the database screen, there is a little icon to copy the URL in your clipboard. (it should look like `https://\u003cdb-id\u003e-\u003cdb-region\u003e.apps.astra.datastax.com`).\n\n![](https://github.com/datastaxdevs/mini-demo-astradb-glean/blob/main/images/05-create-token-db.png?raw=true)\n\nTo get a token click the `[Generate Token]` button on the right. It will generate a token that you can copy to your clipboard.\n\n\n## 2. Installation\n\n### 2.1 Python Environment\n\n- `✅ 2.1.a`: Create and activate a virtual environment\n\n```console\npython3 -m venv venv\n```\n\n_macOS_\n```\nsource venv/bin/activate\n```\n\n_Windows_\n```\nvenv\\Scripts\\activate\n```\n\n- `✅ 2.1.b`:Install the dependencies\n\n```console\npip install astrapy==1.4.1 --no-deps\npip install -r requirements.txt\n```\n\n- `✅ 2.1.c`: Edit `.env`\n\n_Copy `.env.example` as `.env`_\n\n```ini\n# Astra Configuration\nexport ASTRA_DB_APPLICATION_TOKEN=\u003cchange_me\u003e\nexport ASTRA_DB_API_ENDPOINT=\u003cchange_me\u003e\nexport ASTRA_DB_COLLECTION_NAME=\"plain_collection\"\n\n# Glean Configuration\nexport GLEAN_CUSTOMER=\u003cyou\u003e\nexport GLEAN_DATASOURCE_NAME=\u003cchange_me\u003e\nexport GLEAN_API_TOKEN=\u003cchange_me\u003e\n```\n\n\n- `✅ 2.1.d`:Run the script\n\n```console\npython3 astra-glean-import-job.py\n```\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatastaxdevs%2Fmini-demo-astradb-glean","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatastaxdevs%2Fmini-demo-astradb-glean","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatastaxdevs%2Fmini-demo-astradb-glean/lists"}