{"id":15132535,"url":"https://github.com/lazauk/azureml-featurestore-cli","last_synced_at":"2026-01-20T21:02:23.331Z","repository":{"id":256504501,"uuid":"852988212","full_name":"LazaUK/AzureML-FeatureStore-CLI","owner":"LazaUK","description":"Tutorial on setup of online and offline feature stores in Azure Machine Learning (Azure ML), using command line interface (Az CLI)","archived":false,"fork":false,"pushed_at":"2024-09-11T00:42:00.000Z","size":336,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T21:44:41.575Z","etag":null,"topics":["adls","ai","azure","feature-store","machine-learning","offline","online","redis"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LazaUK.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-05T19:24:35.000Z","updated_at":"2024-09-13T13:13:30.000Z","dependencies_parsed_at":"2024-09-11T06:43:55.351Z","dependency_job_id":"058d5892-18a0-473e-848d-8a735a2ce0a4","html_url":"https://github.com/LazaUK/AzureML-FeatureStore-CLI","commit_stats":null,"previous_names":["lazauk/azureml-featurestore-cli"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LazaUK%2FAzureML-FeatureStore-CLI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LazaUK%2FAzureML-FeatureStore-CLI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LazaUK%2FAzureML-FeatureStore-CLI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LazaUK%2FAzureML-FeatureStore-CLI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LazaUK","download_url":"https://codeload.github.com/LazaUK/AzureML-FeatureStore-CLI/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247406069,"owners_count":20933802,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adls","ai","azure","feature-store","machine-learning","offline","online","redis"],"created_at":"2024-09-26T04:20:36.426Z","updated_at":"2026-01-20T21:02:23.243Z","avatar_url":"https://github.com/LazaUK.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Azure Machine Learning - Feature Store (CLI Setup)\n\nReady to boost your machine learning projects? Azure Machine Learning (Azure ML) offers a centralised repository called _**Feature Store**_ to simplify feature engineering. It lets you store, manage and efficiently reuse features across projects, saving time and efforts.\n\nThis guide will show you how to set up an Azure ML Feature Store with online and offline materialisation using the **Azure CLI** (Az CLI). The online store is powered by the in-memory capacity of _Azure Redis Cache_, while the offline store utilises the 2nd generation of Azure Data Lake Store (_ADLS Gen2_).\n\nHere's what you'll need:\n- An Azure subscription,\n- Az CLI installed (we'll cover its ML extension installation in the first step).\n\n## Table of contents:\n- [Step 1: Installing Az CLI's ML extension](https://github.com/LazaUK/AzureML-FeatureStore-CLI#step-1-installing-az-clis-ml-extension)\n- [Step 2: Creating ADLS Gen2 storage account](https://github.com/LazaUK/AzureML-FeatureStore-CLI#step-2-creating-adls-gen2-storage-account)\n- [Step 3: Creating container on ADLS Gen2 storage](https://github.com/LazaUK/AzureML-FeatureStore-CLI#step-3-creating-container-on-adls-gen2-storage)\n- [Step 4: Creating Redis Cache instance](https://github.com/LazaUK/AzureML-FeatureStore-CLI#step-4-creating-redis-cache-instance)\n- [Step 5: Creating user-assigned Managed Identity](https://github.com/LazaUK/AzureML-FeatureStore-CLI#step-5-creating-user-assigned-managed-identity)\n- [Step 6: Creating Azure ML feature store](https://github.com/LazaUK/AzureML-FeatureStore-CLI#step-6-creating-azure-ml-feature-store)\n- [HOUSEKEEPING: Deleting feature store](https://github.com/LazaUK/AzureML-FeatureStore-CLI#housekeeping-deleting-feature-store)\n\n## Step 1: Installing Az CLI's ML extension\nTo interact with Azure ML through Az CLI, you need to install the ML extension:\n``` Bash\naz extension add --name ml\n```\n\n## Step 2: Creating ADLS Gen2 storage account\nNow create an ADLS Gen2 account, that will be used as the default storage account by your feature store:\n``` Bash\naz storage account create --name \u003cSTORAGE_ACCOUNT_NAME\u003e --enable-hierarchical-namespace true --resource-group \u003cRESOURCE_GROUP_NAME\u003e --location \u003cAZ_REGION\u003e --subscription \u003cAZ_SUBSCRIPTION_ID\u003e\n```\n\u003e Note: Replace ```\u003cSTORAGE_ACCOUNT_NAME\u003e```, ```\u003cRESOURCE_GROUP_NAME\u003e```, ```\u003cAZ_REGION\u003e``` and ```\u003cAZ_SUBSCRIPTION_ID\u003e``` with required Storage account values.\n\n## Step 3: Creating container on ADLS Gen2 storage\nOnce you've created a Storage account, you can setup a blob container that will be used by the feature store for offline materialisation:\n``` Bash\naz storage fs create --name \u003cSTORAGE_CONTAINER_NAME\u003e --account-name \u003cSTORAGE_ACCOUNT_NAME\u003e --subscription \u003cAZ_SUBSCRIPTION_ID\u003e --connection-string \u003cCONNECTION_STRING\u003e\n```\n\u003e Note: Replace ```\u003cSTORAGE_CONTAINER_NAME\u003e```, ```\u003cSTORAGE_ACCOUNT_NAME\u003e```, ```\u003cCONNECTION_STRING\u003e``` and ```\u003cAZ_SUBSCRIPTION_ID\u003e``` with required Storage account's container values.\n\n## Step 4: Creating Redis Cache instance\nIf you want your ML models to access features with low latency, create a Redis Cache instance for online materialisation:\n``` Bash\naz redis create --name \u003cREDIS_CACHE_NAME\u003e --resource-group \u003cRESOURCE_GROUP_NAME\u003e --location \u003cAZ_REGION\u003e --sku \u003cREDIS_CACHE_SKU_TIER\u003e --vm-size \u003cREDIS_CACHE_SKU_FAMILY\u003e\n```\n\u003e Note: Replace ```\u003cREDIS_CACHE_NAME\u003e```, ```\u003cRESOURCE_GROUP_NAME\u003e```, ```\u003cAZ_REGION\u003e```, ```\u003cREDIS_CACHE_SKU_TIER\u003e``` and ```\u003cREDIS_CACHE_SKU_FAMILY\u003e``` with required Redis Cache resource's values.\n\n## Step 5: Creating user-assigned Managed Identity\nAzure can automatically create a managed identity for your feature store. Alternatively, you can pre-provision your own, e.g to follow corporate naming conventions:\n```\naz identity create --name \u003cMI_NAME\u003e --resource-group \u003cRESOURCE_GROUP_NAME\u003e --location \u003cAZ_REGION\u003e --subscription \u003cAZ_SUBSCRIPTION_ID\u003e\n```\n\u003e Note: Replace ```\u003cMI_NAME\u003e```, ```\u003cRESOURCE_GROUP_NAME\u003e```, ```\u003cAZ_REGION\u003e``` and ```\u003cAZ_SUBSCRIPTION_ID\u003e``` with required managed identity's values.\n\n## Step 6: Creating Azure ML feature store\nThe last step is to create the feature store itself.\n\n1. Download the provided [YAML template](./FeatureStore_Online_Offline.yaml) and update it with details from your previous steps (storage, Redis Cache and managed identity):\n``` YAML\n$schema: http://azureml/sdk-2-0/FeatureStore.json\n\n# General configuration of Azure ML feature store\nname: feature-store-online-offline\ndisplay_name: \"Feature store with both online and offline materialisation\"\nresource_group: \u003crg\u003e\nlocation: \u003caz-region\u003e\ntags:\n  author: Laziz_Turakulov\n\n# Apache Spark configuration\ncompute_runtime:\n  spark_runtime_version: \"3.4\"\n\n# Details of user-assigned managed identity\nmaterialization_identity:\n  client_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\n  principal_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\n  resource_id: \"/subscriptions/\u003csub-id\u003e/resourceGroups/\u003crg\u003e/providers/Microsoft.ManagedIdentity/userAssignedIdentities/\u003cmi-name\u003e\"\n\n# Details of Azure Redis Cache (online materialisation)\nonline_store:\n  type: redis\n  target: \"/subscriptions/\u003csub-id\u003e/resourceGroups/\u003crg\u003e/providers/Microsoft.Cache/Redis/\u003credis-name\u003e\"\n\n# Details of ADLS Gen2 Storage account (offline materialisation)\noffline_store:\n  type: azure_data_lake_gen2\n  target: \"/subscriptions/\u003csub-id\u003e/resourceGroups/\u003crg\u003e/providers/Microsoft.Storage/storageAccounts/\u003cstorage_name\u003e/blobServices/default/containers/\u003ccontainer_name\u003e\"\n  \n# Details of default Storage account\nstorage_account: \"/subscriptions/\u003csub-id\u003e/resourceGroups/\u003crg\u003e/providers/Microsoft.Storage/storageAccounts/\u003cstorage_name\u003e\"\n```\n2. Then run the following Az CLI command to create the feature store:\n``` Bash\naz ml feature-store create --resource-group \u003cRESOURCE_GROUP_NAME\u003e --file FeatureStore_Online_Offline.yaml\n```\n3. Verify successful creation by checking the Azure ML Studio UI for the newly created feature store.\n![AzureML_FeatureStore_Home](images/AzureML_FeatureStore_Home.png)\n4. Your managed identity should have been assigned the following roles:\n- ```AzureML Data Scientist``` to Azure ML feature store,\n- ```Storage Blob Data Contributor``` to default Storage account,\n- ```Storage Blob Data Contributor``` to offline store's Blob container,\n- ```Contributor``` to online store's Redis Cache resource.\n![AzureML_FeatureStore_MI](images/AzureML_FeatureStore_MI.png)\n\n## HOUSEKEEPING: Deleting feature store\nWhile the Azure ML Studio UI currently doesn't support deleting feature stores directly, you can use the following Az CLI command:\n``` Bash\naz ml feature-store delete --name \u003cFEATURE_STORE_NAME\u003e --resource-group \u003cRESOURCE_GROUP_NAME\u003e --all-resources\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flazauk%2Fazureml-featurestore-cli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flazauk%2Fazureml-featurestore-cli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flazauk%2Fazureml-featurestore-cli/lists"}