{"id":20408726,"url":"https://github.com/feast-dev/feast-custom-online-store-demo","last_synced_at":"2025-04-12T15:35:43.958Z","repository":{"id":39632734,"uuid":"398345553","full_name":"feast-dev/feast-custom-online-store-demo","owner":"feast-dev","description":null,"archived":false,"fork":false,"pushed_at":"2021-10-24T06:15:27.000Z","size":54,"stargazers_count":1,"open_issues_count":1,"forks_count":7,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-26T10:11:17.343Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/feast-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-20T16:57:19.000Z","updated_at":"2022-09-27T19:31:53.000Z","dependencies_parsed_at":"2022-09-20T07:12:00.850Z","dependency_job_id":null,"html_url":"https://github.com/feast-dev/feast-custom-online-store-demo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-online-store-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-online-store-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-online-store-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-online-store-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/feast-dev","download_url":"https://codeload.github.com/feast-dev/feast-custom-online-store-demo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248590172,"owners_count":21129762,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T05:35:53.064Z","updated_at":"2025-04-12T15:35:43.915Z","avatar_url":"https://github.com/feast-dev.png","language":"Python","readme":"# Feast Custom Online Store\n[![test_custom_online_store](https://github.com/feast-dev/feast-custom-online-store-demo/actions/workflows/test_custom_online_store.yml/badge.svg?branch=main)](https://github.com/feast-dev/feast-custom-online-store-demo/actions/workflows/test_custom_online_store.yml)\n\n### Overview\n\nThis repository demonstrates how developers can create their own custom `online store`s for Feast. Custom online stores allow users to use any underlying \ndata store to store features for low-latency retrieval, typically needed during model inference.\n\n### Why create a custom online store?\n\nFeast materializes data to online stores for low-latency lookup at model inference time. Typically, key-value stores are used for \nthe online stores, however relational databases can be used for this purpose as well.\n\nFeast comes with some online stores built in, e.g, Sqlite, Redis, DynamoDB and Datastore. However, users can develop their\nown online stores by creating a class that implements the contract in the [OnlineStore class](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/online_stores/online_store.py#L26).\n\n### What is included in this repository?\n\n* [feast_custom_online_store/](feast_custom_online_store): An example of a custom online store, `MySQLOnlineStore`, which implements `OnlineStore`. This example online store uses MySQL as the backing database.\n* [feature_repo/](feature_repo): A simple feature repository that is used to test the custom online store. The repository has been configured to use the `MySQLOnlineStore` as part of it's `feature_store.yaml`\n* [test_custom_online_store.py](test_custom_online_store.py): A test case that uses `MySQLOnlineStore` through the `feature_repo/`\n\n### Testing the custom online store in this repository\n\nRun the following commands to test the custom online store ([MySQLOnlineStore](https://github.com/feast-dev/feast-custom-online-store-demo/blob/master/feast_custom_online_store/mysql.py))\n\n```bash\npip install -r requirements.txt\n```\n\n```\npytest test_custom_online_store.py\n```\n\nIt is also possible to run a Feast CLI command, which interacts with the online store. It may be necessary to add the \n`PYTHONPATH` to the path where your online store module is stored.\n```\nPYTHONPATH=$PYTHONPATH:/$(pwd) feast -c basic_feature_repo apply\n\n```\n```\nRegistered entity driver_id\nRegistered feature view driver_hourly_stats\nDeploying infrastructure for driver_hourly_stats\n```\n\n```\n$ PYTHONPATH=$PYTHONPATH:/$(pwd) feast -c feature_repo materialize-incremental 2021-08-19T22:29:28\n```\n```Materializing 1 feature views to 2021-08-19 15:29:28-07:00 into the feast_custom_online_store.mysql.MySQLOnlineStore online store.\n\ndriver_hourly_stats from 2020-08-24 05:23:49-07:00 to 2021-08-19 15:29:28-07:00:\n100%|████████████████████████████████████████████████████████████████| 5/5 [00:00\u003c00:00, 120.59it/s]\n```\n\n### Testing against the Feast test suite\n\nA subset of the Feast test suite, called \"universal tests\", are designed to test the core behavior of offline and online stores. A custom online store implementation can use the universal tests as follows.\n\nFirst, this repository contains Feast as a submodule. To fetch and populate the directory, run\n```\ngit submodule update --init --recursive\n```\n\nNext, install Feast following the instructions [here](https://github.com/feast-dev/feast/blob/master/CONTRIBUTING.md)\n```\ncd feast\npip install -e \"sdk/python[ci]\"\n```\nand confirm that the Feast unit tests run as expected:\n```\nmake test\n```\n\nThe Feast universal tests can be run with the command\n```\nmake test-python-universal\n```\n\nIf the command is run immediately, the tests should fail. The tests are parametrized based on the `FULL_REPO_CONFIGS` variable defined in `sdk/python/tests/integration/feature_repos/repo_configuration.py`. To overwrite these configurations, you can simply create your own file that contains a `FULL_REPO_CONFIGS`, and point Feast to that file by setting the environment variable `FULL_REPO_CONFIGS_MODULE` to point to that file. In this repo, the file that overwrites `FULL_REPO_CONFIGS` is `feast_custom_online_store/feast_tests.py`, so you would run\n```\nexport FULL_REPO_CONFIGS_MODULE='feast_custom_online_store.feast_tests'\nmake test-python-universal\n```\nto test the MySQL online store against the Feast universal tests. You should notice that some of the tests actually fail; this indicates that there is a mistake in the implementation of this online store!","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeast-dev%2Ffeast-custom-online-store-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffeast-dev%2Ffeast-custom-online-store-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeast-dev%2Ffeast-custom-online-store-demo/lists"}