{"id":20408725,"url":"https://github.com/feast-dev/feast-custom-offline-store-demo","last_synced_at":"2025-07-30T07:05:43.910Z","repository":{"id":39726358,"uuid":"399590721","full_name":"feast-dev/feast-custom-offline-store-demo","owner":"feast-dev","description":null,"archived":false,"fork":false,"pushed_at":"2021-10-24T06:14:48.000Z","size":47,"stargazers_count":8,"open_issues_count":0,"forks_count":10,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-12T15:57:42.403Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/feast-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-24T20:03:43.000Z","updated_at":"2024-08-28T08:30:54.000Z","dependencies_parsed_at":"2022-09-20T09:34:33.109Z","dependency_job_id":null,"html_url":"https://github.com/feast-dev/feast-custom-offline-store-demo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/feast-dev/feast-custom-offline-store-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-offline-store-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-offline-store-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-offline-store-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-offline-store-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/feast-dev","download_url":"https://codeload.github.com/feast-dev/feast-custom-offline-store-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/feast-dev%2Ffeast-custom-offline-store-demo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267827419,"owners_count":24150346,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T05:35:53.054Z","updated_at":"2025-07-30T07:05:43.886Z","avatar_url":"https://github.com/feast-dev.png","language":"Python","readme":"# Feast Custom Offline Store\n[![test-offline-store](https://github.com/feast-dev/feast-custom-offline-store-demo/actions/workflows/test_custom_offline_store.yml/badge.svg?branch=main)](https://github.com/feast-dev/feast-custom-offline-store-demo/actions/workflows/test_custom_offline_store.yml)\n\n### Overview\n\nThis repository demonstrates how developers can create their own custom `offline store`s for Feast.\nCustom offline stores allow users to use any underlying data store as their offline feature store. Features can be retrieved from the offline store for model training, and can be materialized into the online feature store for use during model inference. \n\n\n### Why create a custom offline store?\n\nFeast uses an offline store as the source of truth for features. These features can be retrieved from the offline store for model training. Typically, scalable data warehouses are used for this purpose.\n \nFeast also materializes features from offline stores to an online store for low-latency lookup at model inference time. \n\nFeast comes with some offline stores built in, e.g, Parquet file, Redshift and Bigquery. However, users can develop their own offline stores by creating a class that implements the contract in the [OfflineStore class](https://github.com/feast-dev/feast/blob/5e61a6f17c3b52f20b449214a4bb56bafa5cfcbc/sdk/python/feast/infra/offline_stores/offline_store.py#L41).\n\n### What is included in this repository?\n\n* [feast_custom_offline_store/](feast_custom_offline_store): An example of a custom offline store, `CustomFileOfflineStore`, which implements OfflineStore. This example offline store overrides the File offline store that is provided by Feast.\n* [feature_repo/](feature_repo): A simple feature repository that is used to test the custom offline store. The repository has been configured to use the custom offline store as part of it's `feature_store.yaml`\n* [test_custom_offline_store.py](test_custom_offline_store.py): A test case that uses `CustomFileOfflineStore` through the `feature_repo/`\n\n### Testing the custom offline store in this repository\n\nRun the following commands to test the custom offline store ([FileCustomOfflineStore](https://github.com/feast-dev/feast-custom-offline-store-demo/blob/master/feast_custom_offline_store/file.py))\n\n```bash\npip install -r requirements.txt\n```\n\n```\npytest test_custom_offline_store.py\n```\n\nIt is also possible to run Feast CLI commands, which in turn will call the offline store. It may be necessary to add the \n`PYTHONPATH` to the path where your offline store module is located.\n```\n$ PYTHONPATH=$PYTHONPATH:/$(pwd) feast -c feature_repo apply\n```\n```\nRegistered entity driver_id\nRegistered feature view driver_hourly_stats\nDeploying infrastructure for driver_hourly_stats\n```\n```\n$ PYTHONPATH=$PYTHONPATH:/$(pwd) feast -c feature_repo materialize-incremental 2021-08-19T22:29:28\n```\n```\nMaterializing 1 feature views to 2021-08-19 15:29:28-07:00 into the sqlite online store.\n\ndriver_hourly_stats from 2020-08-24 20:54:03-07:00 to 2021-08-19 15:29:28-07:00:\nPulling latest features from my offline store\n100%|███████████████████████████████████████████████████████████████| 5/5 [00:00\u003c00:00, 2122.19it/s]\n```\n\n### Testing against the Feast test suite\n\nA subset of the Feast test suite, called \"universal tests\", are designed to test the core behavior of offline and online stores. A custom offline store implementation can use the universal tests as follows.\n\nFirst, this repository contains Feast as a submodule. To fetch and populate the directory, run\n```\ngit submodule update --init --recursive\n```\n\nNext, install Feast following the instructions [here](https://github.com/feast-dev/feast/blob/master/CONTRIBUTING.md)\n```\ncd feast\npip install -e \"sdk/python[ci]\"\n```\nand confirm that the Feast unit tests run as expected:\n```\nmake test\n```\n\nThe Feast universal tests can be run with the command\n```\nmake test-python-universal\n```\n\nIf the command is run immediately, the tests should fail. The tests are parametrized based on the `FULL_REPO_CONFIGS` variable defined in `sdk/python/tests/integration/feature_repos/repo_configuration.py`. To overwrite these configurations, you can simply create your own file that contains a `FULL_REPO_CONFIGS`, and point Feast to that file by setting the environment variable `FULL_REPO_CONFIGS_MODULE` to point to that file. In this repo, the file that overwrites `FULL_REPO_CONFIGS` is `feast_custom_offline_store/feast_tests.py`, so you would run\n```\nexport FULL_REPO_CONFIGS_MODULE='feast_custom_offline_store.feast_tests'\nmake test-python-universal\n```\nto test the custom file offline store against the Feast universal tests. You should notice that a couple of the tests actually fail; this indicates that there is a mistake in the implementation of this offline store!","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeast-dev%2Ffeast-custom-offline-store-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffeast-dev%2Ffeast-custom-offline-store-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffeast-dev%2Ffeast-custom-offline-store-demo/lists"}