{"id":28921555,"url":"https://github.com/localstack-samples/snowflake-smart-factory-app","last_synced_at":"2026-03-08T09:31:09.071Z","repository":{"id":296296076,"uuid":"982892668","full_name":"localstack-samples/snowflake-smart-factory-app","owner":"localstack-samples","description":"End-to-end smart-factory pipeline on LocalStack’s Snowflake emulator: S3 → Snowpipe → dbt → Dagster + Streamlit dashboard, real-time SES alerts, and zero-copy clone recovery.","archived":false,"fork":false,"pushed_at":"2025-07-07T10:57:50.000Z","size":641,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-07-07T11:41:57.399Z","etag":null,"topics":["dagster","dbt","localstack","s3","ses","smart-factory","snowflake","streamlit"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/localstack-samples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-13T14:59:32.000Z","updated_at":"2025-06-02T13:01:34.000Z","dependencies_parsed_at":"2025-05-29T23:33:01.565Z","dependency_job_id":"1fba8f4f-fbd1-431b-91f1-9be45601315f","html_url":"https://github.com/localstack-samples/snowflake-smart-factory-app","commit_stats":null,"previous_names":["localstack-samples/snowflake-smart-factory-app"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/localstack-samples/snowflake-smart-factory-app","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/localstack-samples%2Fsnowflake-smart-factory-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/localstack-samples%2Fsnowflake-smart-factory-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/localstack-samples%2Fsnowflake-smart-factory-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/localstack-samples%2Fsnowflake-smart-factory-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/localstack-samples","download_url":"https://codeload.github.com/localstack-samples/snowflake-smart-factory-app/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/localstack-samples%2Fsnowflake-smart-factory-app/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30251267,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-08T08:59:44.879Z","status":"ssl_error","status_checked_at":"2026-03-08T08:58:02.867Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dagster","dbt","localstack","s3","ses","smart-factory","snowflake","streamlit"],"created_at":"2025-06-22T07:07:04.749Z","updated_at":"2026-03-08T09:31:08.658Z","avatar_url":"https://github.com/localstack-samples.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Smart Factory Monitoring App\n\nThis project showcases a comprehensive smart factory monitoring application specifically designed to demonstrate **LocalStack for Snowflake** capabilities for local data cloud development, debugging, and testing throughout the entire software development lifecycle (SDLC).\n\nThis application showcases the full spectrum of **LocalStack for Snowflake** features, such as:\n\n- Emulating a local Snowflake environment with databases, schemas, tables, and more.\n- Utilizing S3 service for automated data ingestion with Snowpipe integration.\n- Implementing dbt transformations for data quality and processing.\n- Building interactive dashboards with Streamlit and Snowflake Native Apps.\n- Orchestrating data transformations in an automated pipeline with Dagster.\n- Implementing comprehensive data \u0026 integration testing with pytest and dbt tests.\n- Using GitHub Actions for continuous integration to ensure automated testing.\n- Adding real-time alerting with a Python User-Defined Function (UDFs) and Snowflake Tasks.\n- Showcasing Zero-Copy Cloning for instant data recovery scenarios in case of data loss.\n\nThe application serves as a complete reference implementation for developing Snowflake data solutions locally, enabling real-time monitoring of factory machines, automated data ingestion, quality testing, and predictive maintenance alerts—all running entirely on your local machine through LocalStack's Snowflake emulator.\n\n## Architecture\n\nThe application implements a modern data pipeline architecture showcasing a realistic smart factory scenario:\n\n![Architecture](./images/architecture-diagram.png)\n\n## Prerequisites\n\nFeatures in this sample app require a LocalStack for Snowflake license - make sure your Auth Token is configured in your terminal session.\n\n```bash\nexport LOCALSTACK_AUTH_TOKEN=\u003cyour-token\u003e\n```\n\n- [Docker](https://docs.docker.com/get-docker/)\n- [`localstack` CLI](https://docs.localstack.cloud/getting-started/installation/#localstack-cli).\n- [AWS CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) with the [`awslocal` wrapper](https://docs.localstack.cloud/user-guide/integrations/aws-cli/#localstack-aws-cli-awslocal).\n- [Snowflake CLI](https://docs.snowflake.com/developer-guide/snowflake-cli/index) with a [`localstack` connection profile](https://snowflake.localstack.cloud/user-guide/integrations/snow-cli/).\n- [Python](https://www.python.org/downloads/) 3.10+ \u0026 [`pip`](https://pip.pypa.io/en/stable/installation/)\n- [`make`](https://www.gnu.org/software/make/) (**optional**, but recommended for running the sample application)\n\n## Installation\n\nTo run the sample application, you need to install the required dependencies.\n\nFirst, clone the repository:\n\n```bash\ngit clone https://github.com/localstack-samples/snowflake-smart-factory-app.git\n```\n\nThen, navigate to the project directory:\n\n```bash\ncd snowflake-smart-factory-app\n```\n\nNext, install the project dependencies by running the following command:\n\n```bash\nmake install\n```\n\n## Deployment\n\nStart LocalStack with the `LOCALSTACK_AUTH_TOKEN` pre-configured:\n\n```bash\nlocalstack auth set-token \u003cyour-auth-token\u003e\nDOCKER_FLAGS='-e SF_LOG=trace' \\\nDEBUG=1 \\\nIMAGE_NAME=localstack/snowflake:latest \\\nlocalstack start\n```\n\nTo deploy the sample application, run the following command:\n\n```bash\nmake deploy\n```\n\nThis will:\n\n- Setup Snowflake database, tables, and Snowpipe\n- Setup S3 bucket with event notifications\n- Upload sample sensor data (`data/sensor_data_batch_1.csv`)\n- Run dbt transformations to process the data\n\nYou can also run the following command to deploy the Native app:\n\n```bash\nmake app\n```\n\nThe output will be similar to the following:\n\n```bash\n[●] Deployed on LocalStack\n └─ URL: https://snowflake.localhost.localstack.cloud:4566/apps/test/test/FACTORY_APP_{USER}/\n```\n\nThe dashboard provides:\n\n- Real-time machine health monitoring\n- Interactive sensor data visualizations\n- Predictive maintenance recommendations\n- Anomaly detection and alerting\n\n## Testing\n\nYou can run full end-to-end integration tests using the following command:\n\n```bash\nmake test\n```\n\nThis executes:\n\n- **Machine Health Tests**: Verifies health metrics calculations and thresholds\n- **Snowpipe Integration Tests**: Tests automated data ingestion workflows\n\n## Use Cases\n\n### Pipeline Orchestration\n\nTo run automated dbt transformations on new data, you can launch a Dagster pipeline:\n\n```bash\nmake pipeline\n```\n\nThis will:\n\n- Setup a pipeline monitoring dashboard\n- Create a S3 sensor to monitor new data\n- Trigger dbt transformations on new data\n\nTo trigger dbt transformations on new data, you can run the following command:\n\n```bash\nmake upload LATEST=true\n```\n\nThis will upload the latest sensor data batch (`data/sensor_data_batch_2.csv`) to the S3 bucket. The Dagster pipeline will automatically detect the new data and trigger the dbt transformations.\n\nTo generate new sensor data, you can run the following command:\n\n```bash\nmake generate\n```\n\n### Real-Time Alerting\n\nTo set up automated email alerts for critical machine conditions, you can run the following command:\n\n```bash\nmake alerts\n```\n\nThis will:\n\n- Create a Python UDF to send email alerts with SES integration\n- Create a Snowflake Task to trigger the UDF every 30 seconds\n- Create a Snowflake View to query critical machine data\n- Send HTML emails for critical machine conditions\n\nIn 30 seconds, you can query the SES developer endpoint to see the alert email:\n\n```bash\ncurl -s http://localhost.localstack.cloud:4566/_aws/ses\n```\n\nYou can also use the [Mailhog extension](https://github.com/localstack/localstack-extensions/tree/main/mailhog) to view the alert email via a user-friendly interface.\n\n### Data Recovery with Zero-Copy Cloning\n\nTo demonstrate instant data recovery capabilities, you can run the following command:\n\n```bash\nsnow sql -f solutions/data_recovery_clones.sql -c localstack\n```\n\nThis showcases:\n\n- Instant table cloning without data duplication\n- Point-in-time recovery scenarios\n- Disaster recovery best practices\n\n### Test Data Management with Cloud Pods\n\nCloud Pods enable persistent state snapshots of your LocalStack instance that can be easily stored, versioned, shared, and restored across development teams and CI/CD environments.\n\nTo save a Cloud Pod, run the following command:\n\n```bash\nlocalstack pod save smart-factory-app\n```\n\nTo load a Cloud Pod, run the following command:\n\n```bash\n# Restore the saved state in a fresh LocalStack instance\nlocalstack pod load smart-factory-app\n```\n\nThe key advantages are:\n\n- Pre-seed CI environments with complex factory data, Snowflake schemas, and S3 assets in seconds\n- Share identical test scenarios across development teams for consistent debugging\n- Guarantee the same test data state across different pipeline runs and environments  \n- Capture problematic states for collaborative debugging and issue reproduction\n\nFor automated Cloud Pods management in CI/CD pipelines, check out the sample workflow in [`.github/workflows/cloud-pods.yml`](.github/workflows/cloud-pods.yml).\n\n## License\n\nThis project is licensed under the [Apache License 2.0](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flocalstack-samples%2Fsnowflake-smart-factory-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flocalstack-samples%2Fsnowflake-smart-factory-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flocalstack-samples%2Fsnowflake-smart-factory-app/lists"}