{"id":22893382,"url":"https://github.com/iliashad/visual-product-search-app","last_synced_at":"2025-07-22T11:41:01.782Z","repository":{"id":267899900,"uuid":"897858090","full_name":"IliasHad/visual-product-search-app","owner":"IliasHad","description":"Visual Product Search API using Shopify API, Algolia, Google Vision API, Remix.js and React Native","archived":false,"fork":false,"pushed_at":"2025-03-10T01:35:06.000Z","size":1115,"stargazers_count":6,"open_issues_count":5,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-14T13:10:04.665Z","etag":null,"topics":["google-cloud","react-native","shopify"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IliasHad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-03T11:11:26.000Z","updated_at":"2025-05-20T05:05:56.000Z","dependencies_parsed_at":"2024-12-13T05:28:09.330Z","dependency_job_id":"00d8f5a0-ef50-45ba-8cdc-f88a4bec2516","html_url":"https://github.com/IliasHad/visual-product-search-app","commit_stats":null,"previous_names":["iliashad/visual-product-search-app"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/IliasHad/visual-product-search-app","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IliasHad%2Fvisual-product-search-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IliasHad%2Fvisual-product-search-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IliasHad%2Fvisual-product-search-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IliasHad%2Fvisual-product-search-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IliasHad","download_url":"https://codeload.github.com/IliasHad/visual-product-search-app/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IliasHad%2Fvisual-product-search-app/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266486926,"owners_count":23937054,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["google-cloud","react-native","shopify"],"created_at":"2024-12-13T23:14:15.026Z","updated_at":"2025-07-22T11:41:01.668Z","avatar_url":"https://github.com/IliasHad.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# React Native: Visual Product Search API using Shopify API\n\n## Overview\n\n![visual-search](https://github.com/user-attachments/assets/f8477560-2a6f-4807-b139-19ea81c28d07)\n\n\n\n| License | Mobile Technology    | Image Labeling | eCommerce CMS | Search API |\n| :---:   | :---: |  :---: |  :---: |   :---: |\n | [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)  | ![React Native](https://img.shields.io/badge/react_native-%2320232a.svg?style=for-the-badge\u0026logo=react\u0026logoColor=%2361DAFB)  | [![Google Cloud](https://img.shields.io/badge/Google%20Cloud-%234285F4.svg?logo=google-cloud\u0026logoColor=white)](#) | [![Shopify](https://img.shields.io/badge/Shopify-7AB55C?logo=shopify\u0026logoColor=fff)](#) | Algolia Search API\n\n\n| Browsing | Image Searching    | Displaying Results    |\n| :---:   | :---: | :---: |\n |![IMG_7232](https://github.com/user-attachments/assets/73118f63-ec74-44af-8ff0-dc8edbe216dd) | ![IMG_7234](https://github.com/user-attachments/assets/9ac14cdb-9b64-40a2-a006-d952a29e3935) |   ![IMG_7233](https://github.com/user-attachments/assets/179fa312-5293-4c74-9a49-c791efe3d34a)|\n\n### Developer Note: Keeping it Simple\n\nWhile building this demo, I intentionally simplified many aspects that would be crucial for a production system. Here's why:\n\nThe main goal was to demonstrate how to:\n\n- Connect Google Cloud Vision API with Shopify product search\n- Process product images to enable visual search\n- Create a basic mobile interface for image uploads\n\nI deliberately omitted production concerns like error handling, security, and scalability because:\n\n1. This is a learning demo focused on the core visual search workflow\n2. Adding production-grade features would obscure the main technical concepts\n3. Each production requirement (auth, caching, monitoring etc.) deserves its own deep dive\n\nThink of this implementation as a \"proof of concept\" that shows:\n\n- The basic architecture works\n- Visual search can enhance product discovery\n- The core APIs integrate successfully\n\nFor anyone looking to build on this demo, you'd want to address production concerns based on your specific needs - like authentication, error handling, and performance optimization. But those additions shouldn't overshadow understanding the core visual search mechanics demonstrated here.\n\nThe goal was to keep the code approachable and focused on demonstrating value, rather than building a production-ready system. This lets developers understand the core concepts first, then layer in production requirements as needed.\n\n\n### Demo Video\n\nhttps://github.com/user-attachments/assets/28e109bb-14d1-40e8-a1ec-1b8415c54a19\n\n\n## Features\n\n- 📸 Visual Product Search\n- 🚀 Instant Product Matching\n- 📱 Cross-Platform Mobile Experience (iOS \u0026 Android)\n\n## Tech Stack\n\n- **Frontend**: React Native\n- **Image Recognition**: Google Cloud Platform Vision API\n- **Search**: Algolia Search API\n- **Backend**: Remix.js\n\n## Prerequisites\n\n- Node.js (v16+ recommended)\n- npm or Yarn\n- React Native CLI\n- Google Cloud Platform Account\n- Algolia Account\n- Shopify Partner Account or a Shopify development store\n\n## Get started\n\n1. Install dependencies\n\n   ```bash\n   npm install\n   ```\n\n2. Start the app\n\n   ```bash\n    npx expo start\n   ```\n\nIn the output, you'll find options to open the app in a\n\n- [development build](https://docs.expo.dev/develop/development-builds/introduction/)\n- [Android emulator](https://docs.expo.dev/workflow/android-studio-emulator/)\n- [iOS simulator](https://docs.expo.dev/workflow/ios-simulator/)\n- [Expo Go](https://expo.dev/go), a limited sandbox for trying out app development with Expo\n\nYou can start developing by editing the files inside the **app** directory. This project uses [file-based routing](https://docs.expo.dev/router/introduction).\n\n\n3. Set up environment variables:\nCreate a `.env` file in the project root with the following:\n\n```\nEXPO_PUBLIC_APP_HOST=\"\"\nEXPO_PUBLIC_SHOPIFY_API_SECRET=\"\"\nEXPO_PUBLIC_SHOPIFY_DOMAIN=\"example.myshopify.com\"\n```\n\n## Configuration\n\n### Google Cloud Platform Vision API\n1. Create a GCP project\n2. Enable Vision API\n3. Generate and download service account credentials\n\n### Algolia\n1. Create an Algolia account\n2. Set up your product index\n3. Configure search settings\n\n### Shopify Storefront API\n1. Create a Shopify development store\n2. Set up Shopify Storefront API\n3. Get your API key\n\n## Running the App\n\n### iOS\n```bash\nnpm run ios\n```\n\n### Android\n```bash\nnpm run android\n```\n## Workflow\n\n1. The user uploads a product photo\n2. GCP Vision API analyzes the image\n3. Extracted tags/labels sent to Algolia\n4. Algolia returns matching products\n5. Results displayed to the user\n\n## Server Setup\n\nI used Remix.js [Cloudflare Template](https://remix.run/resources/remix-worker-template)\n\nI made an API route for searching using an image in `base64` format using [Algolia Search API](https://www.algolia.com/doc/rest-api/search/) which I used before to index our Shopify product image with image labels we got from Google Cloud Vision API. I used the Shopify product variant id and product id as an index.\n\n```tsx\n// app/routes/search.tsx\nimport type { LoaderFunctionArgs } from '@remix-run/cloudflare';\nimport { json } from '@remix-run/react';\nimport {\n\tgetImageLabels,\n\treduceLabelsToFilters,\n} from '../services/vision.server';\nimport { algoliaClient } from '../services/algolia.server';\n\nexport async function action({ request }: LoaderFunctionArgs) {\n\tconst formData = await request.formData();\n\tconst image = formData.get('image') as string; // Image Base64\n\tif (!image) {\n\t\treturn json({ error: 'No image provided' }, { status: 400 });\n\t}\n\n\tconst classifiedImage = await getImageLabels(image);\n\tconst labels = reduceLabelsToFilters(classifiedImage.labels);\n\n\tconst indexName = 'products';\n\tconst { results } = await algoliaClient.search({\n\t\trequests: [\n\t\t\t{\n\t\t\t\tindexName,\n\t\t\t\toptionalFilters: labels,\n\t\t\t\tquery: classifiedImage.labels.map(label =\u003e label.description).join(' '),\n\t\t\t},\n\t\t],\n\t});\n\n\t// ObjectID is a unique identifier for each image in Algolia, Example: \"1234-54355\", first part is the product ID and the second part is the variant ID\n\t// We're indexing each variant image separately, and we're using the product ID to get parent product information\n\tconst products = results[0].hits.map(\n\t\tproduct =\u003e product.objectID \u0026\u0026 product.objectID.split('-')[0],\n\t);\n\t\n\n\treturn json({ results: products });\n}\n```\n\nHere's our [Algolia Search API](https://www.algolia.com/doc/rest-api/search/) client:\n\n```tsx\n// app/services/algolia.server.ts\nimport { searchClient } from '@algolia/client-search';\nimport { configDotenv } from 'dotenv';\nconfigDotenv({\n\tpath: '.dev.vars',\n});\nif (!process.env.ALGOLIA_APP_ID) {\n\tthrow new Error(\n\t\t'Missing Algolia App ID. Please provide it in the .env file.',\n\t);\n}\nif (!process.env.ALGOLIA_SEARCH_KEY) {\n\tthrow new Error(\n\t\t'Missing Algolia Search Key. Please provide it in the .env file.',\n\t);\n}\nexport const algoliaClient = searchClient(\n\tprocess.env.ALGOLIA_APP_ID,\n\tprocess.env.ALGOLIA_SEARCH_KEY,\n);\n\n```\n\nHere's our Shopify Storefront API client:\n\n```tsx\n// app/services/shopify.server.ts\n\nimport { createStorefrontApiClient } from '@shopify/storefront-api-client';\nimport { configDotenv } from 'dotenv';\nconfigDotenv({\n\tpath: '.dev.vars',\n});\nif (!process.env.STORE_DOMAIN) {\n\tthrow new Error('Missing Store Domain. Please provide it in the .env file.');\n}\nif (!process.env.STOREFRONT_ACCESS_TOKEN) {\n\tthrow new Error(\n\t\t'Missing Storefront Access Token. Please provide it in the .env file.',\n\t);\n}\n\nexport const ShopifyClient = createStorefrontApiClient({\n\tstoreDomain: process.env.STORE_DOMAIN,\n\tpublicAccessToken: process.env.STOREFRONT_ACCESS_TOKEN,\n\tapiVersion: '2024-10',\n});\n\n```\nAnd our Google Cloud Vision API to get image labels by passing an image in `base64` format\n\n```tsx\n// app/services/vision.server.ts\n\nimport vision from '@google-cloud/vision';\nimport { configDotenv } from 'dotenv';\nconfigDotenv({\n\tpath: '.dev.vars',\n});\nif (!process.env.GOOGLE_APPLICATION_CREDENTIALS_BASE64) {\n\tthrow new Error(\n\t\t'Missing Google Application Credentials. Please provide it in the .env file.',\n\t);\n}\n\nconst credentials = JSON.parse(\n\tBuffer.from(\n\t\tprocess.env.GOOGLE_APPLICATION_CREDENTIALS_BASE64,\n\t\t'base64',\n\t).toString('utf-8'),\n);\n\nconst client = new vision.ImageAnnotatorClient({\n\tcredentials,\n});\n\ntype Label = {\n\tdescription: string | null | undefined;\n\tscore: number | null | undefined;\n};\nexport const getImageLabels = async (imageBase64: string) =\u003e {\n\tconst [result] = await client.annotateImage({\n\t\timage: {\n\t\t\tcontent: imageBase64,\n\t\t},\n\t\tfeatures: [\n\t\t\t{\n\t\t\t\ttype: 'LABEL_DETECTION',\n\t\t\t\tmaxResults: 10,\n\t\t\t},\n\t\t],\n\t});\n\tif (!result.labelAnnotations) {\n\t\treturn { labels: [] };\n\t}\n\n\tconst labels = result.labelAnnotations\n\t\t.filter(label =\u003e label.score \u0026\u0026 label.score \u003e 0.5)\n\t\t.map(label =\u003e ({\n\t\t\tdescription: label.description,\n\t\t\tscore: label.score,\n\t\t}));\n\treturn { labels };\n};\n\nexport const reduceLabelsToFilters = (labels: Label[]) =\u003e {\n\tconst optionalFilters = labels.map(\n\t\tlabel =\u003e `labels.description:'${label.description}'`,\n\t);\n\treturn optionalFilters;\n};\n\n\n```\n## Directory Structure\n```\nshopify-search-app/\n│\n├── app/                    # App configuration and entry points\n│   ├── _layout.tsx         # Navigation layout\n│   └── (tabs)/             # Grouped tab navigation\n│       ├── _layout.tsx     # Navigation layout\n│       └── index.tsx       # Main Entry view\n│   └──  +not-found.tsx     # Not found view\n\n├── assets/                 # Static assets\n│   ├── images/\n│   ├── fonts/\n│\n├── components/             # Reusable UI components\n│   │   ├── Button.tsx\n│   │   ├── Container.tsx\n│   │   ├── Text.tsx\n│   │   ├── Card.tsx\n│   │   ├── QueryClient.tsx\n│   │   ├── ProductList.tsx\n│   │   ├── ProductCard.tsx\n│   │   ├── SearchBar.tsx\n│   │   ├── SearchResults.tsx\n│   │   ├── SelectedImagePreview.tsx\n│   │   ├── NotFound.tsx\n│   │   ├── Grid.tsx\n│   │   ├── ExternalLink.tsx\n│   │   ├── HapticTab.tsx\n│   │   ├── LoadingResults.tsx\n│   │   └── SearchImageResults.tsx\n│\n├── constants/              # App-wide constants\n│   ├── Colors.ts\n│   ├── Query.ts\n│\n├── hooks/                  # Custom React hooks\n│   ├── useHomeViewState.ts\n│   ├── useProducts.ts\n│   ├── useImageSearch.ts\n│   └── useProductSearch.ts\n│\n├── navigation/             # Navigation configuration\n│   ├── AppNavigator.tsx\n│   └── types.ts\n│\n├── screens/                # Full screen components\n│   ├── HomeScreen.tsx\n│   ├── LoginScreen.tsx\n│   └── ProfileScreen.tsx\n│\n├── services/               # API and external service calls\n│   ├── search.ts\n│   ├── shopify.ts\n│   └── StorageService.ts\n│\n│\n├── types/                  # TypeScript type definitions\n│   └──  shopify.ts\n│\n│\n├── .env                    # Environment variables\n├── app.json                # Expo app configuration\n├── package.json            # Project dependencies and scripts\n├── tsconfig.json           # TypeScript configuration\n└── README.md               # Project documentation\n```\n\n## Future Roadmap\n\n- [ ] Advanced ML-powered visual search\n\n## Contributing\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n## License\n\nDistributed under the MIT License. See `LICENSE` for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filiashad%2Fvisual-product-search-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Filiashad%2Fvisual-product-search-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filiashad%2Fvisual-product-search-app/lists"}