https://github.com/charmve/gpt-eyes
I GAVE GPT-4 EYES!
https://github.com/charmve/gpt-eyes
agent gpt-4 gpt-4o gpt-4omni gpt4 gpt4v llm llm-inference world-model world-models worldmodel
Last synced: 8 months ago
JSON representation
I GAVE GPT-4 EYES!
- Host: GitHub
- URL: https://github.com/charmve/gpt-eyes
- Owner: Charmve
- Created: 2023-06-24T03:27:52.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-01-24T05:40:33.000Z (over 2 years ago)
- Last Synced: 2025-09-29T06:26:40.797Z (9 months ago)
- Topics: agent, gpt-4, gpt-4o, gpt-4omni, gpt4, gpt4v, llm, llm-inference, world-model, world-models, worldmodel
- Language: JavaScript
- Homepage:
- Size: 13.8 MB
- Stars: 14
- Watchers: 2
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
Awesome Lists containing this project
README
# GPT Eyes
I gave GPT-4 eyes. "眼观六路，耳听八方"
Here’s what I did:
- added some data to a vision model
- gave the AI camera access
- asked it questions about the scene
- it identified objects
- it searched web for info
- used that info to accurately answer
Watch it get 3 questions 100% correct!
- Twitter https://twitter.com/mckaywrigley/status/1651291367224807424?s=20
- YouTube https://www.youtube.com/watch?v=w-wxguIs-0I
## This Package Is Sponsorware 💰💰💰
[](https://github.com/sponsors/Charmve?frequency=one-time&sponsor=Charmve) https://github.com/sponsors/Charmve?frequency=one-time&sponsor=Charmve
This repo was only available to my sponsors on GitHub Sponsors until I reached 15 sponsors.
Learn more about **Sponsorware** at [github.com/sponsorware/docs](https://github.com/sponsorware/docs) 💰.
## Technologies Used
- Frontend: React
- Image Analysis API: TensorFlow Models - MobileNet
- Text Generation API: GPT API
## Installation
1. Clone the repository: `git clone https://github.com/Charmve/gpt-eyes.git`
2. Navigate to the project directory: `cd gpt-eyes`
3. Install dependencies: `npm install`
## Configuration
1. Create an account and obtain API keys for TensorFlow Models - MobileNet and GPT API.
2. Update the configuration file with your API keys:
- TensorFlow Models - MobileNet: `/path/to/config.js`
- GPT API: `/path/to/config.js`
## Usage
1. Start the development server: `npm start`
2. Open your browser and visit: `http://localhost:3000`
## How it Works
1. Device camera analyses an image.
2. The application uses TensorFlow Models - MobileNet API to analyze the image and extract object information.
3. The application sends the analyzed object information to the GPT API.
4. The GPT API generates text describing the analyzed object.
5. The application displays the analyzed image and the generated text.