{"id":40816984,"url":"https://github.com/coatsnmore/keeping-your-intelligence-local","last_synced_at":"2026-01-21T21:42:27.129Z","repository":{"id":271027581,"uuid":"912161668","full_name":"coatsnmore/keeping-your-intelligence-local","owner":"coatsnmore","description":"Teaching how to run AI locally.","archived":false,"fork":false,"pushed_at":"2025-08-14T03:27:12.000Z","size":26858,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-14T04:27:05.872Z","etag":null,"topics":["agentic-ai","ai","presentation","tutorial"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coatsnmore.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-04T19:20:31.000Z","updated_at":"2025-08-14T03:27:15.000Z","dependencies_parsed_at":"2025-08-06T04:18:28.179Z","dependency_job_id":null,"html_url":"https://github.com/coatsnmore/keeping-your-intelligence-local","commit_stats":null,"previous_names":["coatsnmore/keeping-your-intelligence-local"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/coatsnmore/keeping-your-intelligence-local","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatsnmore%2Fkeeping-your-intelligence-local","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatsnmore%2Fkeeping-your-intelligence-local/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatsnmore%2Fkeeping-your-intelligence-local/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatsnmore%2Fkeeping-your-intelligence-local/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coatsnmore","download_url":"https://codeload.github.com/coatsnmore/keeping-your-intelligence-local/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coatsnmore%2Fkeeping-your-intelligence-local/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28644149,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T21:29:11.980Z","status":"ssl_error","status_checked_at":"2026-01-21T21:24:31.872Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic-ai","ai","presentation","tutorial"],"created_at":"2026-01-21T21:42:26.618Z","updated_at":"2026-01-21T21:42:27.122Z","avatar_url":"https://github.com/coatsnmore.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Keeping Your Intelligence Local (and Cheap)\n\n## Preprequisites\n\n* Command Line Tools (only if you don't use Docker): `jq`, `bc`, `git`, `base64`, `node`, `curl`\n* Docker Engine (or Podman)\n* Docker Compose or equivalent client (e.g. `docker-compose`, `podman`)\n* Docker Socket access (only worry about this if something breaks - OpenWebUI assumes socket access by default)\n* general HTTP egress acccess (fetches model and container binaries)\n\n## Project Structure\n\n```\n.\n├── collections/     # Bruno API collections for testing\n├── models/         # Modelfiles for custom models\n├── requests/       # JSON request templates\n│   ├── function-calling.json    # Function calling examples\n│   └── structured-request.json  # Structured output examples\n├── resources/      # Static resources\n│   ├── cats.base64             # Test image for vision models\n│   ├── cats.jpeg              # Original test image\n│   └── technical-documentation.md  # Test document for embeddings\n└── src/           # Node.js test scripts\n    ├── cats.js         # Vision model testing\n    ├── code.js         # Code generation testing\n    ├── embed.js        # Embedding generation\n    ├── function-calling.js  # Function calling tests\n    ├── inspect.js      # Model inspection\n    ├── load-model.js   # Model loading utility\n    ├── structured.js   # Structured output testing\n    ├── test.js         # Basic generation testing\n    └── utils.js        # Shared utilities\n```\n\n## Clone the Repo\n\n```bash\n# Clone this repo with \ngit clone https://github.com/coatsnmore/keeping-your-intelligence-local\n```\n\n## Run the Docker Compose Stack and Attach to the Workspace Container\n\n```bash\n# start ollama, open webui, and a shell environment with some tooling\ndocker compose up --build -d\n\n# check out the deployed containers\ndocker ps\n\n# attach to the intelligence container\ndocker exec -it intelligence /bin/bash\n\n# after you are done, you can turn this off with\n# docker compose down\n```\n\n## Available Scripts\n\nRun these scripts from within the intelligence container:\n\n```bash\n# load the models\nnpm run load tinyllama\nnpm run load llava\nnpm run load all-minilm\n\n# test the model\nnpm run test tinyllama\n\n# test image recognition\nnpm run cats llava\n\n# test function calling\nnpm run function tinyllama\n\n# test structured output\nnpm run structured tinyllama\n\n# test embeddings\nnpm run embed\n```\n\n\n## Open Web UI\n\n1. Open the Open Web UI application in your browser at [`http://localhost:3000`](http://localhost:3000)\n1. Sign up an account. This stays local, but use credentials you don't care about. Use `admin@admin.com/admin`\n\n## References\n\n### General Knowledge\n\n* [Blog - Reasoning with Ollama](https://heidloff.net/article/reasoning-ollama/)\n* [Youtube - Matt Berman](https://www.youtube.com/@matthew_berman)\n* [Youtube - Matt Wolfe](https://www.youtube.com/@mreflow)\n* [Youtube - Matt Williams](https://www.youtube.com/@technovangelist)\n\n### Technology and Tools\n\n* [Ollama - API Docs](hhttps://github.com/ollama/ollama/blob/main/docs/api.md)\n* [QwQ - Qwen with Questions](https://qwenlm.github.io/blog/qwq-32b-preview/)\n* [Ollama with Spring Boot](https://docs.spring.io/spring-ai/reference/api/chat/ollama-chat.html#:~:text=Ollama%20is%20OpenAI%20API%2Dcompatible,openai.)\n* [Ollama Function Calling aka Format JSON](https://www.youtube.com/watch?v=RXDWkiuXtG0)\n\n\n## Some Common Issues\n\n```bash\n# clean it up\nsudo docker system prune --all --volumes`\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoatsnmore%2Fkeeping-your-intelligence-local","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoatsnmore%2Fkeeping-your-intelligence-local","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoatsnmore%2Fkeeping-your-intelligence-local/lists"}