https://github.com/pablotor/aiawsinstace
A Terraform configuration for deploying an AI API on an AWS EC2 instance.
https://github.com/pablotor/aiawsinstace
ai ai-api-integration aws chatgpt ollama terraform
Last synced: 3 months ago
JSON representation
A Terraform configuration for deploying an AI API on an AWS EC2 instance.
- Host: GitHub
- URL: https://github.com/pablotor/aiawsinstace
- Owner: pablotor
- License: mit
- Created: 2025-03-08T18:12:05.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2025-03-09T21:46:45.000Z (3 months ago)
- Last Synced: 2025-03-09T22:27:11.305Z (3 months ago)
- Topics: ai, ai-api-integration, aws, chatgpt, ollama, terraform
- Language: HCL
- Homepage:
- Size: 6.84 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# AI AWS Instance
AI APIs are great. You can experiment with them, integrate them into your apps, and probably do a bunch of other things I haven’t even started to think about. But there’s a catch: all the ones I found are either paid or have an incredibly low usage limit. So, if you have some AWS credits to burn, this might help you out. And if you don’t, [ask for them!](https://pages.awscloud.com/GLOBAL_NCA_LN_ARRC-program-A300-2023.html)
This Terraform setup deploys an EC2 instance with Ollama and all the necessary infrastructure to use the Ollama API from the internet with a basic level of authentication. This is not a production-ready script, but it can be a good starting point.
## Pre-requisites
- An AWS account with some funds to burn
- An AWS user with an appropriate permission policy to manage EC2s instances, VPCs and DNS records
- [The AWS cli installed and set up](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
- [The Terraform cli installed and set up](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)
- [An SSH key generated WITHOUT A PASSPHRASE](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent)
- (Optional) Using a custom domain and SSL requires a preconfigured domain in AWS or NS records pointing to corresponding AWS NS servers
## Before deployment
1. Create your `terraform.tfvars` file from the example:
```bash
cp terraform.tfvars.example terraform.tfvars
```
2. Update the AWS profile, instance names, and both SSH key paths to match yours.
3. Create your secret.tfvars file from the example:
```bash
cp secret.tfvars.example secret.tfvars
```
4. Generate your API token and append it to secret.tfvars. The key must be enclosed in double quotes:
```bash
openssl rand -base64 32
```
5. Initialize Terraform:
```bash
terraform init
```
## How to deploy
From here, it’s all pretty straightforward. The only difference from other Terraform projects is that we’re adding the API key secret manually.
1. List all elements to be deployed:
```bash
terraform plan -var-file="secret.tfvars"
```
2. Then deploy them:
```bash
terraform apply -var-file="secret.tfvars"
```
The deployment will output your instance public ip. Save it for later.
3. And at some point, destroy them:
```bash
terraform destroy -var-file="secret.tfvars"
```
## Testing the deployment
If everything went well, you should be able to hit the Ollama API using your API key. Here’s a simple script to test connectivity and functionality from the terminal:
```bash
curl http:///api/chat \
-H "Authorization: Bearer " -d '{
"model": "",
"messages": [
{
"role": "user",
"content": "why is the sky blue?"
}
],
"stream": false
}'
```
## Debugging
If your requests aren’t working, you can log into the instance via SSH and check the Ollama and Nginx process statuses and configurations:
```bash
ssh -i ec2-user@
```
## How to use
I'm using the [OpenAI Node](https://github.com/openai/openai-node) package to interact with the API because the configuration is quite simple:
```js
import OpenAI from 'openai';
const openai = new OpenAI({
baseURL: 'http:///api',
apiKey: '',
});
async function main() {
const chatCompletion = await client.chat.completions.create({
messages: [{ role: 'user', content: 'Say this is a test' }],
model: 'tinyllama',
});
}
main();
```
Enjoy!
## What's comming
- User configuration
- Some proper key management would be nice