https://github.com/liuhuapiaoyuan/chatgptcopilot

ChatGPTCopilot
https://github.com/liuhuapiaoyuan/chatgptcopilot

Last synced: about 1 month ago
JSON representation

ChatGPTCopilot

Host: GitHub
URL: https://github.com/liuhuapiaoyuan/chatgptcopilot
Owner: liuhuapiaoyuan
Created: 2024-01-15T01:05:25.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2024-01-15T01:06:23.000Z (over 1 year ago)
Last Synced: 2025-02-17T12:18:46.699Z (4 months ago)
Language: Python
Size: 182 KB
Stars: 3
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# localCopilot

[donate here](https://sahem.ksrelief.org/Pages/ProgramDetails/1ca8852b-9e6d-ee11-b83f-005056ac5498)

![](https://user-images.githubusercontent.com/37570492/212965203-c9623e27-4fff-4961-a7f4-4d14625dd17c.gif)

This project allows you to host your own GitHubCopilot-like model locally while using the official GitHubCopilot VSCode extension.

## Installation:

1. Download and install the **[oobabooga](https://github.com/oobabooga/text-generation-webui#installation)** backend

2. **Download a model**

open the oobabooga UI, go to the **models** tab and download a code completion model. I'm using: `Deci/DeciCoder-1b`, paste that name, then click download, then click load once complete

**Which model should I choose?**
Use smaller models for faster predictions, especially if you have a weaker PC.
I tested DeciCoder-1b

![](https://www.1552.cn/wp-content/uploads/2023/06/03a46779cf1ed51-17.gif)

Optional testing

A. (optional) Test the backend using `curl`:

```sh
curl -X 'POST' 'http://localhost:5000/v1/engines/codegen/completions' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{"prompt":"def hello_w","suffix":"","max_tokens":500,"temperature":0.4,"top_p":1,"n":10,"stop":["\ndef ","\nclass ","\nif ","\n\n#"],"logprobs":2,"stream":true}'
```

B. (optional) Test that the model is working by going to the "chat" tab and clicking "generate".
![](https://www.1552.cn/wp-content/uploads/2023/06/839f671a9313274-1.gif)

3. Go to VSCode and modify the settings and add the following:

```json
"github.copilot.advanced": {
"debug.overrideEngine": "codegen",
"debug.testOverrideProxyUrl": "http://localhost:8000", // address:port of middleware
"debug.overrideProxyUrl": "http://localhost:8000",
},
```

4. (optional for authentication) Update `~/.vscode/extensions/github.copilot-*/dist/extension.js` with the following:
- Replace `https://api.github.com/copilot_internal` with `http://127.0.0.1:8000/copilot_internal`
- replace `https://copilot-proxy.githubusercontent.com` with `http://127.0.0.1:8000`

5. Run the proxy:

```sh
pip install git+https://github.com/FarisHijazi/localCopilot
localCopilot --port 7000
```

If you have oobabooga running on a separate server use the --backend argument {hostname:port}
```sh
pip install git+https://github.com/FarisHijazi/localCopilot
localCopilot --port 8000 --backend http://10.0.0.1:5002
```

(Optional): testing the middleware

```sh
curl -X 'POST' 'http://localhost:8000/v1/engines/codegen/completions' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{"prompt":"def hello_w","suffix":"","max_tokens":500,"temperature":0.4,"top_p":1,"n":2,"stop":["\ndef ","\nclass ","\nif ","\n\n#"],"logprobs":2,"stream":true}'
```

expected output

```sh
data: {"id": "conv-1692741316942825472", "object": "text_completion.chunk", "created": 1692741316, "model": "Deci_DeciCoder-1b", "choices": [{"index": 0, "finish_reason": "stop", "text": "", "logprobs": {"top_logprobs": [{"<|endoftext|>": -0.4215908944606781, "": -1.2965909242630005, "\n": -3.0741329193115234}]}}], "usage": {"prompt_tokens": 4, "completion_tokens": 13, "total_tokens": 17}}

data: [DONE]
```

6. install the [official GitHub copilot extension](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot)

7. HAPPY CODING!

To test that the copilot extension is working, either type some code and hope for a completion
or use the command pallet (`Ctrl+Shift+P`) and search for `GitHub Copilot: Open Completions Panel`

---

## How this works

This is done using a single script: `localCopilot/middleware.py` (only 90 lines of code), which is a compatibility layer between the official GitHub copilot VSCode extension and [oobabooga](https://github.com/oobabooga/text-generation-webui) as a backend.

Credit: I learned about the traffic redirecting from the Fauxpilot project [here](https://github.com/fauxpilot/fauxpilot/blob/main/documentation/client.md#copilot-plugin).

Advanced experimental hacks

The tokenizers used by Copilot are not the same, so you can overwrite them. However, I'm not sure how useful this actually is as I don't notice any change in performance

```sh
COPILOTPATH=$HOME/.vscode/extensions/github.copilot-1.105.353
MODELPATH=$HOME/Projects/oobabooga_linux/text-generation-webui/models/Deci_DeciCoder-1b

mv $COPILOTPATH/dist/resources $COPILOTPATH/dist/resources.backup
mkdir -p $COPILOTPATH/dist/resources/cushman001
mkdir -p $COPILOTPATH/dist/resources/cushman002

cp $MODELPATH/tokenizer.json $COPILOTPATH/dist/resources/cushman001/tokenizer_cushman001.json
cp $MODELPATH/merges.txt $COPILOTPATH/dist/resources/cushman001/vocab_cushman001.bpe

cp $MODELPATH/tokenizer.json $COPILOTPATH/dist/resources/cushman002/tokenizer_cushman002.json
cp $MODELPATH/merges.txt $COPILOTPATH/dist/resources/cushman002/vocab_cushman002.bpe

```

And to revert your changes, just uninstall and reinstall the extension.

OR:

```sh
rm -rf $COPILOTPATH/dist/resources
mv $COPILOTPATH/dist/resources.backup $COPILOTPATH/dist/resources
```

## 😁 Benefits of self-hosting

1. 🔏 **Privacy**: No more sending your code to the cloud! This is the main benefit especially for enterprise. No code is sent to the cloud when self-hosting since everything runs on your machine(s).
2. 🌐 **Works without internet**: use it on the plane! ✈️
3. 💰 **Free**: No need to pay for your monthly subscription

## 😞 Limitations

1. GitHub copilot looks at multiple files for context. The current hack only looks at the current file
2. **Quality** Open source models might not have suggestions as good as copilot, but still as good *most* of the time
3. GitHub copilot gives 10 suggestions, while this hack gives only 1 suggestion per completion
4. 🐛 There's a bug where the first space in autocompletion is skipped, this is due to the oobabooga backend, not the model

## Why is this project setup this way?

There are many other projects for having an open source alternative for copilot, but they all need so much maintenance, I tried to use an existing large project that is well maintained: [oobabooga](https://github.com/oobabooga/text-generation-webui), since it supports almost all open source LLMs and is commonly used, and is well maintained

I know that the middleware method might not be optimal, but this is a minimal hack that's easy to run, and this repository should be really easy to maintain.

Once oobabooga supports multiple requests in a single call, then the middleware should no longer be needed.

## Other works

Here are some helpful open source projects I found while doing my research:

### 以下问题需要修复：
1. "github.copilot.advanced": {
"debug.overrideEngine": "codegen",
"debug.filterLogCategories": [],
"debug.overrideCapiUrl": "http://127.0.0.1:8000",
"debug.overrideProxyUrl": "http://127.0.0.1:8000"
},

2. 增加环境变量
3. 实例化环境变量

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/liuhuapiaoyuan/chatgptcopilot

Awesome Lists containing this project

README