https://github.com/phelps-sg/llm-cooperation

Code and materials for the paper S. Phelps and Y. I. Russell, Investigating Emergent Goal-Like Behaviour in Large Language Models Using Experimental Economics, working paper, arXiv:2305.07970, May 2023
https://github.com/phelps-sg/llm-cooperation

ai-alignment ai-safety behavioral-economics economics experimental-economics experimental-psychology gametheory gpt-3 gpt-4 llm principal-agent-problem prisoners-dilemma social-dilemmas

Last synced: 5 months ago
JSON representation

Code and materials for the paper S. Phelps and Y. I. Russell, Investigating Emergent Goal-Like Behaviour in Large Language Models Using Experimental Economics, working paper, arXiv:2305.07970, May 2023

• Host: GitHub
• URL: https://github.com/phelps-sg/llm-cooperation
• Owner: phelps-sg
• License: mit
• Created: 2023-05-04T17:12:30.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2024-12-10T16:16:18.000Z (over 1 year ago)
• Last Synced: 2024-12-10T17:30:27.983Z (over 1 year ago)
• Topics: ai-alignment, ai-safety, behavioral-economics, economics, experimental-economics, experimental-psychology, gametheory, gpt-3, gpt-4, llm, principal-agent-problem, prisoners-dilemma, social-dilemmas
• Language: Python
• Homepage:
• Size: 11.2 MB
• Stars: 12
• Watchers: 4
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE

Awesome Lists containing this project

README

          
# llm-cooperation

[![GitHub Workflow Status](https://github.com/phelps-sg/llm-cooperation/actions/workflows/tests.yaml/badge.svg)](https://github.com/phelps-sg/llm-cooperation/actions/workflows/tests.yaml)

This repo contains code, explanations and results of experiments to ascertain the propensity of large-language models to cooperate in social dilemmas.  The experiments are described in the following papers.

S. Phelps and Y. I. Russell, *Investigating Emergent Goal-Like Behaviour in Large Language Models Using Experimental Economics*, working paper, May 2023, [arXiv:2305.07970](https://arxiv.org/abs/2305.07970)

S. Phelps and R. Rannson, *Of Models and Tin Men - a behavioural economics study of principal-agent problems in AI alignment using large-language models*, working paper, July 2023, [arXiv:2307.11137](https://arxiv.org/abs/2307.11137)

## Getting started

1. Install [miniforge](https://github.com/conda-forge/miniforge#mambaforge).

2. In a shell:

~~~bash

export OPENAI_API_KEY=''

make install

make run

~~~

## Configuration

To run specific experiments and parameter combinations follow instructions below.

1. In a shell:

~~~bash

mkdir ~/.llm-cooperation

cat > ~/.llm-cooperation/llm_config.py << EOF

grid = {

        "temperature": [0.1, 0.6],

        "model": ["gpt-3.5-turbo", "gpt-4"],

        "max_tokens": [300]

}

num_replications = 3

experiments = ["dictator", "dilemma"]

EOF

~~~

2. Edit `$HOME/.llm-cooperation/llm_config.py` with required values.

3. In a shell:

~~~bash

export OPENAI_API_KEY=''

make run

~~~

## Contributing

If you have a new experiment then please submit a pull request.

All code should have corresponding tests and all experiments should be replicable.