https://github.com/liyucheng09/creativity

https://github.com/liyucheng09/creativity

Last synced: 15 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/liyucheng09/creativity
• Owner: liyucheng09
• Created: 2023-05-18T23:55:36.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-05-03T16:02:10.000Z (8 months ago)
• Last Synced: 2024-10-30T03:50:23.490Z (2 months ago)
• Language: Python
• Size: 6.6 MB
• Stars: 0
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
# TO-DO List

## Data

- [x] Reddit crawler

- [x] subreddits

  - [x] BrainStorming

    - [x] r/AskEngineers

    - [x] r/financialindependence

    - [x] r/Entrepreneur

    - [x] r/smallbusiness

    - [x] r/lifehacks

    - [x] r/productivity

    - [x] r/GetMotivated

    - [x] r/GetStudying

    - [x] r/Cooking

  - [x] Creative Writing

    - [x] r/fantasywriters

    - [x] r/WritingPrompts

    - [x] r/ShortStories

    - [x] r/Jokes

  - [ ] modification of dataset

    - [ ] is text in QA pair better than pure text when considering creativity

    

## Models

- [x] Base LLM

- [x] LLMs with SFT and RLHF 

- [ ] Fine-tuned with creative dataset

## Decoding methods

### Exsisting/Baselines

- [x] repetition penalty

- [ ] [https://aclanthology.org/2023.acl-long.34.pdf](https://aclanthology.org/2023.acl-long.34.pdf)

- [ ] NS-FH [https://aclanthology.org/2023.acl-demo.6.pdf](https://aclanthology.org/2023.acl-demo.6.pdf)

- [ ] Contrastive decoding ? [https://openreview.net/pdf?id=V88BafmH9Pj](https://openreview.net/pdf?id=V88BafmH9Pj)

- [x] Samplaing temperature

### Ours

- [ ] semantic diverse beam search

- [ ] AI Feedbacks

- [ ] Random words in prompts

- [ ] Model ensembling

  - [ ] ranking

  - [ ] voting

- [ ] NLTK topic normalisation

## Metrics

- [ ] lexcial diversity

  - [ ] Diverse-N Grams

  - [ ] output distribution (entropy?) or Kurtosis/Skewness?

- [ ] semantic diversity

  - [ ] embedding model for semantic similarity comparison (SimCSE/SensentenceBERT?)

# Related Works

- [ ] [https://aclanthology.org/W19-2311.pdf](https://aclanthology.org/W19-2311.pdf)

- [ ] [https://aclanthology.org/2023.emnlp-main.31.pdf](https://aclanthology.org/2023.emnlp-main.31.pdf)

- [ ] [https://openreview.net/pdf?id=SJeYe0NtvH](https://openreview.net/pdf?id=SJeYe0NtvH)

- [ ] https://ceur-ws.org/Vol-3359/paper7.pdf

- [ ] https://arxiv.org/pdf/2305.08493.pdf

- [ ] https://arxiv.org/pdf/2311.01937.pdf

- [ ] https://arxiv.org/pdf/2312.02439.pdf

- [ ] https://ojs.aaai.org/index.php/AIIDE/article/view/27539/27312

- [ ] https://arxiv.org/pdf/2311.09682.pdf