https://github.com/nightmachinery/random-shuffle-sgd

An implementation of the paper "How Good is SGD with Random Shuffling?"
https://github.com/nightmachinery/random-shuffle-sgd

Last synced: 3 months ago
JSON representation

An implementation of the paper "How Good is SGD with Random Shuffling?"

• Host: GitHub
• URL: https://github.com/nightmachinery/random-shuffle-sgd
• Owner: NightMachinery
• Created: 2023-07-11T13:42:27.000Z (almost 3 years ago)
• Default Branch: master
• Last Pushed: 2023-07-12T05:41:10.000Z (almost 3 years ago)
• Last Synced: 2024-12-31T17:48:08.456Z (over 1 year ago)
• Language: Python
• Size: 200 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.org

Awesome Lists containing this project

README

          
#+TITLE: random-shuffle-SGD

* Algorithms

#+ATTR_HTML: :width 900

[[file:readme.org_imgs/20230712_060555_UOdC4A.png]]

* Experiments

** One Seed

#+begin_example zsh

modes=(

  shuffle_once

  cyclic

  shuffle_all

  shuffle_all_replacement

)

batch_sizes=(64 1)

for bs in "${batch_sizes[@]}" ; do

    for mode in "${modes[@]}" ; do

        echo "mode=${mode}, batch_size=${bs}"

        

        python rsgd.py --batch-size "${bs}" --log-interval 10000 --epochs 10 --mode "$mode" --save-model auto --prefix-path "./results/batch_size_${bs}/"

    done

done

#+end_example

** Different Seeds

#+begin_example zsh

seeds=(

  1

  2

  3

  4

  5

)

modes=(

  shuffle_once

  cyclic

  shuffle_all

  shuffle_all_replacement

)

batch_sizes=(

512

# 64

# 1

)

for bs in "${batch_sizes[@]}" ; do

    for seed in "${seeds[@]}" ; do

        for mode in "${modes[@]}" ; do

            echo "mode=${mode}, batch_size=${bs}"

            python rsgd.py --seed "$seed" --batch-size "${bs}" --lr 0.1 --log-interval 10000 --epochs 10 --mode "$mode" --save-model auto --prefix-path "./results/batch_size_${bs}/seed_${seed}/"

        done

    done

done

#+end_example

#+begin_example zsh

python ./seed_averager.py | tee /dev/tty > results_avg_summary.txt

#+end_example

* Results

- [[./results.org]]