{"id":26815621,"url":"https://github.com/robinka/freq-domain-ar","last_synced_at":"2025-03-30T02:31:33.413Z","repository":{"id":284027088,"uuid":"953464573","full_name":"RobinKa/freq-domain-ar","owner":"RobinKa","description":"Autoregressive frequency-domain image generation","archived":false,"fork":false,"pushed_at":"2025-03-23T19:13:36.000Z","size":468,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-23T19:21:43.327Z","etag":null,"topics":["ai","auto-regressive","decoder","fourier","frequency","genai","image-generation","lightning","python","spectrum","torch","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RobinKa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-23T12:39:48.000Z","updated_at":"2025-03-23T19:13:40.000Z","dependencies_parsed_at":"2025-03-23T19:21:50.090Z","dependency_job_id":"74e406bd-d27e-4dc3-b034-62fbb29f52cc","html_url":"https://github.com/RobinKa/freq-domain-ar","commit_stats":null,"previous_names":["robinka/freq-domain-ar"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Ffreq-domain-ar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Ffreq-domain-ar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Ffreq-domain-ar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinKa%2Ffreq-domain-ar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RobinKa","download_url":"https://codeload.github.com/RobinKa/freq-domain-ar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246269063,"owners_count":20750210,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","auto-regressive","decoder","fourier","frequency","genai","image-generation","lightning","python","spectrum","torch","transformer"],"created_at":"2025-03-30T02:31:32.950Z","updated_at":"2025-03-30T02:31:33.396Z","avatar_url":"https://github.com/RobinKa.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Autoregressive frequency-domain image generation\n\nUses a decoder-only transformer to autoregressively generate images in the frequency domain.\n\n## How it works\n\nHere are some bulletpoints on how it works:\n\n- A few invertible transformations are done on the image to put them into frequency domain in a nice format for autoregressive prediction. These need to be undone again for plotting.\n  - fourier-transform without redundancy using rfft2\n  - log1p is applied to bring the values into a reasonable range\n  - flatten and sort by the magnitude of the frequency, because we want to start predicting the low frequencies first\n- prepend the label (0-9 for mnist) so we know what to generate\n- causal decoder-only transformer is trained in the usual way\n- the model uses a size 15 patchification layer\n\nThe current implementation uses mnist as a dataset. Note that no time was spent optimizing the architecture etc. and this is more of a proof of concept.\n\n## Examples\n\nBelow are some examples output where we autoregressively sample 15 pixels at a time (because that's the patchification size). The left part is the frequency domain.\n\nhttps://github.com/user-attachments/assets/2c3d4b83-f5b8-499e-bcbd-0bc6cb524cf1\n\nhttps://github.com/user-attachments/assets/697cae57-2faa-46a9-9966-c14fd63a3927\n\n## How to run\n\n- `uv sync` to install dependencies\n- `python -m freq_ar.train --log_every_n_steps 1000` to run\n\nHere's how the loss looks during training\n\n![](media/loss.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinka%2Ffreq-domain-ar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobinka%2Ffreq-domain-ar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinka%2Ffreq-domain-ar/lists"}