{"id":13770697,"url":"https://github.com/Twigonometry/writeup-converter","last_synced_at":"2025-05-11T03:32:52.080Z","repository":{"id":129042394,"uuid":"363622138","full_name":"Twigonometry/writeup-converter","owner":"Twigonometry","description":"Script for grabbing markdown files and Obsidian attachments from one folder and copying them to another.","archived":false,"fork":false,"pushed_at":"2021-09-21T21:27:27.000Z","size":55,"stargazers_count":26,"open_issues_count":18,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-17T06:40:26.042Z","etag":null,"topics":["obsidian","obsidian-md","obsidian-vault","productivity","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Twigonometry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-02T10:20:40.000Z","updated_at":"2024-09-11T07:33:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"a516b448-c51c-48ce-a2ed-9d741ce91663","html_url":"https://github.com/Twigonometry/writeup-converter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Twigonometry%2Fwriteup-converter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Twigonometry%2Fwriteup-converter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Twigonometry%2Fwriteup-converter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Twigonometry%2Fwriteup-converter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Twigonometry","download_url":"https://codeload.github.com/Twigonometry/writeup-converter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253514352,"owners_count":21920327,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["obsidian","obsidian-md","obsidian-vault","productivity","python3"],"created_at":"2024-08-03T17:00:40.777Z","updated_at":"2025-05-11T03:32:51.680Z","avatar_url":"https://github.com/Twigonometry.png","language":"Python","funding_links":[],"categories":["Publishing","Python"],"sub_categories":[],"readme":"# writeup-converter\n\nA Python script for grabbing markdown files and Obsidian attachments from one folder and copying them to another. Also contains a 'website formatter' that uses regex to parse markdown headers and links and reformat them to create Jekyll-friendly links and contents tables.\n\nAlso contains a quick and dirty bash script that does the same thing with less pizzazz.\n\n## Why did I make this?\n\nI got sick of manually copying the images and attachments I'd used in a writeup when I moved from my private vault to my public one. There seems to be no official way of exporting a folder in Obsidian, so I made one myself.\n\nI use this to copy a writeup across to another folder, but you could in theory use it for copying any folder with attachments in it to any location.\n\nI may in future turn this into an Obsidian plugin (but no promises).\n\n## Installation\n\nUsing git:\n\n```\n$ git clone git@github.com:Twigonometry/writeup-converter.git\n```\n\n**Python**\n\nTo use the Python script (recommended) you also need to [install Python 3.x](https://www.python.org/downloads/). For example, on Ubuntu:\n\n```bash\n$ sudo apt install python3.8\n```\n\nThere are no dependencies to install as of writing this, as only core packages are used. But in future if your program will not run you can use pipreqs to generate a `requirements.txt` file locally and then install from it:\n\n```bash\n$ python3 -m pip install pipreqs\n$ pipreqs /path/to/writeup-converter\n$ python3 -m pip install -r /path/to/writeup-converter/requirements.txt\n```\n\n(I've mostly included this in the README because I thought it was cool and didn't want to forget it)\n\n**Bash Script**\n\nIf you're using the bash script, make it executable:\n\n```bash\n$ cd writeup-converter\n$ chmod +x writeup-converter\n```\n\n## Usage\n\n**General Notes on Arguments**\n\nPositional Arguments:\n- Source Folder: The writeup you want to copy. Copies the entire directory\n- Source Attachments: The path to the obsidian attachments folder where attachments are saved in your writeup\n- Target Folder: Where you want to copy the writeup to. No need to make the folder beforehand - copying `/path/to/source/Writeups/Hack\\ the\\ Box/Boxes/Blue/` to `/path/to/target/Writeups/Hack\\ the\\ Box/Boxes/` will create the `Blue` directory for you\n- Target Attachments: The path to the obsidian attachments folder in your new location\n\nOptional Arguments\n- `-r REMOVE_PREFIX` specifies a prefix to remove from the attachment links that are copied across - e.g. if writeups in source folder live in a subdirectory `/Cybersecurity`, internal links to `[[Cybersecurity/Writeups/...]]` will become `[[Writeups/...]]`\n- `-v VERBOSE` enables verbose mode, where all file names are outputted while copying. Can make the screen quite busy for a large directory\n- `-w` tells the script to format your files for a website. This will combine them all into a single markdown file and reformat links, as well as adding a contents section to replace the obsidian index\n- `-l` tells the script the relative path of your site's assets folder to use when creating image links when website formatting\n\n### Python\n\nUsing any command line tool that has Python installed with it:\n\n```bash\n$ python3 writeup-converter.py -h\nusage: writeup-converter.py [-h] [-a ADD_PREFIX] [-r REMOVE_PREFIX] [-v]\n                            source_folder source_attachments target_folder\n                            target_attachments\n\nTakes a folder of Obsidian markdown files and copies them across to a new\nlocation, automatically copying any attachments. Options available include\nconverting to a new set of Markdown files, removing and adding prefixes to\nattachments, and converting for use on a website\n\npositional arguments:\n  source_folder         The folder of markdown files to copy from.\n  source_attachments    The attachments folder in your Obsidian Vault that\n                        holds attachments in the notes.\n  target_folder         The place to drop your converted markdown files\n  target_attachments    The place to drop your converted attachments. Must be\n                        set as your attachments folder in Obsidian (or just\n                        drop them in the root of your vault if you hate\n                        yourself)\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -a ADD_PREFIX, --add_prefix ADD_PREFIX\n                        Prefix to add to all your attachment file paths.\n  -v, --verbose         Verbose mode. Gives details of which files are being\n                        copied. Disabled by default in case of large\n                        directories\n```\n\nFor example, when I copied my Cereal writeup:\n\n```bash\n$ python3 writeup-converter.py -r Cybersecurity \"/mnt/d/path/to/vault/Cybersecurity/Writeups/Hack the Box/Boxes/Cereal\" /mnt/d/path/to/vault/Attachments/ \"/mnt/d/OneDrive/OneDrive/Documents/Cybersecurity-Notes/Writeups/Hack the Box/Boxes/Cereal\" /mnt/d/OneDrive/OneDrive/Documents/Cybersecurity-Notes/Attachments/\n```\n\nFile paths with spaces in them must be wrapped in quotes. The program checks the source files exist before running, but it will create directories for targets if they don't exist:\n\n```bash\n$ python3 writeup-converter.py \"/home/user/file with a space\" /home/user/notreal /home/user/target/ /home/user/target-attachments/\nSource folder path (/home/user/file with a space) is not a directory. Exiting\n```\n\nThere's no need to also escape the quotes with `\\` characters - some terminals will do this automatically if you autocomplete, but these extra backslashes should be removed if they're added.\n\n#### Website Formatter\n\nTo copy a writeup but format it for a website, use the `--website` or `-w` flag. You must provide the name of the file to be outputted, but you can leave the rest of the options the same (where attachments folder here is an images directory etc rather than an obsidian attachments folder).\n\nI use this after my initial conversion - i.e. I use the normal script to copy from a private vault to a public one and editing out anything I don't want, then use the `-w` flag to send it to my website repository.\n\nThe formatter will perform the following operations:\n- concatenate all `.md` files it finds in the folder\n- turn links of form `[[x]]` into `\u003ca href=\"#x\"\u003ex\u003c/a\u003e`\n- turn links of form `[[x#y]]` into `\u003ca href=\"#y\"\u003ey\u003c/a\u003e`\n- turn links of form `[[x|z]]` into `\u003ca href=\"#x\"\u003ez\u003c/a\u003e`\n- turn links of form `[[x#y|z]]` into `\u003ca href=\"#y\"\u003ez\u003c/a\u003e`\n- turn links of form `![[a.png]]` into `\u003cimg src=\"/path/to/attachments/a.png\"\u003e`\n- any links to obsidian notes not part of the folder being copied will just have the `[[` and `]]` strings stripped (TODO)\n\nExample usage:\n\n```bash\npython3 writeup-converter.py -w 2021-06-12-htb-cereal.md -l /assets/images/blogs \"/path/to/Cybersecurity-Notes/Writeups/Hack the Box/Boxes/Cereal/\" \"/path/to/Cybersecurity-Notes/Attachments/\" \"/path/to/Personal Site/mac-goodwin.com/mac-goodwin/blog/HTB/_posts/\" \"/path/to/Personal Site/mac-goodwin.com/mac-goodwin/assets/images/blogs/\"\n```\n\nIMPORTANT NOTE: Sometimes copying large amount of files over to Jekyll folder while the server is running will crash the server and make it unresponsive to Ctrl+C, `pkill -9` etc. It's worth stopping serving before running the converter.\n\nYou may have to do a bit of manual work each time:\n- Add an initial title to the markdown file\n- Depending on how you number your files, the sections may be out of order (I commonly have `5 - Enumeration` and `10 - Website`, so Enumeration often ends up at the bottom)\n- Remove any markdown files you *don't* want to include (for example, I often have an index file for linking the obsidian notes which is unnecessary on a website)\n- Some links may not be turned into markdown links if they're just plain `http://...` links - this may be added as a feature in future\n- You may need to add an initial yaml if using a templating engine like Jekyll\n- Add/remove tags from the writeup as you see fit\n- Add image captions/alt text (i.e. inside the square brackets in a `![]()` tag)\n- Check all the links in the contents page work - it does a decent job, but can't always predict how element IDs will be generated, especially for ones with special characters\n- If you're using a templating engine like Liquid, you may have to escape certain characters (for example, using a `{% raw %}` and `{% endraw %}` tag around occurrences of `{{}}`)\n\n### Bash Script\n\n```bash\n$ ./writeup-converter [OPTIONS] [SOURCE_FOLDER] [SOURCE_ATTACHMENTS] [TARGET_FOLDER] [TARGET_ATTACHMENTS]\n```\n\nOptions:\n- `--help` displays usage\n\nIt's easiest to specify full paths. There is no need to wrap paths in quotation marks (it will actually break `grep` if you do), but you must escape spaces in paths with a backslash `\\`\n\nPrefix is not supported (it was easier to do it all in python at this point).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTwigonometry%2Fwriteup-converter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTwigonometry%2Fwriteup-converter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTwigonometry%2Fwriteup-converter/lists"}