{"id":13619320,"url":"https://github.com/0dB/dayone-json-to-obsidian","last_synced_at":"2025-04-14T16:31:12.823Z","repository":{"id":50508735,"uuid":"519288057","full_name":"0dB/dayone-json-to-obsidian","owner":"0dB","description":"Update Obsidian vault from Day One (“DayOne”) JSON using command line scripts.","archived":false,"fork":false,"pushed_at":"2022-08-20T19:33:45.000Z","size":8,"stargazers_count":25,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-06T22:11:29.862Z","etag":null,"topics":["awk","bash","jq","json","markdown","obsidian"],"latest_commit_sha":null,"homepage":"","language":"Awk","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/0dB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-29T16:55:48.000Z","updated_at":"2025-03-19T06:08:14.000Z","dependencies_parsed_at":"2022-08-12T21:40:19.140Z","dependency_job_id":null,"html_url":"https://github.com/0dB/dayone-json-to-obsidian","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0dB%2Fdayone-json-to-obsidian","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0dB%2Fdayone-json-to-obsidian/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0dB%2Fdayone-json-to-obsidian/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/0dB%2Fdayone-json-to-obsidian/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/0dB","download_url":"https://codeload.github.com/0dB/dayone-json-to-obsidian/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248916593,"owners_count":21182836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awk","bash","jq","json","markdown","obsidian"],"created_at":"2024-08-01T21:00:38.073Z","updated_at":"2025-04-14T16:31:12.477Z","avatar_url":"https://github.com/0dB.png","language":"Awk","funding_links":[],"categories":["Awk"],"sub_categories":[],"readme":"# Update Obsidian vault from Day One (“DayOne”) JSON using command line scripts\n\nNEW: The scripts now also update the Obsidian vault with new Day One entries (those with new UUIDs).\n\nNOTE: Please only use these scripts if you have a grasp of what they do and they either do exactly what you need or you can edit them to your taste.\n\nThe Markdown files are named in the form `YYYY-MM-DDTHH-\u003cDay One UUID\u003e`.\nA \"title\" inside the file is not used for the filename but can easily be added.\nThe files will be moved into `Day One Export/YYYY/YYYY-MM` directories which are created as needed.\nAll known Day One attachment types are supported. The script creates Wiki-style links of the form `[[Link]]`. Obsidian will find the files.\n\nThe scripts need `awk` and `sed` (the usual suspects for text processing), `make`, all of which a Linux or Mac system will have if set up for development, and you will probably need to install `jq` (e. g. with `sudo apt install jq`) for reading and filtering JSON. If you develop on Windows you know how to get all this there, too.\n\nHere a description of the chosen approach, where each script does one thing, and each file is easy to read and understand and change:\n- The `update.awk` script uses some interim files created by two preceding calls (one with the UUIDs from the export file, the other with the UUIDs already in the file system) and creates a `.jq`script which then only has the new UUIDs for further processing.\n- The `journal.jq` script flattens the JSON file and creates a long Markdown file divided into segments for further processing. \n  Some segments will not appear in the final output but are solely for the following stages. Tags are processed and a meaningful set of Day One metadata is stored, but Obsidian will actually only make use of the tags.\n- The `journal.awk` script reads this file, uses the special data segments and the metadata for its own logic (creating filenames, replacing Day One links to attachments to Obsidian links, …), and creates one file per Day One entry.\n- The `move.sh` script looks at the outputted `.md` files and creates folders of the form `YYYY/YYYY-MM` as needed and moves the files.\n- An `.sed` script is used to fix some character escaping that Day One does.\n\nThe `.md` files have metadata (\"Front-matter\" in Obsidian). The Day One UUID is in the filename and inside the metadata as well.\nThis will allow for later\nprogramatically adding further data from `Journal.json` (weather, ...; so be sure\nto keep the `Journal.json` file around) and also creates unique file names.\nIf you won't be needing any of that and don't want long file names you could probably just shorten the UUID and things will still work fine.\n\nHow to use:\n\n1. Clone the project into a directory parallel to your Obsidian vault directory. (Both assumed to be in your home directory \"~\".)\n2. Inside the vault directory create a soft link to the makefile:\n\n        ln -s ~/dayone-json-to-obsidian/makefile makefile\n    \n3. Download the Day One JSON export ZIP file into the vault directory.\n4. Now depending on if this is the first time or an update, from inside your vault directory:\n    1. Call `make initial`, if this is the first time around.\n    2. Just call `make`, if this is an update, and remember, the zip file does not have to contain the whole history, just the latest entries with some overlap to be safe. Only entries with new  UUIDs will be imported.\n\nThe ZIP file will be deleted so you can repeat these steps every time you do a new export.\n\nCreating the markdown files is really fast and will only take a few seconds per 10,000 entries.\n\nMoving the files will take a few minutes per 10,000 files,\nprobably because 99.9% of the calls to `mkdir -p` are actually redundant. No optimization here yet but easy to come up with a script-based solution here.\n\nThe 'videos', 'audios', 'photos', 'pdfs' subfolders and their contents stay unchanged. (FYI, Day One uses the MD5 hash for the filenames.)\n\nChanges you can make:\n- The YYYY directories will be inside a “Day One Export” directory. Replace this value inside `move.sh` by a name of your choosing or by `.` if you want everything at the top level.\n- If you want `MM:SS` to be in the filename, you can add that.\n- If you don't want to use the MD5 value for the attachment filenames, and instead want to use the original filenames (where available) or the Day One `identifier` field,\nyou could modify the files here or later create a separate `.jq` and `.awk` script along the approach here to do that.\n\nObsidian will find all files even if you don't use path names, especially if they are unique like in the current mechanism chosen here.\n\nPlans:\n- Timestamp is currently still UTC. Might want to change this to timezone of the Day One entry (stored in the metadata / front-matter).\n- Make the script more robust so that if the EOF separator is used inside text that the process doesn't trip over that.\n\nThanks to the following work\n\n* [https://github.com/ze-kel/DayOne-JSON-to-MD/blob/main/app.py](https://github.com/ze-kel/DayOne-JSON-to-MD/blob/main/app.py)\n* [https://github.com/quantumgardener/dayone-to-obsidian/blob/master/splitfile.py](https://github.com/quantumgardener/dayone-to-obsidian/blob/master/splitfile.py)\n* [https://github.com/edoardob90/dayone-to-obsidian/blob/main/utils.py](https://github.com/edoardob90/dayone-to-obsidian/blob/main/utils.py)\n* [https://github.com/arnaudlimbourg/dayone-to-obsidian/blob/main/utils.py](https://github.com/arnaudlimbourg/dayone-to-obsidian/blob/main/utils.py)\n\nthat I “compared notes” with (but decided on a different, Linux-style approach, and added the update feature).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0dB%2Fdayone-json-to-obsidian","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F0dB%2Fdayone-json-to-obsidian","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F0dB%2Fdayone-json-to-obsidian/lists"}