{"id":21637785,"url":"https://github.com/obazin/herodote","last_synced_at":"2026-04-05T21:37:17.437Z","repository":{"id":264492162,"uuid":"893524601","full_name":"obazin/herodote","owner":"obazin","description":"Command-line tool designed to save GPT-generated conversation data in md files","archived":false,"fork":false,"pushed_at":"2024-11-26T12:58:53.000Z","size":23,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-01T01:41:21.844Z","etag":null,"topics":["chat-gpt","cli-tool","data-structure","json","markdown","obsidian","openai-export","rust","serde"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/obazin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-24T17:03:37.000Z","updated_at":"2024-11-26T12:58:57.000Z","dependencies_parsed_at":"2024-11-24T18:19:03.932Z","dependency_job_id":"97faaac5-ed6c-4e4e-9422-ef1f9fc59750","html_url":"https://github.com/obazin/herodote","commit_stats":null,"previous_names":["obazin/herodote"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obazin%2Fherodote","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obazin%2Fherodote/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obazin%2Fherodote/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obazin%2Fherodote/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/obazin","download_url":"https://codeload.github.com/obazin/herodote/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244327528,"owners_count":20435438,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chat-gpt","cli-tool","data-structure","json","markdown","obsidian","openai-export","rust","serde"],"created_at":"2024-11-25T04:06:57.355Z","updated_at":"2025-12-31T00:12:44.698Z","avatar_url":"https://github.com/obazin.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Herodote\nThis Rust project is a command-line tool designed to process GPT-generated conversation data (in JSON format) and convert it into structured Markdown files. It is optimized for performance, modular, and easy to use, making it a reliable tool for archiving, publishing, or analyzing GPT conversations.\n\n### Features\n- JSON Parsing: Reads GPT conversation data stored in JSON format.\n- Markdown Export: Converts conversations into clean, human-readable Markdown files.\n- Parallel Processing: Uses multi-threading (via rayon) for efficient file writing, even with large datasets.\n- Customizable Output: Normalizes filenames and ensures compatibility with Markdown editors.\n- Error Handling: Handles file system and parsing errors gracefully.\n\n### Purpose\nGPT tools often generate structured JSON data containing user interactions and assistant responses. This project provides a simple way to transform that raw data into Markdown files, which can then be:\n- Archived for later reference.\n- Published in blogs or documentation.\n- Analyzed for research or development purposes.\n\nThis project is well-suited for developers, researchers, or writers working with GPT-generated data who want a streamlined solution for managing and exporting conversations.\n\n### Build \u0026 Run\nBuild the project: `cargo build`\nRun the tool: `cargo run -- -i path/to/input.json -o path/to/output/`\nTesting : `cargo test`\n\n### Usage\nPrerequisites\n- Rust installed on your machine (use Rustup to install).\n- A valid JSON file containing GPT interaction data.\n\nCommand-line Interface\nThis tool is designed to be used via the command line. Below are the supported options:\n```sh\nUSAGE:\n    herodote [OPTIONS]\n\nOPTIONS:\n    -i, --input \u003cFILE\u003e          Path to the input JSON file containing GPT conversations\n    -o, --output-folder \u003cDIR\u003e   Path to the folder where Markdown files will be saved\n    -h, --help                  Show this help message\n    -V, --version               Show version information\n```\n### Example Usage\nConvert a JSON file of GPT conversations into Markdown files:\n\n```bash\n./target/release/gpt-converter -i conversations.json -o output/\n```\nThis will:\n\nParse conversations.json.\nCreate Markdown files in the output/ directory, one file per conversation.\n\n### Input File Format\nThe tool expects a JSON file with the following structure:\n\n```json\n{\n  \"title\": \"Conversation Title\",\n  \"update_time\": 1672531200,\n  \"mapping\": {\n    \"1\": {\n      \"message\": {\n        \"author\": {\n          \"role\": \"user\"\n        },\n        \"content\": {\n          \"parts\": [\"Hello!\"]\n        },\n        \"create_time\": 1672531200\n      }\n    },\n    \"2\": {\n      \"message\": {\n        \"author\": {\n          \"role\": \"assistant\"\n        },\n        \"content\": {\n          \"parts\": [\"Hi! How can I assist you?\"]\n        },\n        \"create_time\": 1672531220\n      }\n    }\n  }\n}\n```\nEach conversation is represented as a mapping of interaction nodes, where:\n\n- title is the conversation title.\n- mapping contains the individual user and assistant messages.\n- update_time specifies the last update timestamp of the conversation.\n\n### Output Example\nFor the above JSON, the tool generates a Markdown file like this:\n\nFile: 2023-01-01-Conversation_Title.md\n```markdown\n# Conversation Title\n\n## Question\nHello!\n\n## Answer\nHi! How can I assist you?\n```\n### Key Points\n- Efficient Multi-threading: Uses rayon for concurrent file writing, ensuring scalability for large datasets.\n- Data Validation: Ensures only valid interactions (e.g., non-empty text) are processed.\n- Filename Normalization: Converts titles into safe, human-readable filenames.\n- Extensibility: The modular design allows easy integration with other tools or formats (e.g., HTML export).\n\n### Development\nDirectory Structure\n```shell\nsrc/\n├── conversation_writer.rs  # Handles Markdown file writing\n├── main.rs                 # CLI entry point\n├── model.rs                # Data structures mapping GPT export and target Model\n├── utils.rs                # Helper functions for filenames and dates\n```\n\n\n### Contributing\nContributions are welcome! If you find a bug, want to suggest a feature, or improve documentation, feel free to open an issue or pull request.\n\n### To-Do List\n- Add support for HTML export.\n- Implement logging instead of eprintln!.\n\n### License\nThis project is licensed under the MIT License. See the LICENSE file for details.\n\n### Acknowledgments\n- [Rayon](https://github.com/rayon-rs/rayon) for parallel processing.\n- [Serde](https://github.com/serde-rs/serde) for JSON parsing.\n- Rust community for its excellent libraries and tools.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobazin%2Fherodote","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fobazin%2Fherodote","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobazin%2Fherodote/lists"}