{"id":39606558,"url":"https://github.com/compscifutures/delog","last_synced_at":"2026-01-18T08:01:07.008Z","repository":{"id":173300269,"uuid":"650532050","full_name":"CompSciFutures/delog","owner":"CompSciFutures","description":"JSON/XML logfile deserializer - converts unstructured CSV data to tabular","archived":false,"fork":false,"pushed_at":"2023-06-16T15:48:33.000Z","size":19277,"stargazers_count":1,"open_issues_count":17,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-05-02T04:20:10.220Z","etag":null,"topics":["csv","csv-files","cybersecurity","excel","infosec","infosectools","json","matlab","powerbi","siem","tableau","tabular","tabular-data","tabular-data-formatter","xml","xml-parser","xml-parsing","xml-serialization"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CompSciFutures.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-06-07T09:11:35.000Z","updated_at":"2023-06-16T15:03:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"40564363-37da-4f96-95f8-a6ba065d21fd","html_url":"https://github.com/CompSciFutures/delog","commit_stats":{"total_commits":16,"total_committers":2,"mean_commits":8.0,"dds":0.1875,"last_synced_commit":"34cf77f5a2893c6f491ef45f69988b402c95606a"},"previous_names":["compscifutures/delog"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/CompSciFutures/delog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompSciFutures%2Fdelog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompSciFutures%2Fdelog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompSciFutures%2Fdelog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompSciFutures%2Fdelog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CompSciFutures","download_url":"https://codeload.github.com/CompSciFutures/delog/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompSciFutures%2Fdelog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28533736,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T00:39:45.795Z","status":"online","status_checked_at":"2026-01-18T02:00:07.578Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","csv-files","cybersecurity","excel","infosec","infosectools","json","matlab","powerbi","siem","tableau","tabular","tabular-data","tabular-data-formatter","xml","xml-parser","xml-parsing","xml-serialization"],"created_at":"2026-01-18T08:00:40.445Z","updated_at":"2026-01-18T08:01:06.683Z","avatar_url":"https://github.com/CompSciFutures.png","language":"Java","readme":"# delog: JSON/XML logfile deserializer\n\n\u003e **_NOTE:_** **TEST DATASETS WANTED!! Do you have TDF/CSV data containing unstructured XML/JSON data that delog didn't handle? email me at ap@andrewprendergast.com and I'll update delog so it works with it automagically (See issue [https://github.com/CompSciFutures/delog/issues/10]).**\n\u003e \nThis tool will de-serialse/expand/explode any embedded XML/JSON data structures found in comma-separated (CSV) logfiles so that each element or field found in JSON or XML arrays or data structures found in the CSV file will be converted into separate, individual CSV columns for each.\n\nThis is particularly useful for reading logfiles that contain unstructured data into Excel, PowerBI, Tableau, Python, MATLAB and many other tools that expect tabular data.\n\nAlot of instrumentation for security operations, IoT devices, drones and data lakes is in the form of logfiles which contains a combination of tabular data \u0026 unstructured data. This tool addresses that complexity and allows one to more readily make use of these files, either to inspect them or for applications such as data science.\n\nA mix of JSON objects, JSON arrays and XML can all be combined into a single CSV file.\n\nTab-delimited is not currently implemented as it's support for multi-line isn't as well defined - let me know if there's a need for it.\n\n## Philosophy\n\n\u003ca href=\"https://x.com/CompSciFutures/status/1820016959133483324\"\u003e![image](https://github.com/user-attachments/assets/ae35ec0d-8de9-4bb0-a83f-3a6a0c958a3e)\u003c/a\u003e\n\n\n## Installing\n\nDownload the contents of the **dist** folder - the delog.jar is an uber-jar with everything you need.\n\nOr just grab the \u003ca href=\"https://github.com/CompSciFutures/delog/releases/download/HEAD/delog-1.0.0.zip\"\u003elatest release ZIP\u003c/a\u003e file from GitHub.\n\n## Example usage\n\n```text\njava -jar delog --include ColC --include ColB --include ColA --exclude ColD testdata.csv\n```\n\nOr on Windows:\n\n```text\ndelog.exe --include ColC --include ColB --include ColA --exclude ColD testdata.csv\n```\n\nWill convert the following CSV:\n\n```text\nColA,ColB,ColC,ColD\nd,e,f,\"      \u003cIngredient\u003e\n         \u003cQty unit=\"ml\"\u003e500\u003c/Qty\u003e\n         \u003cQty unit=\"g\"\u003e9\u003c/Qty\u003e\n         \u003cItem\u003eCottage cheese\u003c/Item\u003e\n      \u003c/Ingredient\u003e\"\n```\n\nInto:\n\n```text\n\"ColC\",\"ColB\",\"ColA\",\"Ingredient.Qty\",\"Ingredient.Qty2\",\"Ingredient.Item\",\"Ingredient\"\n\"f\",\"e\",\"d\",\"500\",\"9\",\"Cottage cheese\",\n```\n\nNote that JSON \u0026 XML are supported, so a mix of JSON objects, JSON arrays and XML can all be combined into a single CSV file.\n\n## Example: Office 365 Audit logs with embedded JSON\n\nHere is a typical exmaple of unstructured data in security logs: if you have Office 365 E3 or higher, you probably have access to audit logs in O365 admin. They are CSV logs, but they contain within them unstructured information about each event that's in the form of a JSON object inside a single CSV cell.\n\nDelog allows you to read them, easily.  To get the logs, follow these instructions.\n\n**1. Go to Office 365 Admin:**\n\n![O365-1.png](images%2FO365-1.png)\n\n**2. Click 'Show All'**\n\n![O365-2.png](images%2FO365-2.png)\n\n**3. Click 'Compliance'**\n\n![O365-3.png](images%2FO365-3.png)\n\n**4. Click 'Audit'**\n\n![O365-4.png](images%2FO365-4.png)\n\n**5. Select your search scope and click 'Search' then wait for your search to finish. Click 'Completed' once it's ready**\n\n![O365-5.png](images%2FO365-5.png)\n\nThere, you should get a CSV with embedded JSON representing unstructured data which can be run through **delog** before being able to inspect it in Excel. \n\n## Versioning\n\nWe use [SemVer](http://semver.org/) for versioning.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcompscifutures%2Fdelog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcompscifutures%2Fdelog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcompscifutures%2Fdelog/lists"}