{"id":13646140,"url":"https://github.com/eggsyntax/datawalk","last_synced_at":"2025-04-05T19:14:21.963Z","repository":{"id":62432246,"uuid":"107864690","full_name":"eggsyntax/datawalk","owner":"eggsyntax","description":"Interactively explore complex data structures at the REPL with minimum keystrokes","archived":false,"fork":false,"pushed_at":"2018-07-20T18:21:06.000Z","size":1559,"stargazers_count":277,"open_issues_count":1,"forks_count":2,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-29T18:08:41.088Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eggsyntax.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-22T12:40:45.000Z","updated_at":"2024-10-23T01:01:23.000Z","dependencies_parsed_at":"2022-11-01T21:00:45.265Z","dependency_job_id":null,"html_url":"https://github.com/eggsyntax/datawalk","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eggsyntax%2Fdatawalk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eggsyntax%2Fdatawalk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eggsyntax%2Fdatawalk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eggsyntax%2Fdatawalk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eggsyntax","download_url":"https://codeload.github.com/eggsyntax/datawalk/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247386266,"owners_count":20930619,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:02:49.296Z","updated_at":"2025-04-05T19:14:21.933Z","avatar_url":"https://github.com/eggsyntax.png","language":"Clojure","funding_links":[],"categories":["Clojure","Libraries"],"sub_categories":[],"readme":"# datawalk\n\nhttps://github.com/eggsyntax/datawalk\n\nProvides a simple interactive data explorer.\n\n```\n(datawalk.core/repl my-data-structure)\n```\n\nIn Clojure and ClojureScript, we spend a lot of time exploring and manipulating\ndata structures, sometimes enormous ones. I've whiled away innumerable hours at\nthe REPL navigating through complex nested data structures, often heavily-nested\nmaps with long namespaced keys. I prefer to minimize the amount of typing I have\nto do for repetitive tasks, and this is certainly one of those tasks.\n\nSo datawalk pretty-prints the top level of your data structure, and then lets\nyou drill down repeatedly, often with just a single keystroke (the number\nrepresenting the item's position in the printed list, as shown next to the\nitem) plus enter.\n\nOften the reason for drilling down at the REPL is to determine a path to pass to\n`get-in`, so datawalk will track and [p]rint the path from the original root to\nthe level you're on (as long as it's a viable get-in path, which it's not in the\ncase of sets and lists).\n\nAt any point in this process, you can [s]ave the current item or sa[v]e the\npath (from root to item) into a map of saved items which will be returned when\nyou [q]uit. You can also e[x]it and return just the current item.\n\nAs mentioned, you can [p]rint the path to current data. You can also print\n[c]urrent data (useful when you want to see it untruncated), print the [m]ap\nof saved data, or print [h]elp to get a reminder of these commands. Capital\nC and M give you pretty-printed versions of current and saved data.\n\nYou can also move [b]ackward or [f]orward in time, or move [u]p a level, or jump\nback to the original [r]oot, or use [!] (function) to call an arbitrary\nsingle-arg │ function on the current data (jumping to the result).\n\ndatawalk tries to do one thing well: eliminate tedium and typing when navigating\ndata structures. The learning curve should be trivial; please let me know if it's\nnot and I'll try to change that. I hope you find it as useful as I have.\n\n## Demo (fully-interactive mode):\n\n\n![Demo](resources/fully-interactive.gif?raw=true \"Demo\")\n\n\n## Installation:\n\nLeiningen:\n```\n[datawalk \"0.1.12\"]\n```\n\nDetails at https://clojars.org/datawalk\n\n## Usage summary:\n\n```\nClojure:\n  (datawalk.core/repl my-data-structure) to start datawalking.\nClojure and ClojureScript:\n  (look-at my-data-structure), and then (w [command])\n\nCommands:\n  numbers: Enter any number to jump to the corresponding item\n  q :quit              ; exit and return saved values if any\n  x :exit-with-current ; exit \u0026 return just this value\n  s :save-current      ; save to map of return values\n  v :save-path         ; save path to map of return values\n  b :backward          ; step backward in history\n  f :forward           ; step forward in history\n  r :root              ; jump back to original root\n  ! :function          ; call a 1-arg fn on data, jump to result (clj only)\n  u :up                ; step upward [provides list of referring entities]\n  h :help              ; print help \u0026 return same ent\n  p :print-path        ; print path from root to current item.\n  m :print-saved-map   ; print the map of saved data\n  c :print-full-cur    ; print the current data in full, without truncation\n```\n\n## Fully-interactive version (Clojure-only)\n\nStart the fully-interactive version with `(datawalk.core/repl my-data-structure)`.\nFrom there, you just enter a single character followed by enter -- a number\nto drill down to a specific item in the displayed list, `b` to step back to\nwhere you just were, `h` for help, and so on. It should be extremely self-\nexplanatory.\n\n## Semi-interactive version (Clojure and ClojureScript)\n\nTo use the semi-interactive version, let's assume you've referred the two\nkey functions:\n\n`(require '[datawalk.core :as dw :refer [look-at w]])`\n\nStart by calling\n\n`user\u003e (look-at my-data-structure)`\n\nThis initializes datawalk's internal state, so that it's always tracking a\npiece of current data. From there, call `(w *)`, where `*` represents any of\nthe commands listed in the usage summary. So for example:\n```\nuser\u003e (w 2) ; drill down to item 2 in the numbered sequence\nuser\u003e (w b) ; step backward in time\nuser\u003e (w s) ; save the current data to the map of saved values\nuser\u003e (w h) ; view the help (command summary)\n```\n\nAfter each step, the current state will be stored in datawalk atoms and can\nbe freely accessed:\n\n`datawalk.datawalk/data` stores the current data.\n\n`datawalk.datawalk/saved` holds the map of stored values.\n\nIn the same ns, you can also access `the-root`, `the-past`, and `the-future`.\n\nAdditionally, datawalk's built-in printing tools are available:\n\n```\nuser\u003e (w m) prints the map of saved values.\nuser\u003e (w p) prints the path from the root to the current data.\n```\n\n## Demo (semi-interactive mode):\n\n\n![Demo](resources/semi-interactive.gif?raw=true \"Demo\")\n\n\n## Why two versions?\n\nI developed the semi-interactive version so I could use datawalk in\nClojureScript. ClojureScript presents a challenge: while you can trivially get\nuser input in Clojure with `read-line`, ClojureScript has no consistent way to\ndo so (although some specific cljs repls like planck add that functionality\nthemselves). Because of this issue, it's very difficult to create a true\ninteractive program as described above.\n\nI've gotten some interesting tips about it, and @mfikes kindly pointed to\nhis abio library (https://github.com/abiocljs) which begins to address the\nproblem. I'm hoping that at some point I'll manage to put together a truly\nagnostic solution (PRs welcome!).\n\nBut now that I've created the semi-interactive version as an interim solution,\nit turns out it has some real advantages. Some people may actually prefer it,\nsince it keeps you in an ordinary REPL at all times. Its only downside is that\nyou have to type a few more characters.\n\n```\n         Fully-interactive:                |   Semi-interactive:\n                                           |\nPROS:  - fewer keystrokes                  | - Runs anywhere\n       - extremely fast                    | - Easy to do other stuff in the middle of a session\n                                           | - You're always in a normal REPL\n                                           |\nCONS:  - Clojure-only                      | - Requires more keystrokes\n       - Can't do other stuff in the       | - Not quite as fast to explore data\n         middle of a session               |\n```\n\n## Configuration\n\n`datawalk.print` contains a `config` atom holding several entries which you can\n  alter to change the behavior of datawalk. These configs all affect the\n  representation of the current data structure, not the underlying behavior.\n\n* `:max-items` the top level of your data structure may be a sequence with\n  hundreds or thousands of items; you probably don't want to print them all.\n  Defaults to 30.\n\n* `:max-line-length` on each printed line, datawalk attempts to represent the\n  entire contents of the data on that line. This can sometimes be enormously\n  long. You may wish to tune it to match the width of your repl environment.\n\n* `:max-key-length` when displaying maps, the keys may be so long that they take\n  up most of the space defined by \\*max-line-length\\*. This is most often true\n  with namespaced keys. \\*max-key-length\\* limits the space taken up by the\n  keys, in order to be sure to leave room for the values. When keys must be\n  chopped, they're chopped from the right to try to capture the name rather than\n  the namespace.\n\n* `:debug-mode` when this is truthy, datawalk will print the values of the current\n  path, saved values, the-past, and the-future at each step.\n\n`datawalk.core` also has a convenience function, `set-line-length`, which can be\n  passed a single number. `max-line-length` will be set to this value, and\n  `max-key-length` will be set to 20% of the value.\n\n## Note for Emacs users\n\nEmacs accepts user input (via `read-line`) in the minibuffer -- this can be\nconfusing when you first encounter it; the REPL buffer looks like it's hung.\nJust look down at the minibuffer and you'll see you can enter a command there.\nNote that it's important to q)uit the datawalk session, or your REPL buffer may\nbe left in a problematic state.\n\n## License\n\nCopyright © 2017 Jesse Davis (aka Egg Syntax)\n\nDistributed under the Eclipse Public License either version 1.0 or (at\nyour option) any later version.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feggsyntax%2Fdatawalk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feggsyntax%2Fdatawalk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feggsyntax%2Fdatawalk/lists"}