Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/pprevos/denote-explore

Helper functions to analyse and visualise a collection of notes created with the Denote package.
https://github.com/pprevos/denote-explore

emacs

Last synced: 2 months ago
JSON representation

Helper functions to analyse and visualise a collection of notes created with the Denote package.

Awesome Lists containing this project

README 

        
#+title:  Denote Explore: Explore your Denote digital garden

#+author: Peter Prevos



The Denote-Explore package provides auxiliary functions to maintain and explore your collection of Denote files. Denote-Explore provides three groups of Emacs commands:



1. /Summary statistics/: Count and visualise notes, attachments and keywords.

2. /Random walks/: Generate new ideas using serendipity.

3. /Janitor/: Manage your denote collection.

4. /Visualisations/: Visualise your Denote network  as a network graph.



This package exists because I started to move many files on my computer into Denote silos, leveraging the ingenious file naming convention and associated functionality. These functions helped me to add some structure to the organic chaos of my digital garden.



Read the [[https://lucidmanager.org/productivity/denote-explore][denote-explore manual]] for a comprehensive explanation of all functionality (=C-h R denote-explore=).



This package is available on MELPA and is easily installed with the ~use-package~ macro.



The configuration below sets all available variables. The first two variables define where graph files are stored, sans extension. If the directory does not yet exist, then the package creates it when required. The third variable defines the output format and as such the file extensions. Possible options are [[https://graphviz.org/][graphviz]], [[https://d3js.org/][D3.js]] (JSON) and [[https://gexf.net/][GEXF]]. You will need to install GraphViz and/or the R language to visualise Denote graphs.



#+begin_src elisp :results none

  (use-package denote-explore

    :custom

    ;; Location of graph files

    (denote-explore-network-directory "~/documents/notes/graphs/")

    (denote-explore-network-filename "denote-network")

    ;; Output format

    (denote-explore-network-format 'graphviz)

    (denote-explore-network-graphviz-filetype "svg")

    ;; Exlude keywords or regex

    (denote-explore-network-keywords-ignore '("bib")))

#+end_src



Alternatively, you can install it directly from GitHub:



#+begin_src elisp :eval no

  (unless (package-installed-p 'denote-explore)

    (package-vc-install

     '(denote-explore

       :url "https://github.com/pprevos/denote-explore/")))

#+end_src



* Change Log

** Version 3.0

- Compatability with Denote 3.0 

- Renamed barchart functions to all start with ~denote-explore-barchart-~.

- Using universal argument with ~denote-explore-barchart-extensions~ visualises only attachment file types.

- All functions that rename files updated for compatability with Denote 3.0

- Enhanced info manual

  

** Version 1.6

- New functions:

  - ~denote-explore-backlinks-barchart~ to view top-n notes with backlinks

  - ~denote-explore-random-regex~ to jump to random note matching a regular expression.

  - ~denote-explore-identify-duplicate-notes-dired~: View notes with duplicates ID in Dired.

  

** Version 1.5.1

- Fixed various compilation errors.

- Enhanced output for ~denote-explore-identify-duplicate-notes~

- Update customization types for note ignore variables

- Updated ~defgroup~ definition and ~defcustom~ types



** Version 1.5

- Published Info mode manual

- Clarified obsolete variables and functions

- ~denote-explore-network-keywords~ no longer interactive

- Updates to Denote functions



** Version 1.4.2

- Geneal code refactoring with assistance from Prot

- Added ability to filter keywords network by minimum weight of edges

- Added ability for free text in ~denote-explore-network-graphviz-filetype~

- GEXF format now includes the total degree for each node

- Enforced consistency with use of universal argument (its use will include attachments)

- ~denote-explore--retrieve-title~ deprecated. Now using Denote function.

- Updated manual with clarifications



** Version 1.4.1

- Fixed bugs with random links and keywords from nil file-name

- New variable ~denote-explore-network-regex-ignore~ to ignore files matching a regular expression when creating neighbourhood or community graphs.



** Version 1.4

+ Network visualisation

  - SVG output includes links to source file.

  - Configure [[https://graphviz.org/docs/outputs/][GraphViz output format]] with ~denote-explore-network-graphviz-filetype~ (SVG (default) or PDF advised). 

  - Improved scaling of GraphViz graphics.

  - ~denote-explore-network-regenerate~ recreates the previous graph

+ New functions:

  - ~denote-explore-isolated-notes~ to select Denote files without any links or backlinks.

  - ~denote-explore-degree-barchart~ draws the distribution of total number of (back)links per file ([[https://en.wikipedia.org/wiki/Degree_distribution][Degree distribution - Wikipedia]]).

+ Updated functions:

  - ~denote-explore-rename-keyword~ allows renaming more than one keyword to a new version.

  - ~denote-explore-sync-metadata~ enforces renaming confirmation.

+ Minor bug fixes.



** Version 1.3

- Complete rewrite of the graph visualisation functionality. Now includes three formats (JavaScript, GraphViz and GEXF) and three graph types (notes that match a regular expression, note neighbourhood and keyword graph).

- Fixed bugs in =denote-explore--retrieve-title= and =denote-explore--retrieve-keywords=.

- Enhanced function and variable documentation.



** Version 1.2

- Removed =pandoc= dependency in network generation.

-  ~denote-explore-identify-duplicate-notes~ replaces ~denote-explore-identify-duplicate-identifiers~. This new version either detects duplicate identifiers, or duplicate file names (using the universal argument). Comparing duplicate filenames ignores any duplicate identifiers caused by exporting Org mode files.



** Version 1.1

- Added helper function to call R script for network visualisation.

- =denote-explore-dashboard.el= deprecated.

- Added to MELPA.



** Version 1.0

This version is a complete rewrite of the code to improve coding quality and compatibility with internal changes since Denote version 2.2.



Also major changes to the network visualisation, which can now take a regular expression for partial networks. Most of the code is now undertaken in Emacs Lisp, which saves that Denote notes and attachment structures in JSON format.



The functionality for a Dashboard widget has been moved to a separate file to not enforce the requirements for those users not seeking to implement the widget.



New functionality since the previous version:

- ~denote-explore-identify-duplicate-identifiers~: Provide a list of duplicate identifiers.

- ~denote-explore-single-keywords~: Select a note or attachment with a keyword that is only used once.

- ~denote-explore-zero-keywords~: Select a note or attachment without any keywords.

- ~denote-explore-sort-keywords~: Order the keywords of all Denote notes and attachments alphabetically.

- ~denote-explore-rename-keyword~: Rename or remove a keyword across the whole Denote collection.

- ~denote-explore-sync-metadata~: Synchronise the filenames with the metadata for all Denote files.



* Ideas for future development

- [-] Create a Denote widget for the Emacs Dashboard with links to statistics.

- [-] Functions to rename Denote attachments using EXIF metadata.

- [-] Jump to random note within a time frame



** Visualising Denote

- [-] Ring of previously-generated networks instead of overwriting ~denote-explore-network-previous~

- [ ] [[https://orgmode.org/manual/Dynamic-Blocks.html][Dynamic org mode blocks]] to visualise the neighbourhood of a node

- [ ] Use [[https://github.com/alphapapa/org-graph-view/][org-graph-view]] for live neighbourhood view of the current buffer

- [ ] [[https://graphviz.org/pdf/cluster.1.pdf][Community detection]] in GraphViz versions?

- [ ] Visualise signatures ([[https://zettelkasten.de/introduction/2020-08-13_folgezettel-sequence.png][example]])

- [ ] Store the Denote metadata in a [[https://www.gnu.org/software/emacs/manual/html_node/elisp/Hash-Tables.html][hash table]] that is updated after every modification to a Denote file for more efficient visualisation.

- [ ] The current network visualisation uses the [[https://christophergandrud.github.io/networkD3/][networkD3 package]] of the R language as a detour to generate a HTML file with associated JavaScript code. I am slowly working my way to not have to use R, but I first need to learn more JavaScript and the [[https://d3js.org/][D3 visualisation language]].

- [ ] Scaling GraphViz text size based on number of nodes

- [ ] Use a plain text method to perhaps visualise the network neighbourhood. The [[https://github.com/DamienCassou/hierarchy][hierarchy package]] can display information in a tree format, which might be helpful to visualise the local community of a Denote buffer, e.g. two links backwards and forwards.

- [ ] Plain text time series graph of the creation of new notes.