Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ahmed-shariff/org-roam-ql

Query language for org-roam
https://github.com/ahmed-shariff/org-roam-ql

emacs org-mode org-roam query-language

Last synced: 3 months ago
JSON representation

Query language for org-roam

Awesome Lists containing this project

README 

        
#+author: Shariff AM Faleel

#+language: en

#+HTML:    org-roam-ql           org-roam-ql-ql         



#+TITLE: org-roam-ql - query language for org-roam



This package provides an interface to easily query and display results from your ~org-roam~ database. 



* Contents

- [[#screen-shots][Screen-shots]]

- [[#installationSetup][Installation/Setup]]

- [[#usage][Usage]]

  - [[#installationsetup][Installation/Setup]]

  - [[#valid-values-for-source-or-query][Valid values for SOURCE-OR-QUERY]]

  - [[#adding-new-predicates][Adding new predicates]]

  - [[#org-dynamic-block][Org dynamic block]]

- [[#working-with-org-ql][Working with org-ql]]



* Screen-shots



You can query org-roam with ~org-roam-ql-search~. The results are displayed in an org-roam-like buffer. org-roam-ql also comes with a transient that can be used to modify the results viewed. The transient can be activated with ~v~. You can modify the title (~t~), query (~q~), sort (~s~) and specify if the query is a subquery (apply query on the results of the buffer) or query against the whole org-roam database (~i~). Refreshing the buffer (~r~) will display the updated results.



[[images/demo4.gif]]



The transient is available in the org-roam buffer as well, this allows you to start a query from the results in the org-roam buffer. You also can view the results in an agenda-like buffer (~S~). When entering a query in any interactive function of org-roam-ql, it also has completion-at-point. ~org-roam-ql~ buffer also supports bookmark, i.e., you can set a bookmark by calling ~bookmark-set~. For convenience, the transient also has a shortcut for that (~b~).



[[images/demo6.gif]]



* Installation/Setup

~org-roam-ql~ can be installed from [[https://melpa.org/#/org-roam-ql][MELPA]] or with other package management tools like [[https://framagit.org/steckerhalter/quelpa][quelpa]] and [[https://github.com/radian-software/straight.el][straight]]



Example configuration:

#+begin_src emacs-lisp

  (use-package org-roam-ql

    ;; If using straight

    :straight (org-roam-ql :type git :host github :repo "ahmed-shariff/org-roam-ql"

                           :files (:defaults (:exclude "org-roam-ql-ql.el")))

    ;; If using quelpa

    :quelpa (org-roam-ql :fetcher github :repo "ahmed-shariff/org-roam-ql"

                         :files (:defaults (:exclude "org-roam-ql-ql.el")))

    ;; Simple configuration

    :after (org-roam)

    :bind ((:map org-roam-mode-map

                 ;; Have org-roam-ql's transient available in org-roam-mode buffers

                 ("v" . org-roam-ql-buffer-dispatch)

                 :map minibuffer-mode-map

                 ;; Be able to add titles in queries while in minibuffer.

                 ;; This is similar to `org-roam-node-insert', but adds

                 ;; only title as a string.

                 ("C-c n i" . org-roam-ql-insert-node-title))))

#+end_src



* Usage

** Commands/functions

- =org-roam-ql-search (SOURCE-OR-QUERY &optional TITLE SORT-FN)= :: This is an *interactive* command that creates a ~org-roam-ql~ buffer with the nodes of the corresponding [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] with ~TITLE~. An ~org-roam-ql~ buffer is functionally similar to the ~org-roam-buffer~, but allows displaying any list of nodes ([[#screen-shots][see screen-shots above]]). When called interactively, it will prompt for the ~SOURCE-OR-QUERY~ and ~TITLE~. Note that when entering queries interactively either in ~org-roam-ql-search~ or in the transient, you can get completion-at-point with ~tab~. ~SORT-FN~ is used for sorting the results. It can be a string name of a registered sort function or a predicate function that can be used to sort the nodes (should take two nodes as input and return a non-nil value if the first node should be before the second). By default the following sort function are registered:  ~file-mtime~, ~file-atime~, ~deadline~, ~scheduled~, ~point~, ~level~, ~file-title~, ~file~ and ~title~. Each corresponds to the respective slot of an org-roam-node. It is possible to register new sort functions with ~org-roam-ql-register-sort-fn~. These registered functions will also appear as options for completion in the transient.

- =org-roam-ql-nodes (SOURCE-OR-QUERY)= :: Given a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] , return a list of nodes.

- ~org-roam-ql-agenda-block (QUERY)~ :: Meant to be used in ~org-agenda-custom-commands~ as a user-defined function. Insert items from processing ~QUERY~ (which is a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]]) into current buffer. QUERY is the `match' item in the custom command form. Currently this doesn't respect agenda restrict. Example:

  #+begin_src emacs-lisp

    (setq org-agenda-custom-commands

          ("cr" "Node a" org-roam-ql-agenda-block '(title "Node a")))

  #+end_src

- ~org-roam-ql-nodes-files (SOURCE-OR-QUERY)~ :: Given a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] , returns a list of files of the nodes. Can be used in ~org-agenda-custom-commands~. Example:

  #+begin_src emacs-lisp

    (setq org-agenda-custom-commands

          ("cr" "todo nodes" todo "TODO" ((org-agenda-files (org-roam-ql-nodes-files '(title "Node"))))))

  #+end_src

- ~org-roam-ql-add-saved-query (NAME DOCSTRING QUERY)~ :: Stores valid [[#valid-values-for-source-or-query][QUERY]] with ~NAME~ and ~DOCSTRING~. The ~NAME~ can be used as a query in place of any other ~SOURCE-OR-QUERY~. ~NAME~ can be a string or a symbol.

** Valid values for ~SOURCE-OR-QUERY~

- A list of ~org-roam-nodes~ :: This should self explanatory.

- A list of parameters that can be passed to ~org-roam-db-query~ :: It should be a list of the form ~(QUERY ARG1 ARG2...)~. The result of calling ~org-roam-db-query~ with these parameters should return a list of records where the first element is the ID of a corresponding node. For example:

  #+begin_src emacs-lisp

  (org-roam-ql-nodes '([:select [id] :from nodes :where (= todo \"TODO\")]))

  #+end_src

- Saved query name :: Name of a saved query (see ~org-roam-ql-add-saved-query~). This can be string or a symbol. Note that the name of the saved query is always stored as a symbol. If a string is passed, it gets interned into a symbol before looking up the query. The returned nodes will be a result of executing the query represented by the name.

- Bookmark name :: Name of a bookmark of a org-roam-ql-buffer. This matched against the name given when ~bookmark-set~ is called from a org-roam-ql-buffer.

- Buffer name :: A buffer or buffer-name of a ~org-roam~ buffer, a ~org-roam-ql~ buffer or an agenda-like buffer displaying a list of org-roam nodes.

- Function :: A function that returns a list of ~org-roam-nodes~

- A ~QUERY~ :: This is a predicate, similar to the predicates in [[https://github.com/alphapapa/org-ql][org-ql]]. Returns all nodes that pass for the given predicate. For example, consider the following call to ~org-roam-ql-nodes~:

    #+begin_src emacs-lisp

    (org-roam-ql-nodes '(and (todo "TODO") (tags "tag1" "tag2") "*org-roam*"))

    #+end_src

    In the above example, the result would contain any nodes whose todo state is =TODO=, have tags "tag1" and "tag2" and are in the org-roam buffer.

    The following are predicates available by default in org-roam-ql:

  - =or (SOURCE-OR-QUERY1 SOURCE-OR-QUERY2 ...)= :: Tests if a node matches/contained-in any of the ~SOURCE-OR-QUERY~'s. 

  - =and (SOURCE-OR-QUERY1 SOURCE-OR-QUERY2 ...)= :: Similar to ~or~, but should satisfy all predicates or contained in all the results of ~SOURCE-OR-QUERY~'s.

  - =not (SOURCE-OR-QUERY)= :: Tests if a node doesn't match the result or not contained in the result of ~SOURCE-OR-QUERY~.

  - =file (REGEXP &optional EXACT)= :: Test if nodes file name matches ~REGEXP~. If ~EXACT~ is non-nil, the file slot should be an exact match to ~REGEXP~. Note the slot ~file~ of an org-roam-node would contain the absolute path.

  - =file-title (REGEXP &optional EXACT)= :: Similar to ~file~, tests the ~file-title~ slot of a node.

  - =id (ID)= :: Tests if the ~ID~ of a node is a match to the value passed.

  - =level (LEVEL)= :: Tests if the ~level~ of a node is equal to ~LEVEL~.

  - =todo (REGEXP &optional EXACT)= :: Similar to ~file~, tests the todo state of a node.

  - =priority (REGEXP &optional EXACT)= :: Similar to ~file~, tests the priority of a node.

  - =scheduled (COMPARISON TIME-STRING)= :: Compares the ~scheduled~ of the node with ~TIME-STAMP~ based on ~COMPARISON~. ~TIME-STAMP~ is any valid value for [[https://orgmode.org/manual/The-date_002ftime-prompt.html][org date-time prompt]]. ~COMPARISON~ is either ~<~ or ~>~. Example: ~(scheduled > "-3w")~

  - =deadline= (COMPARISON TIME-STRING) :: Same as ~scheduled~, tests the ~deadline~ of a node.

  - =title (REGRXP &optional EXACT)= :: Similar to ~file~, tests the title of a node.

  - =properties (PROP PROP-VAL)= :: Tests if the value of the property of a node PROP is a match to PROP-VAL. PROP-VAL can be a regular expression.

  - =tags (TAG1 TAG2 ...)= :: Tests if the tags of a node have TAG1, TAG2, etc.

  - =refs (REGEXP &optional EXACT)= :: Similar to ~file~, tests the nodes refs slot.

  - =backlink-to (SOURCE-OR-QUERY)= :: Tests if the node has a backlink to any of the nodes from the results ~SOURCE-OR-QUERY~.

  - =backlink-from (SOURCE-OR-QUERY)= :: Similar to ~backlink-to~, tests if there are any backlinks from (aka forwardlinks) the resulting nodes from ~SOURCE-OR-QUERY~.

  - =in-buffer (BUFFER-NAME)= :: This is similar to passing a buffer-name as ~SOURCE-OR-QUERY~. Tests if a node is in the org-roam buffer named ~BUFFER-NAME~.

  - =nodes-list (NODES-LIST)= :: This is similar to passing a list of nodes as ~SOURCE-OR-QUERY~. Tests if a node is in the ~NODES-LIST~.

  - =function (FUNC)= :: This is similar to passing a function as ~SOURCE-OR-QUERY~. Tests if the node is in the result of executing the function ~FUNC~.

  - =funcall (FUNC)= :: Tests a node with the function ~FUNC~, which takes an org-roam node as parameter. Test passes if the function returns non-nil.

** Adding new predicates

There are two ways to add a new predicate to org-roam-ql: 

- =org-roam-ql-defpred (NAME DOCSTRING EXTRACTION-FUNCTION COMPARISON-FUNCTION)= :: Creates a predicate that can be used as ~SOURCE-OR-QUERY~. For example, for a predicate defined as follows:

  #+begin_src emacs-lisp

  (org-roam-ql-defpred sample "A sample predicate" extraction-function comparison-function)

  #+end_src



  When the following predicate is used as ~SOURCE-OR-QUERY~ :

  #+begin_src emacs-lisp

  (org-roam-ql-nodes '(sample arg1 arg2))

  #+end_src



  It tests each ~node~ in the whole org-roam database as follows:

  #+begin_src emacs-lisp

  (apply comparison-function (append (list (funcall extraction-function node)) arg1 arg2))

  #+end_src



  The ~EXTRACTION-FUNCTION~ takes an org-roam-node and returns a value that will be passed as the first parameter to ~COMPARISON-FUNCTION~. The remainder of the parameters when calling the predicate is passed as remaining parameters to ~COMPARISON-FUNCTION~. When the ~COMPARISON-FUNCTION~ returns a non-nil value, it will be included in the result.



- =org-roam-ql-defexpansion (NAME DOCSTRING EXPANSION-FUNCTION)= :: Adds an ~EXPANSION-FUNCTION~ which will be identified by ~NAME~ in a org-roam-ql query. The ~EXPANSION-FUNCTION~ should take the parameters passed in the query and return values that can be passed to ~org-roam-nodes~.



** Adding a sorting function

- =org-roam-ql-register-sort-fn (FUNCTION-NAME SORT-FUNCTION)= :: Registers a sort function which can be used with ~org-roam-ql-nodes~. ~FUNCTION-NAME~ is the string name used to refer to this function with. ~SORT-FUNCTION~ is a function that takes two org-roam-nodes and return a truth value, which is used to sort, i.e., if non-nil, the first node would be before the second node passed to the function.  Uses `seq-sort'.  If a sort-function with the given name already exists, it would be overwritten.

  The following example registers a sort function named "custom-prop" which sorts the values based on the "CUSTOM-PROP" property of a node.

  #+begin_src emacs-lisp

    (org-roam-ql-register-sort-fn "custom-prop"

                                  (lambda (el1 el2)

                                    (string< (cdr (assoc "CUSTOM-PROP" (org-roam-node-properties el1)))

                                             (cdr (assoc "CUSTOM-PROP" (org-roam-node-properties el2))))))

  #+end_src



** Org dynamic block

Similar to ~org-ql~, ~org-roam-ql~ also provides a dynamic block. The header parameters are as follows:

- ~:query~ - A valid [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]]

- ~:columns~ - A list of columns to display. Each column name is a slot name of ~org-roam-nodes~. For any function/accessor with a name of the form ~org-roam-node-~, which takes an org-roam-node as a parameter, ~~ can also be used column name. For example, if there is a function named ~org-roam-node-short-title~, ~short-title~ can be used as a column name, this will result in a column with the title ~short-title~ where the content of each row is the result of calling the respective function.

- ~:sort~ - Name of a registered sort functions. See [[#commandsfunctions][ ~org-roam-ql-search~ ]] for more info on the values for sort functions.

- ~:take~ (optional) - If a positive integer N, take the first N elements, if a negative -N, take the last N nodes.

- ~:no-link~ (optional) - If a non-nil value is set, the first column containing the links will be dropped.



If no-link is not provided as a parameter, the first column is a link to the node. Since it is an id link, it will be a backlink to the node.



Following is an example of a dynamic block and its result.



[[file:images/dynamic-block.jpg]]

* Working with org-ql

Optionally, ~org-roam-ql~ results can be visualized with [[https://github.com/alphapapa/org-ql][org-ql]], available through the extension ~org-roam-ql-ql~ (naming things is hard!!). This also can be installed from [[https://melpa.org/#/org-roam-ql-ql][MELPA]] or with other package management tools like [[https://framagit.org/steckerhalter/quelpa][quelpa]] and [[https://github.com/radian-software/straight.el][straight]].



#+begin_src emacs-lisp

  (use-package org-roam-ql-ql

    ;; If using straight

    :straight (org-roam-ql-ql :type git :host github :repo "ahmed-shariff/org-roam-ql"

                              :files (:defaults (:exclude "org-roam-ql.el")))

    ;; If using quelpa

    :quelpa (org-roam-ql-ql :fetcher github :repo "ahmed-shariff/org-roam-ql"

                            :files (:defaults (:exclude "org-roam-ql.el")))

    ;; Simple config

    :after (org-ql org-roam-ql)

    :config

    (org-roam-ql-ql-init))

#+end_src



Note that org-ql works only with org entries, i.e., `heading nodes`. Hence, if there are any file nodes in the result, they will not be displayed. To be clear about that, when org-roam-ql results are displayed in an org-ql-view buffer, a warning is added to the end mentioning how many file nodes were there in the result. If the extension is loaded, you may view the org-roam-ql results with ~Q~ from the org-roam-ql transient. An org-ql-view can be viewed in an org-roam-like buffer with ~R~ from the org-ql-view transient.



[[images/demo5.gif]]