Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/ahmed-shariff/org-roam-ql

Query language for org-roam
https://github.com/ahmed-shariff/org-roam-ql

emacs org-mode org-roam query-language

Last synced: about 2 months ago
JSON representation

Query language for org-roam

Awesome Lists containing this project

README 

        
#+author: Shariff AM Faleel

#+language: en

#+HTML:    org-roam-ql           org-roam-ql-ql         



#+TITLE: org-roam-ql - query language for org-roam



This package provides an interface to easily query and display results from your ~org-roam~ database. 



* Contents

- [[#screen-shots][Screen-shots]]

- [[#installationSetup][Installation/Setup]]

- [[#usage][Usage]]

  - [[#installationsetup][Installation/Setup]]

  - [[#valid-values-for-source-or-query][Valid values for SOURCE-OR-QUERY]]

  - [[#adding-new-predicates][Adding new predicates]]

  - [[#org-dynamic-block][Org dynamic block]]

  - [[#custom-preview-function-for-org-roam-ql-buffer][Custom preview function for org-roam-ql buffer]]

- [[#working-with-org-ql][Working with org-ql]]

- [[#tips-and-tricks][Tips and tricks]]



* Screen-shots



You can query org-roam with ~org-roam-ql-search~. The results are displayed in an org-roam-like buffer. org-roam-ql also comes with a transient that can be used to modify the results viewed. The transient can be activated with ~v~. You can modify the title (~t~), query (~q~), sort (~s~) and specify if the query is a subquery (apply query on the results of the buffer) or query against the whole org-roam database (~i~). Refreshing the buffer (~r~) will display the updated results.



[[images/demo4.gif]]



The transient is available in the org-roam buffer as well, this allows you to start a query from the results in the org-roam buffer. You also can view the results in an agenda-like buffer (~S~). When entering a query in any interactive function of org-roam-ql, it also has completion-at-point. ~org-roam-ql~ buffer also supports bookmark, i.e., you can set a bookmark by calling ~bookmark-set~. For convenience, the transient also has a shortcut for that (~b~).



[[images/demo6.gif]]



* Installation/Setup

~org-roam-ql~ can be installed from [[https://melpa.org/#/org-roam-ql][MELPA]] or with other package management tools like [[https://framagit.org/steckerhalter/quelpa][quelpa]] and [[https://github.com/radian-software/straight.el][straight]]



Example configuration:

#+begin_src emacs-lisp

  (use-package org-roam-ql

    ;; If using straight

    :straight (org-roam-ql :type git :host github :repo "ahmed-shariff/org-roam-ql"

                           :files (:defaults (:exclude "org-roam-ql-ql.el")))

    ;; If using quelpa

    :quelpa (org-roam-ql :fetcher github :repo "ahmed-shariff/org-roam-ql"

                         :files (:defaults (:exclude "org-roam-ql-ql.el")))

    ;; Simple configuration

    :after (org-roam)

    :bind ((:map org-roam-mode-map

                 ;; Have org-roam-ql's transient available in org-roam-mode buffers

                 ("v" . org-roam-ql-buffer-dispatch)

                 :map minibuffer-mode-map

                 ;; Be able to add titles in queries while in minibuffer.

                 ;; This is similar to `org-roam-node-insert', but adds

                 ;; only title as a string.

                 ("C-c n i" . org-roam-ql-insert-node-title))))

#+end_src



* Usage

** Commands/functions

- =org-roam-ql-search (SOURCE-OR-QUERY &optional TITLE SORT-FN PREVIEW-FN)= :: This is an *interactive* command that creates a ~org-roam-ql~ buffer with the nodes of the corresponding [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] with ~TITLE~. An ~org-roam-ql~ buffer is functionally similar to the ~org-roam-buffer~, but allows displaying any list of nodes ([[#screen-shots][see screen-shots above]]). When called interactively, it will prompt for the ~SOURCE-OR-QUERY~ and ~TITLE~. Note that when entering queries interactively either in ~org-roam-ql-search~ or in the transient, you can get completion-at-point with ~tab~.



   ~SORT-FN~ is used for sorting the results. It can be a string name of a registered sort function or a predicate function that can be used to sort the nodes (should take two nodes as input and return a non-nil value if the first node should be before the second). By default the following sort function are registered:  ~file-mtime~, ~file-atime~, ~deadline~, ~scheduled~, ~point~, ~level~, ~file-title~, ~file~ and ~title~. Each corresponds to the respective slot of an org-roam-node. It is possible to register new sort functions with ~org-roam-ql-register-sort-fn~. These registered functions will also appear as options for completion in the transient.



  ~PREVIEW-FN~ is a function used to generate the preview content on the ~org-roam-ql-mode~ buffer. See [[#custom-preview-function-for-org-roam-ql-buffer][Custom preview function for org-roam-ql buffer]] for more information on what this value can be.  If ~PREVIEW-FN~ is nil, the value of ~org-roam-ql-preview-function~ will be used. If it is non-nil, this argument will be set as the buffer local value for  ~org-roam-ql-preview-function~ in the corresponding ~org-roam-ql-mode~ buffer.

- =org-roam-ql-nodes (SOURCE-OR-QUERY)= :: Given a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] , return a list of nodes.

- ~org-roam-ql-agenda-block (QUERY)~ :: Meant to be used in ~org-agenda-custom-commands~ as a user-defined function. Insert items from processing ~QUERY~ (which is a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]]) into current buffer. QUERY is the `match' item in the custom command form. Currently this doesn't respect agenda restrict. Example:

  #+begin_src emacs-lisp

    (setq org-agenda-custom-commands

          ("cr" "Node a" org-roam-ql-agenda-block '(title "Node a")))

  #+end_src

- ~org-roam-ql-nodes-files (SOURCE-OR-QUERY)~ :: Given a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] , returns a list of files of the nodes. Can be used in ~org-agenda-custom-commands~. Example:

  #+begin_src emacs-lisp

    (setq org-agenda-custom-commands

          ("cr" "todo nodes" todo "TODO" ((org-agenda-files (org-roam-ql-nodes-files '(title "Node"))))))

  #+end_src

- ~org-roam-ql-add-saved-query (NAME DOCSTRING QUERY)~ :: Stores valid [[#valid-values-for-source-or-query][QUERY]] with ~NAME~ and ~DOCSTRING~. The ~NAME~ can be used as a query in place of any other ~SOURCE-OR-QUERY~. ~NAME~ can be a string or a symbol.

** Valid values for ~SOURCE-OR-QUERY~

- A list of ~org-roam-nodes~ :: This should self explanatory.

- A list of parameters that can be passed to ~org-roam-db-query~ :: It should be a list of the form ~(QUERY ARG1 ARG2...)~. The result of calling ~org-roam-db-query~ with these parameters should return a list of records where the first element is the ID of a corresponding node. For example:

  #+begin_src emacs-lisp

  (org-roam-ql-nodes '([:select [id] :from nodes :where (= todo \"TODO\")]))

  #+end_src

- Saved query name :: Name of a saved query (see ~org-roam-ql-add-saved-query~). This can be string or a symbol. Note that the name of the saved query is always stored as a symbol. If a string is passed, it gets interned into a symbol before looking up the query. The returned nodes will be a result of executing the query represented by the name.

- Bookmark name :: Name of a bookmark of a org-roam-ql-buffer. This matched against the name given when ~bookmark-set~ is called from a org-roam-ql-buffer.

- Buffer name :: A buffer or buffer-name of a ~org-roam~ buffer, a ~org-roam-ql~ buffer or an agenda-like buffer displaying a list of org-roam nodes.

- Function :: A function that returns a list of ~org-roam-nodes~

- A ~QUERY~ :: This is a predicate, similar to the predicates in [[https://github.com/alphapapa/org-ql][org-ql]]. Returns all nodes that pass for the given predicate. For example, consider the following call to ~org-roam-ql-nodes~:

    #+begin_src emacs-lisp

    (org-roam-ql-nodes '(and (todo "TODO") (tags "tag1" "tag2") "*org-roam*"))

    #+end_src

    In the above example, the result would contain any nodes whose todo state is =TODO=, have tags "tag1" and "tag2" and are in the org-roam buffer.

    The following are predicates available by default in org-roam-ql:

  - ~or (SOURCE-OR-QUERY1 SOURCE-OR-QUERY2 ...)~ :: Tests if a node matches/contained-in any of the ~SOURCE-OR-QUERY~'s. 

  - ~and (SOURCE-OR-QUERY1 SOURCE-OR-QUERY2 ...)~ :: Similar to ~or~, but should satisfy all predicates or contained in all the results of ~SOURCE-OR-QUERY~'s.

  - ~not (SOURCE-OR-QUERY)~ :: Tests if a node doesn't match the result or not contained in the result of ~SOURCE-OR-QUERY~.

  - ~file (MATCH &optional EXACT USE-REGEXP)~ :: Test if nodes file name matches ~MATCH~. If ~EXACT~ is non-nil, the file slot should be an exact match to ~MATCH~. By default, this uses sql LIKE, but if ~USE-REGEXP~ is non-nil, then ~MATCH~ will be treated as a regex and regex matching will be used. Note the slot ~file~ of an org-roam-node would contain the absolute path.

  - ~file-title (MATCH &optional EXACT USE-REGEXP)~ :: Similar to ~file~, tests the ~file-title~ slot of a node.

  - ~id (ID)~ :: Tests if the ~ID~ of a node is a match to the value passed.

  - ~level= (LEVEL)~ :: Tests if the ~level~ of a node is equal to ~LEVEL~.

  - ~level< (LEVEL)~ :: Tests if the ~level~ of a node is less than ~LEVEL~.

  - ~level> (LEVEL)~ :: Tests if the ~level~ of a node is greater than ~LEVEL~.

  - ~level<> (LEVEL)~ :: Tests if the ~level~ of a node is not equal to ~LEVEL~.

  - ~point= (POINT)~ :: Tests if the ~point~ of a node is equal to ~POINT~.

  - ~point< (POINT)~ :: Tests if the ~point~ of a node is less than ~POINT~.

  - ~point> (POINT)~ :: Tests if the ~point~ of a node is greater than ~POINT~.

  - ~point<> (POINT)~ :: Tests if the ~point~ of a node is not equal to ~POINT~.

  - ~todo (MATCH &optional EXACT USE-REGEXP)~ :: Similar to ~file~, tests the todo state of a node.

  - ~priority (MATCH &optional EXACT USE-REGEXP)~ :: Similar to ~file~, tests the priority of a node.

  - ~scheduled (COMPARISON TIME-STRING)~ :: Compares the ~scheduled~ of the node with ~TIME-STAMP~ based on ~COMPARISON~. ~TIME-STAMP~ is any valid value for [[https://orgmode.org/manual/The-date_002ftime-prompt.html][org date-time prompt]]. ~COMPARISON~ is either ~<~ or ~>~. Example: ~(scheduled > "-3w")~

  - ~scheduled-is-after (TIME-STRING)~ :: ~TIME-STRING~ is the same as ~scheduled~. This filters for nodes with ~schedule~ after ~TIME-STRING~.

  - ~scheduled-is-before (TIME-STRING)~ :: ~TIME-STRING~ is the same as ~scheduled~. This filters for nodes with ~schedule~ before ~TIME-STRING~.

  - ~deadline (COMPARISON TIME-STRING)~ :: Same as ~scheduled~, tests the ~deadline~ of a node.

  - ~deadline-is-after (TIME-STRING)~ :: ~TIME-STRING~ is the same as ~scheduled~. This filters for nodes with ~deadline~ after ~TIME-STRING~.

  - ~deadline-is-before (TIME-STRING)~ :: ~TIME-STRING~ is the same as ~scheduled~. This filters for nodes with ~deadline~ before ~TIME-STRING~.

  - ~title (MATCH &optional EXACT USE-REGEXP)~ :: Similar to ~file~, tests the title of a node. Note this, this only considers the ~title~, and not any of the ~aliases~.

  - ~aliases (MATCH &optional EXACT USE-REGEXP)~ :: Similar to ~file~, tests the aliases of a node.

  - ~title-or-alias (MATCH &optional EXACT)~ :: Similar to ~file~, tests both ~title~ and ~aliases~ of a node. If not ~EXACT~ will always use regex matching.

  - ~properties (PROP PROP-VAL)~ :: Tests if the value of the property of a node PROP is a match to PROP-VAL. PROP-VAL can be a regular expression.

  - ~tags (TAG1 TAG2 ...)~ :: Tests if the tags of a node have TAG1, TAG2, etc.

  - ~refs (MATCH &optional EXACT USE-REGEXP)~ :: Similar to ~file~, tests the nodes refs slot.

  - ~backlink-to (SOURCE-OR-QUERY)~ :: Tests if the node has a backlink to any of the nodes from the results ~SOURCE-OR-QUERY~.

  - ~backlink-from (SOURCE-OR-QUERY)~ :: Similar to ~backlink-to~, tests if there are any backlinks from (aka forwardlinks) the resulting nodes from ~SOURCE-OR-QUERY~.

  - ~in-buffer (BUFFER-NAME)~ :: This is similar to passing a buffer-name as ~SOURCE-OR-QUERY~. Tests if a node is in the org-roam buffer named ~BUFFER-NAME~.

  - ~nodes-list (NODES-LIST)~ :: This is similar to passing a list of nodes as ~SOURCE-OR-QUERY~. Tests if a node is in the ~NODES-LIST~.

  - ~function (FUNC)~ :: This is similar to passing a function as ~SOURCE-OR-QUERY~. Tests if the node is in the result of executing the function ~FUNC~.

  - ~funcall (FUNC)~ :: Tests a node with the function ~FUNC~, which takes an org-roam node as parameter. Test passes if the function returns non-nil.

  - ~file-atime (COMPARISON TIME-STRING)~ :: Same as ~scheduled~, tests the ~deadline~ of a node.

  - ~file-atime-is-after (TIME-STRING)~ :: ~TIME-STRING~ is the same as ~scheduled~. This filters for nodes ~file-atime~ is after ~TIME-STRING~.

  - ~file-atime-is-before (TIME-STRING)~ :: ~TIME-STRING~ is the same as ~scheduled~. This filters for nodes ~file-atime~ is before ~TIME-STRING~.

  - ~file-mtime (COMPARISON TIME-STRING)~ :: Same as ~scheduled~, tests the ~deadline~ of a node.

  - ~file-mtime-is-after (TIME-STRING)~ :: ~TIME-STRING~ is the same as ~scheduled~. This filters for nodes ~file-mtime~ is after ~TIME-STRING~.

  - ~file-mtime-is-before (TIME-STRING)~ :: ~TIME-STRING~ is the same as ~scheduled~. This filters for nodes ~file-mtime~ is before ~TIME-STRING~.

** Adding new predicates

There are two ways to add a new predicate to org-roam-ql: 

- ~org-roam-ql-defpred (NAME DOCSTRING EXTRACTION-FUNCTION COMPARISON-FUNCTION)~ :: Creates a predicate that can be used as ~SOURCE-OR-QUERY~. For example, for a predicate defined as follows:

  #+begin_src emacs-lisp

  (org-roam-ql-defpred sample "A sample predicate" extraction-function comparison-function)

  #+end_src



  When the following predicate is used as ~SOURCE-OR-QUERY~ :

  #+begin_src emacs-lisp

  (org-roam-ql-nodes '(sample arg1 arg2))

  #+end_src



  It tests each ~node~ in the whole org-roam database as follows:

  #+begin_src emacs-lisp

  (apply comparison-function (append (list (funcall extraction-function node)) arg1 arg2))

  #+end_src



  The ~EXTRACTION-FUNCTION~ takes an org-roam-node and returns a value that will be passed as the first parameter to ~COMPARISON-FUNCTION~. The remainder of the parameters when calling the predicate is passed as remaining parameters to ~COMPARISON-FUNCTION~. When the ~COMPARISON-FUNCTION~ returns a non-nil value, it will be included in the result.



- ~org-roam-ql-defexpansion (NAME DOCSTRING EXPANSION-FUNCTION)~ :: Adds an ~EXPANSION-FUNCTION~ which will be identified by ~NAME~ in a org-roam-ql query. The ~EXPANSION-FUNCTION~ should take the parameters passed in the query and return values that can be passed to ~org-roam-nodes~.



** Adding a sorting function

- ~org-roam-ql-register-sort-fn (FUNCTION-NAME SORT-FUNCTION)~ :: Registers a sort function which can be used with ~org-roam-ql-nodes~. ~FUNCTION-NAME~ is the string name used to refer to this function with. ~SORT-FUNCTION~ is a function that takes two org-roam-nodes and return a truth value, which is used to sort, i.e., if non-nil, the first node would be before the second node passed to the function.  Uses `seq-sort'.  If a sort-function with the given name already exists, it would be overwritten.

  The following example registers a sort function named "custom-prop" which sorts the values based on the "CUSTOM-PROP" property of a node.

  #+begin_src emacs-lisp

    (org-roam-ql-register-sort-fn "custom-prop"

                                  (lambda (el1 el2)

                                    (string< (cdr (assoc "CUSTOM-PROP" (org-roam-node-properties el1)))

                                             (cdr (assoc "CUSTOM-PROP" (org-roam-node-properties el2))))))

  #+end_src



** Org dynamic block

Similar to ~org-ql~, ~org-roam-ql~ also provides a dynamic block. The header parameters are as follows:

- ~:query~ - A valid [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]]

- ~:columns~ - A list of columns to display. Each column name is a slot name of ~org-roam-nodes~. For any function/accessor with a name of the form ~org-roam-node-~, which takes an org-roam-node as a parameter, ~~ can also be used column name. For example, if there is a function named ~org-roam-node-short-title~, ~short-title~ can be used as a column name, this will result in a column with the title ~short-title~ where the content of each row is the result of calling the respective function.

- ~:sort~ - Name of a registered sort functions. See [[#commandsfunctions][ ~org-roam-ql-search~ ]] for more info on the values for sort functions.

- ~:take~ (optional) - If a positive integer N, take the first N elements, if a negative -N, take the last N nodes.

- ~:no-link~ (optional) - If a non-nil value is set, the first column containing the links will be dropped.



If no-link is not provided as a parameter, the first column is a link to the node. Since it is an id link, it will be a backlink to the node.



Following is an example of a dynamic block and its result.



[[file:images/dynamic-block.jpg]]



** Custom preview function for org-roam-ql buffer



~org-roam-ql~ buffers visualizes the node itself. Which is unlike ~org-roam~ buffers, which visualizes relationships (like ~backlinks~ or ~refs~). To allow better customization, ~org-roam-ql~ has the following customizable variables. These are mostly similar to the ~org-roam~ counterparts:

- ~org-roam-ql-preview-function~ :: This differs from the ~org-roam~ counterpart. The preview function used to generate the content of a nodes preview in ~org-roam-ql-mode~ buffer. Before calling this, the point will be set to ~org-roam-node-point~. Will be called with two arguments: (1) The corresponding node, and (2) the source-or-query of the corresponding buffer. See [[#valid-values-for-source-or-query][Valid values for SOURCE-OR-QUERY]] for what the value of source-or-query can be. By default, it calls the ~org-roam-preview-function~. Note that this can be set local to the buffer and will be used to generate the preview when the buffer is refreshed. The value set for this variable also can be overridden by passing it as a parameter to ~org-roam-ql-search~ when called programatically - in this case, the preview function passed as an argument is set as the buffer local value for this variable.

- ~org-roam-ql-buffer-postrender-functions~ :: Functions to run after a section of a ~org-roam-ql-mode~ buffer is rendered. Each function accepts no arguments, and is run with the ~org-roam-ql-mode~ buffer as the current buffer. By default, it is set to the value of ~org-roam-buffer-postrender-functions~.

- ~org-roam-ql-preview-postprocess-functions~ :: A list of functions to postprocess the preview content. This is the same as ~org-roam-preview-postprocess-functions~, and the default value is set to this. Each function takes a single argument, the string for the preview content, and returns the post-processed string. The functions are applied in order of appearance in the list.



See ~M-x customize-group RET org-roam-ql~ for more customization options.



* Working with org-ql

Optionally, ~org-roam-ql~ results can be visualized with [[https://github.com/alphapapa/org-ql][org-ql]], available through the extension ~org-roam-ql-ql~ (naming things is hard!!). This also can be installed from [[https://melpa.org/#/org-roam-ql-ql][MELPA]] or with other package management tools like [[https://framagit.org/steckerhalter/quelpa][quelpa]] and [[https://github.com/radian-software/straight.el][straight]].



#+begin_src emacs-lisp

  (use-package org-roam-ql-ql

    ;; If using straight

    :straight (org-roam-ql-ql :type git :host github :repo "ahmed-shariff/org-roam-ql"

                              :files (:defaults (:exclude "org-roam-ql.el")))

    ;; If using quelpa

    :quelpa (org-roam-ql-ql :fetcher github :repo "ahmed-shariff/org-roam-ql"

                            :files (:defaults (:exclude "org-roam-ql.el")))

    ;; Simple config

    :after (org-ql org-roam-ql)

    :config

    (org-roam-ql-ql-init))

#+end_src



Note that org-ql works only with org entries, i.e., `heading nodes`. Hence, if there are any file nodes in the result, they will not be displayed. To be clear about that, when org-roam-ql results are displayed in an org-ql-view buffer, a warning is added to the end mentioning how many file nodes were there in the result. If the extension is loaded, you may view the org-roam-ql results with ~Q~ from the org-roam-ql transient. An org-ql-view can be viewed in an org-roam-like buffer with ~R~ from the org-ql-view transient.



[[images/demo5.gif]]



* Tips and tricks

The [[https://github.com/ahmed-shariff/org-roam-ql/wiki][wiki]] has additional tips and tricks, like using org-roam-ql to filter completion candidates in minibuffer.