Skip to content
/ om-dash Public

Building blocks for org-based dashboards.

License

Notifications You must be signed in to change notification settings

gavv/om-dash

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

om-dash

What is this?

om-dash implements dynamic blocks for org-mode that you can use to compose a custom project dashboard.

It was always a struggle to me to keep track of the “big picture” when I’m hopping between projects.

I wanted a tool that can give me a brief summary of all ongoing projects: what’s done, what’s next, and what requires attention. And then I realized that it can be easily implemented using org-mode, so here we go.

om-dash implementats a few configurable dynamic blocks:

  • om-dash-github-topics - generates a table with issues or pull requests from github repository
  • om-dash-github-project-cards - generates a table with cards from github classic project
  • om-dash-orgfile - generates tables with top-level entries from an org file
  • om-dash-imap - generates table with unread email counters for IMAP folder
  • om-dash-command - generates a table from JSON or CSV output of a shell command
  • om-dash-function - generates a table from output of a Elisp function

The package also provides a minor mode (om-dash-mode) to apply color highlighting to the generated tables.

In addition, there is support for templates, which allows to create reusable parameterized configurations of the above blocks (e.g. for specific github query or shell command).

Example workflow

Here I describe my own workflow. Yours can be different of course, but I think this should give the basic idea about this package.

For every project, I have three main sources of “things” to keep track of:

  • github repository with issues and pull requests
  • personal org file with tasks grouped into some kind of milestones (usually releases)
  • a few IMAP directories with email related to this project (mailing lists, notifications, discussions)

On top of that, I have a file called “dashboard.org” with a top-level entry for every project, and a few second-level entries with om-dash dynamic blocks:

  • a block with all open or recently merged pull requests from github
  • another block with open github issues (for big projects, I display only issues from specific column of github kanban board, or from specific milestone)
  • one block for every ongoing or upcoming milestone from my personal org file for this project, showing top level tasks from each milestone
  • block with project’s IMAP directories and unread email counters

Screenshot of a project from “dashboard.org” described above:

Example blocks

Github pull requests

Display all open pull requests and pull requests closed last month.

#+BEGIN: om-dash-github-topics :repo "roc-streaming/roc-toolkit" :type pr :open "*" :closed "-1mo"
...
#+END:

./screenshot/github_pull_requests.png

Github issues

Display all open issues except those which have “help wanted” label.

#+BEGIN: om-dash-github-topics :repo "gavv/signal-estimator" :type issue :open "-label:\"help wanted\""
...
#+END:

./screenshot/github_issues.png

Github project items

Display all items from github project v2 (non-clasic, a.k.a. beta) with project id “5”, item type “Issue”, and item status “In work”.

#+BEGIN: om-dash-github-project-items :owner "roc-streaming" :project 5 :type issue :status "In work"
...
#+END:

./screenshot/github_project_items.png

Github project cards (classic)

Display all cards from github classic project (now being deprecated by github) with project id “2”, card type “issue”, and column name “In work”.

#+BEGIN: om-dash-github-project-cards :repo "roc-streaming/roc-toolkit" :project 2 :column "In work" :type issue :state open
...
#+END:

./screenshot/github_project_cards.png

Tasks from org file

Display 1-level TODO tasks as tables with their child 2-level TODO tasks as table rows. Hide 1-level DONE tasks.

#+BEGIN: om-dash-orgfile :file "~/cloud/org/roc-toolkit.org" :todo 2 :done 0
...
#+END:

./screenshot/org_tasks.png

Unread email counters from IMAP

Display new and unread email counters for IMAP directory tree.

(setq om-dash-imap-host "imap.example.com"
      ;; Optional, if unset, default is used
      om-dash-imap-port 143
      ;; Optional, if unset, read from ~/.authinfo
      om-dash-imap-user "john"
      om-dash-imap-password "secret"
      ;; Optional, if unset, auto-detected for server
      om-dash-imap-stream 'network
      om-dash-imap-auth 'login)
#+BEGIN: om-dash-imap :folder "develop/roc"
...
#+END:

./screenshot/imap_counters.png

Custom command and template

Display table generated by a shell command.

#+BEGIN: om-dash-command :command "my-command arg1 arg2" :columns ("foo" "bar")
...
#+END:

./screenshot/shell_command.png

This example assumes that my-command produces output in JSON format like this:

[
  { "foo": "value1", "bar": "value2" },
  { "foo": "value3", "bar": "value4" }
]

If desired, you can define a template for your command to avoid repitition:

(defun my-command-template (params)
  (let ((args (plist-get params :args)))
    (list :headline (format "my command (%s)" args)
          :command (format "my-command %s" args)
          :columns '("foo" "bar"))))

(add-to-list 'om-dash-templates
           '(my-command . my-command-template))

Then you can use it like this:

#+BEGIN: om-dash-command :template my-command :args "arg1 arg2"
...
#+END:

Contributions

So far I’ve implemented only things that I needed for my own workflow, plus some reasonable customization. I have quite limited time for this project, so if you would like to extend it for your workflow, pull requests are very welcome!

Also, as I’ve never created elisp packages before, I probably missed some conventions or best practices. Again, patches are welcome.

Releases

Changelog file can be found here: changelog.

Installation

Required external tools:

To access private repos on github, follow official instructions.

Elisp dependencies:

Package was tested on Emacs 28.2 on Linux.

Instructions for straight.el:

;; required dependencies
(straight-use-package 'org-ql)
(straight-use-package 's)
(straight-use-package 'ts)

;; optional
(straight-use-package
 '(el-csv
  :type git
  :host github
  :repo "mrc/el-csv"
  :branch "master"
  :files ("parse-csv.el")))

;; om-dash
(straight-use-package
 '(om-dash
  :type git
  :host github
  :repo "gavv/om-dash"
  :branch "main"
  :files ("om-dash.el")))

Updater functions

The following functions can be used to update dynamic blocks (of any kind) in current document. You can bind them to org-mode-map or om-dash-mode-map.

org-update-all-dblocks

Update all dynamic blocks in the buffer. This function can be used in a hook.

org-dblock-update

User command for updating dynamic blocks. Update the dynamic block at point. With prefix ARG, update all dynamic blocks in the buffer.

(fn &optional ARG)

om-dash-update-tree

Update all dynamic blocks in current tree, starting from top-level entry.

E.g., for the following document:

* 1.               ---o
** 1.1    <- cursor   |
*** 1.1.1             | [tree]
*** 1.1.2             |
** 1.2             ---o
* 2.
** 2.1

the function updates all blocks inside 1., 1.1, 1.1.1, 1.1.2, 1.2.

om-dash-update-subtree

Update all dynamic blocks in current subtree, starting from current entry.

E.g., for the following document:

* 1.
** 1.1    <- cursor --o
*** 1.1.1             | [subtree]
*** 1.1.2           --o
** 1.2
* 2.
** 2.1

the function updates all blocks inside 1.1, 1.1.1, 1.1.2.

Dynamic blocks

This section lists dynamic blocks implemented by om-dash. Each block named om-dash-xxx corresponds to a function named org-dblock-write:om-dash-xxx.

om-dash-github-topics

Builds org heading with a table of github issues or pull requests.

Basic example:

#+BEGIN: om-dash-github-topics :repo "octocat/linguist" :type pullreq :open "*" :closed "-1w"
...
#+END:

More advanced example:

#+BEGIN: om-dash-github-topics :repo "octocat/hello-world" :type any :open ("comments:>2" ".title | contains(\"Hello\")") :sort "updatedAt" :limit 100
...
#+END:

Parameters:

parameterdefaultdescription
:reporequiredgithub repo in form “<owner>/<repo>“
:typerequiredtopic type (issue, pullreq, any)
:anymatch none (““)query for topics in any state
:openmatch all (“*“)query for topics in open state
:closedmatch none (““)query for topics in closed state
:sort“createdAt“sort results by given field
:fieldsom-dash-github-fieldsexplicitly specify list of fields
:limitom-dash-github-limitlimit number of results
:table-columnsom-dash-github-columnslist of columns to display
:headlineautotext for generated org heading
:heading-levelautolevel for generated org heading

A query for :any, :open, and :closed can have one of the two forms:

  • “github-query”
  • (“github-query” “jq-selector”)

github-query is a string using github search syntax: https://docs.github.com/en/search-github/searching-on-github/searching-issues-and-pull-requests

Besides standard syntax, a few extended forms are supported:

formdescription
“*“match all
“-123d“match if updated during last 123 days
“-123w“same, but weeks
“-123mo“same, but months
“-123y“same, but years

jq-selector is an optional selector to filter results using jq command: https://jqlang.github.io/jq/

You can specify different queries for open and closed topics, e.g. to show all open issues but only recently closed issues, use:

:open "*" :closed "-1mo"

Alternatively, you can use a single query regardless of topic state:

:any "-1mo"

Under the hood, the block uses combination of gh and jq commands like:

gh -R <repo> issue list \
      --json <fields> --search <github query> --limit <limit> \
  | jq '[.[] | select(<jq selector>)]'

(jq part is optional and is used only when the query has the second form when both github and jq parts are present).

Exact commands being executed are printed to *om-dash* buffer if om-dash-verbose is set.

By default, github query uses all fields from om-dash-github-fields, plus any field from om-dash-github-auto-enabled-fields if it’s present in jq selector.

The latter allows to exclude fields that makes queries slower, when they’re not used. To change this, you can specify :fields parameter explicitly.

om-dash-github-project-cards

Builds org heading with a table of github classic project cards.

Note: if you’re using new github projects (a.k.a. projects v2, a.k.a projects beta), which are currently default, then use om-dash-github-project-items instead.

Usage example:

#+BEGIN: om-dash-github-project-cards :repo "owner/repo" :project 123 :column "name" :type issue
...
#+END:

Parameters:

parameterdefaultdescription
:reporequiredgithub repo in form “<owner>/<repo>“
:projectrequiredproject identifier (number)
:columnrequiredproject column name (string)
:typerequiredtopic type (issue, pullreq, any)
:stateopentopic state (open, closed, any)
:sort“createdAt“sort results by given field
:fieldsom-dash-github-fieldsexplicitly specify list of fields
:limitom-dash-github-limitlimit number of results
:table-columnsom-dash-github-columnslist of columns to display
:headlineautotext for generated org heading
:heading-levelautolevel for generated org heading

:project field specifies project numeric identifier (you can see it in URL on github). :column field specifies the name of a column.

:type defines that types of cards to display: issues, pull requests, or all. :state defines whether to display open and closed issues and pull requests.

All other parameters are identical to om-dash-github-topics, see its docstring for more details.

om-dash-orgfile

Builds org headings with tables based on another org file.

Example usage:

#+BEGIN: om-dash-orgfile :file "~/my/file.org" :todo 2 :done 1
...
#+END:

Parameters:

parameterdefaultdescription
:filerequiredpath to .org file
:todo2nesting level for TODO entries
:done1nesting level for DONE entries
:digestnilgenerate single table with all entries
:table-columnsom-dash-orgfile-columnslist of columns to display
:headlineautotext for generated org headings
:heading-levelautolevel for generated org headings

This block generates an org heading with a table for every top-level (i.e. level-1) org heading in specified :file, with nested headings represented as table rows.

If :digest is t, a single table with all entries is generated, instead of separate table for every top-level entry.

Parameters :todo and :done limit how deep the tree is traversed for top-level headings in TODO and DONE states.

For example:

  • if :done is 0, then level-1 headings in DONE state are not shown at all
  • if :done is 1, then level-1 headings in DONE state are shown “collapsed”, i.e. org heading is generated, but without table
  • if :done is 2, then level-1 headings in DONE state are shown and each has a table with its level-2 children
  • if :done is 3, then level-1 headings in DONE state are shown and each has a table with its level-2 and level-3 children

…and so on. Same applies to :todo parameter.

Whether a heading is considered as TODO or DONE is defined by variables om-dash-todo-keywords and om-dash-done-keywords.

By default they are automatically populated from org-todo-keywords-1 and org-done-keywords, but you can set them to your own values.

:headline parameter defines text for org headings which contains tables. If :digest is t, there is only one table and :headline is just a string. Otherwise, there are many tables, and :headline is a format string where ‘%s’ is title of the top-level entry.

om-dash-imap

Builds org heading with a table of IMAP folder(s) and their unread mail counters.

Usage example:

#+BEGIN: om-dash-imap :folder "foo/bar"
...
#+END:
parameterdefaultdescription
:hostom-dash-imap-hostIMAP server hostmame
:portom-dash-imap-port or defaultIMAP server port
:machineom-dash-imap-machine or host~/.authinfo machine
:userom-dash-imap-user or ~/.authinfoIMAP username
:passwordom-dash-imap-password or ~/.authinfoIMAP password
:streamom-dash-imap-stream or autoSTREAM for imap-open
:authom-dash-imap-auth or autoAUTH for imap-open
:table-columnsom-dash-imap-columnslist of columns to display
:headlineautotext for generated org heading
:heading-levelautolevel for generated org heading

:host and :port define IMAP server address. Host must be always set, and port is optional.

:user and :password define IMAP credentials. If not set, om-dash-imap will read them from ~/.authinfo. If :machine is set, it’s used to search ~/.authinfo, otherwise host is used.

:stream and :auth may be used to force imap-open to use specific connection and authentification types. For example, you can use network and login values to force plain-text unencrypted password.

All these parameters have corresponding variables (e.g. om-dash-imap-host for :host) which are used if paremeter is omitted. Value is considered unset when both parameter is omitted and variable is nil.

om-dash-command

Builds org heading with a table from output of a shell command.

Usage example:

#+BEGIN: om-dash-command :command "curl -s https://api.github.com/users/octocat/repos" :format json :columns ("name" "forks_count")
...
#+END:
parameterdefaultdescription
:commandrequiredshell command to run
:columnsrequiredcolumn names (list of strings)
:formatjsoncommand output format (json or csv)
:headlineautotext for generated org heading
:heading-levelautolevel for generated org heading

If :format is json, command output should be a JSON array of JSON objects, which have a value for every key from :columns.

If :format is csv, command output should be CSV. First column of CSV becomes value of first column from :columns, and so on.

Note: using CSV format requires installing parse-csv package from https://github.com/mrc/el-csv

om-dash-function

Builds org heading with a table from output of a elisp function.

Usage example:

#+BEGIN: om-dash-function :fun example-func
...
#+END:
parameterdefaultdescription
:functionrequiredelisp function to call
:argsniloptional function arguments
:headlineautotext for generated org heading
:heading-levelautolevel for generated org heading

The function should return a list of tables, where each table is a plist with the following properties:

propertydefaultdescription
:keywordTODOkeyword for generated org heading
:headlineautotext for generated org heading
:levelautolevel for generated org heading
:column-namesrequiredlist of column names (strings)
:rowsrequiredlist of rows, where row is a list of cells (strings)

If :headline or :heading-level is provided as the block parameter, it overrides :headline or :level returned from function.

Example function that returns a single 2x2 table:

(defun example-func ()
  ;; list of tables
  (list
   ;; table plist
   (list :keyword "TODO"
         :headline "example table"
         :column-names '("foo" "bar")
         :rows '(("a" "b")
                 ("c" "d")))))

Templates

This section lists built-in templates provided by om-dash. You can define your own templates via om-dash-templates variable.

om-dash-github:milestone

Template for om-dash-github-topics block to display topics from given milestone.

Can be used as :template milestone with om-dash-github-topics block.

Usage example:

#+BEGIN: om-dash-github-topics :template milestone :repo "owner/repo" :type issue :milestone "name"
...
#+END:

Parameters:

parameterdefaultdescription
:reporequiredgithub repo in form “<owner>/<repo>“
:typerequiredtopic type (issue, pullreq, any)
:stateopentopic state (open, closed, any)
:milestonerequiredmilestone name (string)
:headlineautotext for generated org heading
:heading-levelautolevel for generated org heading

Any other parameter is not used by template and passed to om-dash-github-topics as-is.

Minor mode

om-dash-mode

om-dash minor mode.

This is a minor mode. If called interactively, toggle the ‘OM-Dash mode’ mode. If the prefix argument is positive, enable the mode, and if it is zero or negative, disable the mode.

If called from Lisp, toggle the mode if ARG is toggle. Enable the mode if ARG is nil, omitted, or is a positive number. Disable the mode if ARG is a negative number.

To check whether the minor mode is enabled in the current buffer, evaluate om-dash-mode.

The mode’s hook is called both when the mode is enabled and when it is disabled.

This minor mode for .org files enables additional highlighting inside org tables generated by om-dash dynamic blocks.

Things that are highlighted:

  • table header and cell (text and background)
  • org-mode keywords
  • issue or pull request state, number, author
  • tags

After editing keywords list, you need to reactivate minor mode for changes to take effect.

To active this mode automatically for specific files, you can use local variables (add this to the end of the file):

# Local Variables:
# eval: (om-dash-mode 1)
# End:

Variables

om-dash-todo-keywords

List of keywords considered as TODO.

If block has any of the TODO keywords, block’s heading becomes TODO. The first element from this list is used for block’s heading in this case.

If a keyword from this list doesn’t have a face in om-dash-keyword-faces, it uses default TODO keyword face.

When nil, filled automatically from org-todo-keywords, org-done-keywords, and pre-defined github keywords.

om-dash-done-keywords

List of keywords considered as DONE.

If block doesn’t have any of the TODO keywords, block’s heading becomes DONE. The first element from this list is used for block’s heading in this case.

If a keyword from this list doesn’t have a face in om-dash-keyword-faces, it uses default DONE keyword face.

When nil, filled automatically from org-todo-keywords, org-done-keywords, and pre-defined github keywords.

om-dash-keyword-faces

Assoc list to map keywords to faces.

If some keyword is not mapped to a face explicitly, default face is selected, using face for TODO or DONE depending on whether that keyword is in om-dash-todo-keywords or om-dash-done-keywords.

om-dash-tag-map

Assoc list to remap or unmap tag names.

Defines how tags are displayed in table. You can map tag name to a different string or to nil to hide it.

om-dash-templates

Assoc list of expandable templates for om-dash dynamic blocks.

Each entry is a cons of two symbols: template name and template function.

When you pass “:template foo” as an argument to a dynamic block, it finds a function in this list by key foo and uses it to “expand” the template.

This function is invoked with dynamic block parameters plist and should return a new plist. The new plist is used to update the original parameters by appending new values and overwriting existing values.

For example, if org-dblock-write:om-dash-github-topics block has parameters:

(:template milestone
 :repo "owner/repo"
 :type 'issue
 :milestone "1.2.3")

Dynamic block will use milestone as a key in om-dash-templates and find om-dash-github:milestone function.

The function is invoked with the original parameter list, and returns a modified parameter list:

(:repo "owner/repo"
 :type 'issue
 :headline "issues (owner/repo \"1.2.3\")"
 :open "milestone:\"1.2.3\""
 :closed "")

Then modified parameters are interpreted by dynamic block as usual.

om-dash-table-fixed-width

If non-nil, align tables to have given fixed width. If nil, tables have minimum width that fits their contents.

om-dash-table-squeeze-empty

If non-nil, automatically remove empty columns from tables. E.g. if every row has empty tags, :tags column is removed from this table.

om-dash-table-link-style

How links are generated in om-dash tables.

Allowed values:

  • :none - no links are inserted
  • :text - only cell text becomes a link
  • :cell - whole cell becomes a link

om-dash-github-columns

Column list for om-dash-github-topics table.

Supported values:

symbolexample
:stateOPEN, CLOSED, …
:number#123
:author@octocat
:milestone1.2.3
:titletext
:title-link[​[link][text]]
:tags:tag1:tag2:…:

om-dash-orgfile-columns

Column list for om-dash-orgfile table.

Supported values:

symbolexample
:stateTODO, DONE, …
:titletext
:title-link[​[link][text]]
:tags:tag1:tag2:…:

om-dash-imap-columns

Column list for om-dash-imap table.

Supported values:

symbolexample
:stateNEW, UNREAD, CLEAN
:new10
:unread20
:total30
:folderfoo/bar

om-dash-github-limit

Default limit for github queries.

E.g. if you query “all open issues” or “closed issues since january”, only last om-dash-github-limit results are returned.

om-dash-github-fields

List of json fields enabled by default in github queries.

This defines which fields are present in github responses and hence can be used in jq selectors.

We don’t enable all fields by default because some of them noticeably slow down response times.

There is also om-dash-github-auto-enabled-fields, which defines fields that are enabled automatically for a query if jq selector contains them.

In addition, org-dblock-write:om-dash-github-* accept :fields parameter, which can be used to overwrite fields list per-block.

om-dash-github-auto-enabled-fields

List of json fields automatically enabled on demand in github queries.

See om-dash-github-fields for more details.

om-dash-imap-host

Default IMAP server hostname.

Used by om-dash-imap if :host parameter is not provided. Host must be always set, either via :host or om-dash-imap-host.

om-dash-imap-port

Default IMAP server port number.

Used by om-dash-imap if :port parameter is not provided. If port is not set, default IMAP port is used.

om-dash-imap-machine

Default ~/.authinfo machine for IMAP server.

Used by om-dash-imap if :machine parameter is not provided. If machine is not set, value of host is used.

om-dash-imap-user

Default username for IMAP server.

Used by om-dash-imap if :user parameter is not provided. If user is not set, it’s read from ~/.authinfo. See also om-dash-imap-machine.

om-dash-imap-password

Default username for IMAP server.

Used by om-dash-imap if :password parameter is not provided. If password is not set, it’s read from ~/.authinfo. See also om-dash-imap-machine.

om-dash-imap-stream

Default STREAM parameter for imap-open.

Used by om-dash-imap if :stream parameter is not provided. Must be one of the values from imap-streams. If nil, detected automatically.

om-dash-imap-auth

Default AUTH parameter for imap-open.

Used by om-dash-imap if :auth parameter is not provided. Must be one of the values from imap-authenticators. If nil, detected automatically.

om-dash-imap-empty-folders

Whether to display empty IMAP folders. If nil, empty folders are excluded from the table.

om-dash-verbose

Enable verbose logging. If non-nill, all commands and queries are logged to *om-dash* buffer.

Faces

om-dash-header-cell

Face used for entire cell in om-dash table header. You can use it so specify header background.

om-dash-header-text

Face used for text in om-dash table header. You can use it so specify header font.

om-dash-cell

Face used for entire non-header cell in om-dash table. You can use it so specify cell background.

om-dash-text

Face used for text in om-dash table non-header cell. You can use it so specify cell font.

om-dash-number

Face used for issue or pull request numbers in om-dash tables.

om-dash-author

Face used for issue or pull request authors in om-dash tables.

om-dash-todo-keyword

Face used for TODO keyword in om-dash tables.

om-dash-done-keyword

Face used for DONE keyword in om-dash tables.

om-dash-open-keyword

Face used for OPEN keyword in om-dash tables.

om-dash-merged-keyword

Face used for MERGED keyword in om-dash tables.

om-dash-closed-keyword

Face used for CLOSED keyword in om-dash tables.

om-dash-new-keyword

Face used for NEW keyword in om-dash tables.

om-dash-unread-keyword

Face used for UNREAD keyword in om-dash tables.

om-dash-clean-keyword

Face used for CLEAN keyword in om-dash tables.

Authors

See here.

License

GPLv3+