Skip to main content

Inline Diagrams in Orgmode

orgmode-export.svg

Of course it is easy to include a diagram with a post, but the standard procedure to create/edit the diagram with an external GUI tool does not resonate with me. Even back in my diploma thesis I fell in love with Donald Knuth's MetaPost system to describe diagrams in a declarative way and let the tool do the drawing.

For some LaTeX Beamer presentations I used ditaa in the meantime but inside a make flow that took care of the dependencies. The diagrams and the presentation were strictly separated. For the blog posts I did not yet have a good solution, until I switched to blogging in orgmode. Compared to ReST and Asciidoc posts, Emacs orgmode provides a much richer ecosystem with lots of hooks into other systems by means of the org-babel system.

So by switching the post source format, blogging in orgmode should actually hook into e.g. Graphviz out of the box. And indeed with a little fine tuning, this is an amazingly elegant way to include visualizations in the posts.

As org-mode (through exporting to LaTex Beamer) has also become the source format of my presentations, this new freedom easily extends into presentations and even regular PDF documents by exporting to LaTeX only. So the means shown here to produce diagrams for the blog are also usable when you want to generate Beamer presentations or LaTeX documents. As we will see, the choice of output formats is linked to the targeted use-case.

Orgmode and org-babel

The foundation of using this class of diagrams is the org-babel system. It took me a light tweaking of the orgmode installation, and without further ado, here is the complete use-package clause I currently use for my setup:

;; -------------------------------------
;; Org mode
;; -------------------------------------
(use-package org-mode
  :init
  ;; This allows PlantUML, Graphviz and ditaa diagrams
  (org-babel-do-load-languages
   'org-babel-load-languages
   '((ditaa . t)
     (dot . t)
     (plantuml . t)))

  :hook
  (org-babel-after-execute . org-redisplay-inline-images)

  :custom
  (org-edit-src-content-indentation 0)
  ;; PlantUML was too old on Debian Bookworm, so a recent copy is
  ;; installed in /usr/local/share
  (org-plantuml-jar-path "/usr/local/share/plantuml/plantuml.jar")
  ;; ditaa installed through dpkg on Debian
  (org-ditaa-jar-path "/usr/share/ditaa/ditaa.jar")
  ;; Do not ask before evaluating a code block
  (org-confirm-babel-evaluate nil)
  ;; Fix for including SVGs
  (org-latex-pdf-process
   '("%latex -shell-escape -interaction nonstopmode -output-directory %o %f"
     "bibtex %b"
     "%latex -shell-escape -interaction nonstopmode -output-directory %o %f"
     "%latex -shell-escape -interaction nonstopmode -output-directory %o %f"))
  )

The list for org-babel-load-languages specifies the languages that we want org-babel to process. Nothing special here. The :hook clause adds a call to org-redisplay-inline-images on the org-babel-after-execute-hook (note that the :hook syntax of use-package inserts the -hook suffix of the customized hook by itself). This will redisplay the images in the buffer after generating them with C-c C-c.

PlantUML and ditaa are Java based tools and so we need to specify the path to the jar files. Note that currently Debian Bookworm still carries "1.2020.02" and the support for the new activity diagrams is not up to the documentation on the web. So I simply downloaded a newer version of the jar file to /usr/local/shar/plantuml and specify this version to orgmode.

Setting org-confirm-babel-evaluate to nil spares me to answer "yes" on every run of the org-babel flow. And finally messing with org-latex-pdf-process is nothing that I like to do, but for SVG support, it is required to have the -shell-escape argument on the LaTeX command line.

Without a further trick, the images will not be displayed when visiting such an orgmode document for the first time. We can fix this by including a STARTUP option:

#+STARTUP: inlineimages

With all this in place, creating a diagram is as easy as inserting a source code block with the correct type and pressing C-c C-c inside it. org-babel will then evaluate the code block, i.e. run the external tool and insert a link to the resulting file into the buffer. As orgmode honors the :exports option on such source blocks and for diagrams they default to output rather than code, so the export will only have the diagram and leave out the source code. Specifying :exports both (like done below) will export both sections, making it extra easy to show the source code and the result in a document.

Here is a screen shot of Emacs displaying parts of this post:

Emacs screenshot of this post

Graphviz

The dot language from the Graphviz package allows us to draw images like the one at the top of the post. The language only describes the entities in the diagram and their relationships. Positioning is done by the tool automatically with the help of hints that we can include in the representation like connection weights, etc.

The list of possible output formats (specified with -T) is very extensive and includes:

  • PNG
  • SVG
  • EPS
  • PDF

For best results, I always try to use vector based formats, so they can be scaled to the intended target resolution without loss of quality. I really do hate graphics included into documents with a low quality, but I will return to the choice of output formats in a later section.

This is the source for the diagram at the top of this post:

digraph G {
  { rank=same;HTML;PDF; }
  { rank=same;Orgexport;Nikola; }
  Orgbabel [color=lightblue,style=filled,label="org-babel (ob)"];
  Orgmode [color=limegreen,style=filled];
  Graphviz [color=lightblue,style=filled];
  PlantUML [color=lightblue,style=filled];
  others [color=lightblue,style=filled,label="..."];
  Orgexport [color=gold,style=filled,label="org-export (ox)"];
  LaTeX [color=gold,style=filled];
  Beamer [color=gold,style=filled];
  PDF [color=gold,style=filled];
  HTML [color=gold,style=filled];
  Orgnikola [color=orange,style=filled,label="org-nikola"];
  Nikola [color=orange,style=filled,label="Nikola (blog)"];
  Orgbabel -> Orgmode -> Orgbabel;
  Orgbabel -> Graphviz -> Orgbabel;
  Orgbabel -> PlantUML -> Orgbabel;
  Orgbabel -> others -> Orgbabel;
  Orgmode -> Orgexport [weight=2];
  Orgexport -> HTML;
  Nikola -> Orgnikola -> Orgexport;
  Nikola -> HTML;
  Orgexport -> LaTeX;
  Orgexport -> Beamer;
  LaTeX -> PDF;
  Beamer -> PDF;
}

PlantUML

PlantUML is another package to generate diagrams from textual descriptions. It originated with UML diagrams, but nowadays supports a lot of other attractive diagram types. It also is capably of generating a lot of different output formats, again including:

  • PNG
  • SVG
  • EPS
  • PDF

So PlantUML can also be used to generate high quality vector graphics, preventing nasty bitmap artifacts from scaling. Actually PlantUML derives the output type from the output filename, so it is enough to specify graph.svg as the output file to generate SVG.

The homepage lists ~25 different diagram types. I'll show a quick example of the ones I believe will be relevant for me.

Note that like mentioned above, using :exports both in the org-babel code will automatically include the source code and the result without any need of manual duplication. I find this hugely satisfying.

Sequence Diagrams

Here is a simple "sequence diagram":

Client -> Server : Ping
Server -> Client : Pong

sequence-ping.svg

Activity Diagrams

Activity diagrams offer a lot of possibilities to visualize decision processes. Here is a little example to visualize the boot phase of an embedded device:

@startuml
:FSBL;
:U-Boot;
if (SD card\nfound) then (yes)
  :Load kernel\nfrom SD;
  :Setup cmdline for\nrootfs from SD;
elseif (USB medium\nfound) then (yes)
  :Load kernel\nfrom USB;
  :Setup cmdline for\nrootfs from USB;
elseif (eMMC found) then (yes)
  :Load kernel\nfrom eMMC;
  :Setup cmdline for\nrootfs from eMMC;
else (nothing)
  #pink:Drop to shell;
  kill
endif
:Boot Linux kernel;
@enduml

activity-boot.svg

Mind Maps

This diagram type allows nice visualizations of hierarchies. Here is a little example visualizing Zephyr OS:

@startmindmap
+[#FFBBCC] Zephyr OS
++[#lightblue] 3rd Party Libraries
++[#FFBBCC] Application Services
+++[#FFBBCC] MQTT
+++[#FFBBCC] HTTP
+++[#FFBBCC] CoAP
++[#FFBBCC] OS Services
+++[#FFBBCC] TCP/UDP
+++[#FFBBCC] I2C
+++[#FFBBCC] File system
++[#FFBBCC] Kernel
+++[#FFBBCC] Services / Scheduler
+++[#FFBBCC] Power Management
++[#lightblue] HAL
@endmindmap

ob-zephyr-overview.svg

Ditaa

Diagrams Through Ascii Art (ditaa) is a different approach to generating diagrams from text. Instead of using declarative languages, the "source code" is an Ascii drawing generated by any means preferred by the user.. A plain text editor will do, or Artist Mode within the Emacs text editor is an especially elegant way to draw them. Ditaa will then convert this into a nice looking diagram, albeit only as a bitmap, e.g. PNG. It is not possible to generate an SVG with Ditaa, but we can circumvent this by using the Ditaa integration into PlantUML. As PlantUML can output SVG files, we gain one degree of freedom for high quality output by going this way.

Here is a simple example showing the user vs. kernel space split of a regular Linux kernel:

@startditaa
+-------------+ +-------------+ +-------------+
|cBLU         | |cBLU         | |cBLU         |
|  Process 1  | |  Process 2  | |  Process 3  |
|             | |             | |             |
+-------------+ +-------------+ +-------------+
       ^               ^               ^          User space
       |        System |               |  ------------------
       v         call  v               v        Kernel space
+---------------------------------------------+
| Virtual memory,        cPNK Linux Kernel    |
| Scheduler, Device driver                    |
| Filesystems, IPC, ...                       |
+---------------------------------------------+
                                          ------------------
+-----------+-----------+-----------+---------+     Hardware
| Ethernet  |    CAN    |    I2C    |  .....  |
+-----------+-----------+-----------+---------+
@endditaa

ditaa-userspace.svg

Which file format?

Knowing our tools, let's think about which of the file formats to use. As a summary, both tools support these relevant types:

  • PNG (pixel based)
  • EPS (vector based)
  • PDF (vector based)
  • SVG (vector based)

Whenever possible, we should use one of the vector based formats, but we have to be sure that the rest of our "toolchain" can handle those formats. So let's discuss the use-cases I care about.

HTML output (blog)

As most browsers nowadays support SVG, I will use SVG for the blog. Indeed all pictures in this post are SVGs. Whenever you think this will be problematic, switching to PNG is easy.

LaTeX output (plain and beamer)

LaTeX can easily digest PDF and EPS diagrams, so it is conceivable to use those formats for the LaTeX export. The drawback of this approach is that Emacs is currently not able to preview either PDF or EPS files in the buffer like shown in the previous screenshot. Although this may change, it is enough for me to opt for SVGs also in this use-case.

Using SVG in LaTeX requires the svg and svg-extract packages to work. In this setup, LaTeX can even extract the text components of an SVG and render them with its own fonts (even allowing LaTeX maths). This is somewhat overkill for my usecase and does not interoperate nicely with the Emacs preview, so I disable it in the LaTeX preamble. For an orgmode buffer, this can be achieved like this:

#+latex_header: \usepackage[inkscapelatex=false]{svg}

Summary

It took quite a bit of experimentation to arrive at this setup, but it was certainly worth it:

  • I can now easily create high-quality diagrams not only in blog posts, but also in PDF documents and presentations. Sharing material is as easy as copying text blocks around.
  • Creating and modifying the diagrams is part of the writing process and does not involve any other program.
  • As the diagrams are text constituents of the files, they will automatically be checked into version control and even differences in the diagrams itself are documented this way fully transparently. This is the logical extension of generating code documentation from the code itself.

Update 2024-02-22

Added PlantUML Mindmap example

Comments

Comments powered by Disqus