Skip to main content

Blog internals (or learning new things to fix stuff)

nikola-50px-transparent.png

Recently I was made aware of a consequential GDPR ruling in Germany. The summary of the ruling is that a website owner was sentenced to pay a €100 fine to a visitor of his website because the website used Google hosted fonts. The visitor claimed that by loading the Google fonts, his IP address was exposed to the Google servers and this violates the GDPR.

Effectively this exposes every website in Germany using Google hosted web fonts to become the target of an €100 fine. At first, it seemed like a theoretic problem, but more recently it seems that people have turned this into a business-model (link is in German) and there now seem to be a wave of incidents based on this ruling. So it has become pretty urgent to fix this for web site owners in Germany.

Checking my Nikola generated site, I found that the zen theme that I use not only uses Google hosted fonts but also the Fontawesome icon set hosted on its own server. So I quickly decided that I need to switch those over to self-hosting before I get one of these lawyer mails. This blog post documents the things I learned on this journey which is still not over.

And I apologize in advance that this post has become rather large and very technical, but the things are really connected together so I decided not to split them out into smaller posts. I sincerely hope this was the right thing to do.

Finding out what to do

As I am not a web programmer, I tried to get an idea of what is required to self-host the Google fonts and the Fontawesome icon set. Searching the web quickly yielded some articles for the Google fonts problem and Fontawesome has pretty good documentation itself on how to self-host. So the next step would be to integrate this into the zen theme that I use and see how much of this can and should be contributed back to the Nikola project.

Unfortunately until now I used Nikola as a black box that (mostly) did what I expected of it, but to solve this issue I obviously needed to learn more about the inner workings of it. With my specific changes in mind I decided to concentrate my learning on the theming aspect of Nikola, i.e. on the help documents Creating a Theme and Theming Nikola. This entailed reading up on other topics related to web development as well. Studying Cascading Style Sheets, for example, I realized that "basic CSS" seems to be so uncomfortable that people came up with higher level languages compiling into CSS, like sass or less. The zen theme has this to say in its README.md file:

WARNING: The themes use Less for their styles, but you don't need a Less compiler installed to use it.

So we should keep in mind that theme.css is the output of a compiler with the source files located in the less/ directory.

How themes work in Nikola

Nikola comes with 6 themes contained in the Nikola repository itself:

  • base, base-jinja
  • bootblog4, bootblog4-jinja
  • bootstrap4, bootstrap4-jinja

Apart from those, there are also quite a lot of community themes. Looking at the themes gallery, it is not obvious at all, which of the themes are "first level" themes living next to the code itself and which live in their own nikola-themes GitHub repository. Once you decide to use a Nikola theme, you will install a copy of it into the directory you build your blog from and thus loose the connection to improvements in the upstream repository. Unless you check yourself from time to time if something happened there, you will not be able to leverage any subsequent changes or fixes in the upstream theme. If you do find there were changes, there is no easy way to upgrade your theme then to remove your old one, install the new one and see if things still work as you expect.

As I was sure my problem is also relevant for a lot of Nikola users, I explicitly set out to fix things in the proper "upstream" repository. It turns out that community Nikola themes have their own and community Nikola plugins (see below) also live in their own plugins GitHub repo.

Establish a clean baseline

As we learned, the theme used in the blog is a copy of the upstream theme at the time of theme installation. So before doing anything, I decided that I would need to synchronize the theme used in my blog with its current version in the upstream repository. So I renamed theme/zen to theme/zen-old and re-installed the zen theme according to the manual. Having a clean slate would now allow me to introduce my changes to self-host the fonts.

To be on the safe side, I wiped the whole output/ directory of my blog and recreated the site from scratch. As I was only doing incremental builds over the last years, I was also interested in how long a complete build would take - turns out its around 11 seconds.

Uh, an error in the web console?

Opening the newly recreated blog on my local machine looked okay, but opening the web console of Firefox showed a prominent error:

Uncaught ReferenceError: moment is not defined

As I was expecting to start my changes from a clean state, I was surprised to see this error coming from the upstream code base. So I tried inquiring on the Nikola mailing list what this was all about. Not getting any helpful information, I tried to find the reason for myself. After some poking I found that the implementation of "fancy dates" (turning absolute date and timestamps into something like "10 hours ago") in the base themes was switched in commit f1d480d8 from a solution using moment-with-locales.min.js to a solution using luxon.min.js. So it looks like this change in the core Nikola repository was not propagated to at least some)the zen theme in the community themes repository.

So before even considering the changes required for self-hosting, I decided to fix this bug while still here.

Fixing things in Nikola

A closer look at the "zen" family of themes

On a cursory look, the community themes include three themes with zen in their names:

  • zen
  • zen-jinja
  • zen-forkawesome

At the beginning I thought I would only deal with zen itself, but learning more about the inner workings of themes, I discovered that zen-jinja is actually derived with the script jinjify.py from the zen templates. While zen uses the Mako template engine, zen-jinja uses (what a surprise) the jinja library. As this is outside my expertise, I had no idea which one is "better" (for a meaningful interpretation of "better", i.e. in terms of design, larger community, etc.) but at least I found a discussion from 2017 in the Mako Development mailing list where the creator of Mako (Mike Bayer) writes:

I can tell you that Jinja is a lot more polished than Mako at this point. It was developed later with a more organized concept to start with, and it has multiple developers now continuing to put a lot of work into it. Mako of course works great and it's still what I use, but that's because I wrote it :). I don't have the resources to maintain it beyond keeping it basically working these days and I've usually never been able to get dedicated long-term helpers on any of my projects.

Along the way I also found that jinja is used in the very popular ansible orchestration package and thus from a practical perspective probably has a lot more users than Mako today. But even with jinja having a slight advantage, for "twin" themes inside Nikola having both a Mako and a Jinja version, it will be easier to do the changes in the Mako templates and update the twin with the script. So whenever I changed something in zen, I used the script to update zen-jinja also for the twins to stay synchronized.

zen-forkawesome on the other hand turns out to be a theme derived from zen, replacing the Fontawesome icon set with the Fork-Awesome icons. "Deriving" in this context means the theme references zen as its base in zen-forkawesome/parent and provides own versions of base_helper.tmpl and index.tmpl with the changes to replace Fontawesome with Fork-Awesome. But the fact that this "derivation" includes a copy of all the text of the base templates, means that we also have to fix these files with similar fixes that we used for the files in zen. There is no tool for these changes, so we have to derive them manually. This is a consequence of the very primitive "template inheritance" mechanism being used. As we saw, "inheriting" really means to create a copy of a file and only change minor details while keeping the rest in place, but without preserving any relationship to its ancestor that tools could use.

Pull request for nikola-themes

In essence I knew enough to fix the problems, but I would need to make changes similar to the changes in the base Nikola themes for all the zen family. This resulted in a pull request on nikola-themes.

With this in place, I could proceed to the self-hosting, but during the previous work I noticed that the versions of Fontawesome and Fork-Awesome were actually outdated. So why not update them while we are here, so we do not invest work in outdated versions?

Updating the icon sets

Updating the version of Fontawesome is done in commit zen: Update fontawesome from v5.0.13 to 6.1.2 in my clone of the nikola-themes repo (remember it fixes zen-jinja also!). The update of forkawesome is in commit zen-forkawesome: Update from v1.1.7 to v1.2.0 . All of these fixes are now in the zen-self-host-fonts-fa branch in my clone.

Honoring 'updated'

During all of this fixing, I came across the updated metadata (see section 'Metadata field' in the Nikola handbook) of a post and realized that I very much would like to have this on my blog. It happened already a few times that I made updates to my blog posts in the form of changes summarized at the bottom under an 'Update' section. Although this update section contains the date of the changes, this date was not visible in the header area in my blog. Researching this question, I found that (of course) it needs separate machinery in the templates and the zen family did not have this machinery.

Fixing this results in the commit zen family: Add handling for 'updated' pages taken from base.

The effect of this change can be seen for example in the jq, xq and yq - Handy tools for the command line post on my blog. Just beneath the header there is now the text "2 years ago (updated 11 months ago)" which is a very nice summary of the facts provided by the date and updated metadata for the post. You can check yourself by looking at the source of the post.

Self-hosting, finally!

With all this in place, downloading the files for Google Fonts and for Fontawesome and including their usage in the templates was not a big deal anymore:

Self-hosting zen-forkawesome remains to be done, as I wanted to gather feedback on the other changes before doing this.

Background from XScreenSaver

And finally as I learned that the background is of course also replaceable an idea began to form in my mind. Only recently Jamie Zawinski announced the 30th anniversary of XScreenSaver and this reminded me of my time at TU Karlsruhe when I was staring a lot at the graphical gems contained in the XScreenSaver collection. By default, XScreenSaver seems not to be installed in a standard Debian Bookworm installation anymore, but a quick

dzu@krikkit:~$ sudo apt install xscreensaver \
 xscreensaver-data-extra xscreensaver-gl-extra

fixes the situation for me. This allows me to start the GUI for browsing the included screensaver modules:

dzu@krikkit:~$ xscreensaver-demo 

The list is so long that I only looked at a subset from the beginning of the list. But many entries brought back memories from some 25 years ago.

After a while I decided a snapshot from flipflop would be nice to have. Playing around with the parameters I found that the installation in Debian puts all the binaries into /usr/libexec/xscreensaver, so they are not on the regular PATH for any user. XScreenSaver also has a tool called xscreensaver-getimage which is called by some graphical demos to retrieve a screenshot, a random picture or even a snapshot from the webcam. For flipflop the option --texture will make it call out to that program to get a random picture and put it inside a specific window. But calling flipflop --texture from the command line does not have a correct PATH setup and so xscreensaver-getimage cannot be found. This results in a black/dark-grey test pattern to end up being used in the demo hack. Even though it's the result of a user error, I found the irony somehow fitting with my blog as I do write about problems quite a lot.

So after taking a screenshot of flipflip and cropping it to the same x dimension of the original background of the zen theme I had a .png file I wanted to use. Copying it into the assets/images directory in the themes/zen folder in the blog itself will copy it to the output/ directory when building the blog. Searching for the exact place where the old background is specified, I found the file name referenced in assets/css/theme.css. Changing it there is easy, but remember what we found in the beginning of the post, namely that theme.css is a file generated by a less compiler. So the proper place to change the variables is in the source file less/variables.less:

// Resources
//----------
@sidebarBackground: url('../images/blue-mocha-grunge.jpg') @darkBlue;
@primaryBackground: url('../images/cream-dust.png') @white;

lessc and lessc

Having changed this variable definition, one has to compile the less code into theme.css. Googling around for less compilers I happened to come across the Ruby version first. Installation went about right after updating my Ruby installation on my Debian machine. Quickly there was a /usr/local/bin/lessc, so I could run make in the zen theme directory to compile the less code. Checking the magit (Emacs git "user interface") diff output, of course I expected only the file name to change but nothing else. But the diff was rather large:

diff --git a/v8/zen/assets/css/theme.css b/v8/zen/assets/css/theme.css
index 6bc845f..4378fb2 100644
--- a/v8/zen/assets/css/theme.css
+++ b/v8/zen/assets/css/theme.css
@@ -232,7 +232,7 @@ html {
 }
 body {
   height: 100%;
-  background: url('../images/cream-dust.png') #fff;
+  background: url('../images/cream-dust.png') #ffffff;
 }
 section {
   margin: 0;
@@ -241,7 +241,7 @@ section {
 .social {
   float: left;
   min-height: 100%;
-  background: url('../images/flipflop-texture.png') #192029;
+  background: url('../images/blue-mocha-grunge.jpg') #192029;
   position: absolute;
   top: 0;
   bottom: 0;
@@ -443,8 +443,8 @@ dd {
   padding: 4px 12px;
   line-height: 24px;
   text-decoration: none;
-  background-color: #fff;
-  border: 1px solid #ddd;
+  background-color: #ffffff;
+  border: 1px solid #dddddd;
   border-left-width: 0;
 }
 .pagination ul > li > a:hover,
@@ -455,14 +455,14 @@ dd {
 }
 .pagination ul > .active > a,
 .pagination ul > .active > span {
-  color: #999;
+  color: #999999;
   cursor: default;
 }
 .pagination ul > .disabled > span,
 .pagination ul > .disabled > a,
 .pagination ul > .disabled > a:hover,
 .pagination ul > .disabled > a:focus {
-  color: #999;
+  color: #999999;
   background-color: transparent;
   cursor: default;
 }
@@ -591,7 +591,7 @@ dd {
 .pager .disabled > a:hover,
 .pager .disabled > a:focus,
 .pager .disabled > span {
-  color: #999;
+  color: #999999;
   background-color: #fff;
   cursor: default;
 }
@@ -599,10 +599,10 @@ body {
   font-size: 16px;
   font-size: 15px;
   line-height: 24px;
-  color: #333;
+  color: #333333;
 }
 a {
-  color: #08c;
+  color: #0088cc;
   text-decoration: none;
 }
 a:hover,
@@ -611,7 +611,7 @@ a:focus {
   text-decoration: underline;
 }
 .social a {
-  color: #fff;
+  color: #ffffff;
   line-height: 44.4px;
   -webkit-transition: text-shadow 0.5s;
   -moz-transition: text-shadow 0.5s;
@@ -629,7 +629,7 @@ a:focus {
 .social a:hover,
 .social a:focus {
   text-shadow: 0 0 10px rgba(255, 255, 255, 0.8);
-  color: #fff;
+  color: #ffffff;
   text-decoration: none;
   opacity: 0.9;
   filter: alpha(opacity=90);
@@ -697,7 +697,7 @@ a:focus {
 .page-content > .content .h6 small {
   font-weight: normal;
   line-height: 1;
-  color: #999;
+  color: #999999;
 }
 .page-content > .content h1,
 .page-content > .content .h1,
@@ -765,7 +765,7 @@ a:focus {
 blockquote {
   padding: 0 0 0 15px;
   margin: 0 0 24px;
-  border-left: 5px solid #eee;
+  border-left: 5px solid #eeeeee;
 }
 blockquote p {
   margin-bottom: 0;
@@ -773,7 +773,7 @@ blockquote p {
 blockquote small {
   display: block;
   line-height: 24px;
-  color: #999;
+  color: #999999;
 }
 blockquote small:before {
   content: '\2014 \00A0';
@@ -782,7 +782,7 @@ blockquote.pull-right {
   float: right;
   padding-right: 15px;
   padding-left: 0;
-  border-right: 5px solid #eee;
+  border-right: 5px solid #eeeeee;
   border-left: 0;
 }
 blockquote.pull-right p,
@@ -865,7 +865,7 @@ address {
 }
 a.thumbnail:hover,
 a.thumbnail:focus {
-  border-color: #08c;
+  border-color: #0088cc;
   -webkit-box-shadow: 0 1px 4px rgba(0, 105, 214, 0.25);
   -moz-box-shadow: 0 1px 4px rgba(0, 105, 214, 0.25);
   box-shadow: 0 1px 4px rgba(0, 105, 214, 0.25);
@@ -878,7 +878,7 @@ a.thumbnail:focus {
 }
 .thumbnail .caption {
   padding: 9px;
-  color: #555;
+  color: #555555;
 }
 code,
 tt,
@@ -886,7 +886,7 @@ pre {
   padding: 0 3px 2px;
   font-family: Monaco, Menlo, Consolas, "Courier New", monospace;
   font-size: 13px;
-  color: #333;
+  color: #333333;
   -webkit-border-radius: 3px;
   -moz-border-radius: 3px;
   border-radius: 3px;
@@ -971,7 +971,7 @@ article.post-text .backlink + h1.title {
 }
 article.post-text .metadata {
   *zoom: 1;
-  color: #999;
+  color: #999999;
   margin-bottom: 12px;
   margin-top: -6px;
 }
@@ -990,14 +990,14 @@ article.post-text .metadata .commentline {
 article.post-text .metadata p {
   margin: 0;
   display: inline-block;
-  color: #999;
+  color: #999999;
 }
 article.post-text .metadata p a {
-  color: #999;
+  color: #999999;
 }
 article.post-text .metadata p a:hover,
 article.post-text .metadata p:hover {
-  color: #333;
+  color: #333333;
   -webkit-transition: color 1s linear;
   -moz-transition: color 1s linear;
   -o-transition: color 1s linear;
@@ -1073,7 +1073,7 @@ article.post-text .entry-content .figure > a > img {
 }
 article.post-text .entry-content .figure .caption {
   padding: 9px;
-  color: #555;
+  color: #555555;
   font-style: italic;
   padding-bottom: 0;
 }

So somehow it seemed that the less code did not correspond to the compiled file theme.css. I found that strange, but it is a possible consequence of using the compiled sources in a version controlled repository. Changes might escape attention, and so I streamlined the pure re-creation of theme.css through the lessc compiler into its own git commit and proceeded to change the variable in a subsequent commit yielding a proper, short, diff only for the filename. Only later, while reading up more on the less language, I found out that there is yet another compiler for the node ecosystem and also easily installable on my Debian machine with

dzu@krikkit:~$ sudo apt install node-less
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  node-less
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 525 kB of archives.
After this operation, 4346 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bookworm/main amd64 node-less all 3.13.0+dfsg-8 [525 kB]
Fetched 525 kB in 0s (3617 kB/s) 
Selecting previously unselected package node-less.
(Reading database ... 593267 files and directories currently installed.)
Preparing to unpack .../node-less_3.13.0+dfsg-8_all.deb ...
Unpacking node-less (3.13.0+dfsg-8) ...
Setting up node-less (3.13.0+dfsg-8) ...
Processing triggers for man-db (2.10.2-2) ...
dzu@krikkit:~$ 

So taking care to uninstall the Ruby less compiler and now using lessc from the node-less package indeed yielded the diff I expected in the first place:

diff --git a/v8/zen/assets/css/theme.css b/v8/zen/assets/css/theme.css
index 564783a..6bc845f 100644
--- a/v8/zen/assets/css/theme.css
+++ b/v8/zen/assets/css/theme.css
@@ -241,7 +241,7 @@ section {
 .social {
   float: left;
   min-height: 100%;
-  background: url('../images/blue-mocha-grunge.jpg') #192029;
+  background: url('../images/flipflop-texture.png') #192029;
   position: absolute;
   top: 0;
   bottom: 0;
diff --git a/v8/zen/assets/images/flipflop-texture.png b/v8/zen/assets/images/flipflop-texture.png
new file mode 100644
index 0000000..7e34b2d
Binary files /dev/null and b/v8/zen/assets/images/flipflop-texture.png differ
diff --git a/v8/zen/less/variables.less b/v8/zen/less/variables.less
index 2b9b634..6f6464c 100644
--- a/v8/zen/less/variables.less
+++ b/v8/zen/less/variables.less
@@ -36,7 +36,7 @@

 // Resources
 //----------
-@sidebarBackground: url('../images/blue-mocha-grunge.jpg') @darkBlue;
+@sidebarBackground: url('../images/flipflop-texture.png') @darkBlue;
 @primaryBackground: url('../images/cream-dust.png') @white;

 // Typography

So beware, even though less seems to be a very primitive language, the output of different compilers can really be different!

New icon for Codeberg

While reading up on the Fontawesome icons, I decided to also include a link to my Codeberg space into the navigational bar. Unfortunately, the real Codeberg icon is not in the Fontawesome set, there are a lot of icons there with a relation to git. In the end I decided to go for git-alt.

Correct syntax highlighting for shell sessions in Org mode posts

Writing my blog posts in ReST or in Asciidoc the post Syntax Colors For My Blog already described how pygments is being used to do the actual syntax coloring of the content. For various reasons I switched to the Org mode for Emacs syntax in the meantime for posts and even though most things worked as expected, I could not yet get syntax colors for shell-sessions to work at all. I.e. marking sources with console or shell-session did not result in colorful output. Realizing that the Nikola plugin for Org mode was probably something that I could now make sense of with my broadened understanding of the Nikola ecosystem, I dived right in and sure enough found the problem in less than 5 minutes. The plugin consists mostly of a python function to be called with two filenames, the source and the destination. The bulk of the work is done by Emacs itself. From the command line, it is instructed to evaluate the contents of init.el and finally the ELisp function nikola-html-export contained in this file is called to do the hard work.

This function calls another function org-html-src-block defined here to "translate" the babel names for source blocks to the lexer name that pygments understands. And sure enough this function uses the alist org-pygment-language-alist for this mapping. Simply adding entries for console and shell-session also fixes this problem, so I worked it into a pull request for the Nikola plugins repository.

Enable syntax highlighting for diff output

As this was so easy, adding yet another two entries for diff and udiff was easy, so I also enabled them in my blog. The results are already used in this post showing the diffs while changing the background picture.

Lessons learned along the way

Firefox cache

Especially while testing the changes for the self-hosting, I realized that I need to clear the cache of Firefox, because otherwise I will simply not see the Google download in the web console if the copy in the cache is new enough. To be sure that no file is downloaded from the Google Font Server, the cache needs to be cleared before reloading the page.

As a regular user, I thought going through the "Settings" menu, searching for the "Privacy & Security" section, scrolling down to the "Cookies and Site Data" section, clicking on "Clear Data…", deselecting "Cookies and Site Data" (leaving only "Cache" checked) and pressing "Clear" was the easiest way to do this. Note that those are 6(!) actions to perform this step. Somehow I was sure there would be a quicker way and so I started to search the Interwebs again. And lo and behold, in Emacs parlance the key combination "C-S-<delete>" (i.e. Control-Shift-DeleteKey) will pop up the "Clear Recent History" Menu from Firefox:

firefox-clear-cache.png

Once you have only "Cache" selected, pressing Enter will clear only the cache. This is a much more to my taste. When you need to clear the cache even a few times, the keyboard replacement will save you a lot of nerves (and time!).

Summary

The beauty of Free Software is that you are usually able to fix your own problems if you are willing to invest enough to time to learn and understand the context "close" to your specific problem. Even though this appears to be very time-consuming, you will usually learn a lot of things along the way. And you simply have to feel the satisfaction of having fixed a specific problem in a larger code base for yourself. It is worth it!

Comments

Comments powered by Disqus