Blog internals (or learning new things to fix stuff)
Recently I was made aware of a consequential GDPR ruling in Germany. The summary of the ruling is that a website owner was sentenced to pay a €100 fine to a visitor of his website because the website used Google hosted fonts. The visitor claimed that by loading the Google fonts, his IP address was exposed to the Google servers and this violates the GDPR.
Effectively this exposes every website in Germany using Google hosted web fonts to become the target of an €100 fine. At first, it seemed like a theoretic problem, but more recently it seems that people have turned this into a business-model (link is in German) and there now seem to be a wave of incidents based on this ruling. So it has become pretty urgent to fix this for web site owners in Germany.
Checking my Nikola generated site, I found that the zen theme that I use not only uses Google hosted fonts but also the Fontawesome icon set hosted on its own server. So I quickly decided that I need to switch those over to self-hosting before I get one of these lawyer mails. This blog post documents the things I learned on this journey which is still not over.
And I apologize in advance that this post has become rather large and very technical, but the things are really connected together so I decided not to split them out into smaller posts. I sincerely hope this was the right thing to do.
Finding out what to do
As I am not a web programmer, I tried to get an idea of what is
required to self-host the Google fonts and the Fontawesome icon set.
Searching the web quickly yielded some articles for the Google fonts
problem and Fontawesome has pretty good documentation itself on how to
self-host. So the next step would be to integrate this into the zen
theme that I use and see how much of this can and should be
contributed back to the Nikola project.
Unfortunately until now I used Nikola as a black box that (mostly) did
what I expected of it, but to solve this issue I obviously needed to
learn more about the inner workings of it. With my specific changes
in mind I decided to concentrate my learning on the theming aspect of
Nikola, i.e. on the help documents Creating a Theme and Theming
Nikola. This entailed reading up on other topics related to web
development as well. Studying Cascading Style Sheets, for example, I
realized that "basic CSS" seems to be so uncomfortable that people
came up with higher level languages compiling into CSS, like sass or
less. The zen
theme has this to say in its README.md
file:
WARNING: The themes use Less for their styles, but you don't need a Less compiler installed to use it.
So we should keep in mind that theme.css
is the output of a compiler
with the source files located in the less/
directory.
How themes work in Nikola
Nikola comes with 6 themes contained in the Nikola repository itself:
- base, base-jinja
- bootblog4, bootblog4-jinja
- bootstrap4, bootstrap4-jinja
Apart from those, there are also quite a lot of community themes. Looking at the themes gallery, it is not obvious at all, which of the themes are "first level" themes living next to the code itself and which live in their own nikola-themes GitHub repository. Once you decide to use a Nikola theme, you will install a copy of it into the directory you build your blog from and thus loose the connection to improvements in the upstream repository. Unless you check yourself from time to time if something happened there, you will not be able to leverage any subsequent changes or fixes in the upstream theme. If you do find there were changes, there is no easy way to upgrade your theme then to remove your old one, install the new one and see if things still work as you expect.
As I was sure my problem is also relevant for a lot of Nikola users, I explicitly set out to fix things in the proper "upstream" repository. It turns out that community Nikola themes have their own and community Nikola plugins (see below) also live in their own plugins GitHub repo.
Establish a clean baseline
As we learned, the theme used in the blog is a copy of the upstream
theme at the time of theme installation. So before doing anything, I
decided that I would need to synchronize the theme used in my blog
with its current version in the upstream repository. So I renamed
theme/zen
to theme/zen-old
and re-installed the zen
theme
according to the manual. Having a clean slate would now allow me to
introduce my changes to self-host the fonts.
To be on the safe side, I wiped the whole output/
directory of my
blog and recreated the site from scratch. As I was only doing
incremental builds over the last years, I was also interested in how
long a complete build would take - turns out its around 11 seconds.
Uh, an error in the web console?
Opening the newly recreated blog on my local machine looked okay, but opening the web console of Firefox showed a prominent error:
Uncaught ReferenceError: moment is not defined
As I was expecting to start my changes from a clean state, I was
surprised to see this error coming from the upstream code base. So I
tried inquiring on the Nikola mailing list what this was all about.
Not getting any helpful information, I tried to find the reason for
myself. After some poking I found that the implementation of "fancy
dates" (turning absolute date and timestamps into something like "10
hours ago") in the base themes was switched in commit f1d480d8 from a
solution using moment-with-locales.min.js
to a solution using
luxon.min.js
. So it looks like this change in the core Nikola
repository was not propagated to at least some)the zen
theme in the
community themes repository.
So before even considering the changes required for self-hosting, I decided to fix this bug while still here.
Fixing things in Nikola
A closer look at the "zen" family of themes
On a cursory look, the community themes include three themes with
zen
in their names:
- zen
- zen-jinja
- zen-forkawesome
At the beginning I thought I would only deal with zen
itself, but
learning more about the inner workings of themes, I discovered that
zen-jinja
is actually derived with the script jinjify.py from the
zen
templates. While zen
uses the Mako template engine,
zen-jinja
uses (what a surprise) the jinja library. As this is
outside my expertise, I had no idea which one is "better" (for a
meaningful interpretation of "better", i.e. in terms of design, larger
community, etc.) but at least I found a discussion from 2017 in the
Mako Development mailing list where the creator of Mako (Mike Bayer)
writes:
I can tell you that Jinja is a lot more polished than Mako at this point. It was developed later with a more organized concept to start with, and it has multiple developers now continuing to put a lot of work into it. Mako of course works great and it's still what I use, but that's because I wrote it :). I don't have the resources to maintain it beyond keeping it basically working these days and I've usually never been able to get dedicated long-term helpers on any of my projects.
Along the way I also found that jinja
is used in the very popular
ansible orchestration package and thus from a practical perspective
probably has a lot more users than Mako today. But even with jinja
having a slight advantage, for "twin" themes inside Nikola having both
a Mako and a Jinja version, it will be easier to do the changes in the
Mako templates and update the twin with the script. So whenever I
changed something in zen
, I used the script to update zen-jinja
also for the twins to stay synchronized.
zen-forkawesome
on the other hand turns out to be a theme derived
from zen
, replacing the Fontawesome icon set with the Fork-Awesome
icons. "Deriving" in this context means the theme references zen
as
its base in zen-forkawesome/parent
and provides own versions of
base_helper.tmpl
and index.tmpl
with the changes to replace
Fontawesome with Fork-Awesome. But the fact that this "derivation"
includes a copy of all the text of the base templates, means that we
also have to fix these files with similar fixes that we used for the
files in zen
. There is no tool for these changes, so we have to
derive them manually. This is a consequence of the very primitive
"template inheritance" mechanism being used. As we saw, "inheriting"
really means to create a copy of a file and only change minor details
while keeping the rest in place, but without preserving any
relationship to its ancestor that tools could use.
Pull request for nikola-themes
In essence I knew enough to fix the problems, but I would need to make
changes similar to the changes in the base Nikola themes for all the
zen
family. This resulted in a pull request on nikola-themes.
With this in place, I could proceed to the self-hosting, but during the previous work I noticed that the versions of Fontawesome and Fork-Awesome were actually outdated. So why not update them while we are here, so we do not invest work in outdated versions?
Updating the icon sets
Updating the version of Fontawesome is done in commit zen: Update
fontawesome from v5.0.13 to 6.1.2 in my clone of the nikola-themes
repo (remember it fixes zen-jinja
also!). The update of forkawesome
is in commit zen-forkawesome: Update from v1.1.7 to v1.2.0 . All of
these fixes are now in the zen-self-host-fonts-fa branch in my clone.
Honoring 'updated'
During all of this fixing, I came across the updated
metadata (see
section 'Metadata field' in the Nikola handbook) of a post and
realized that I very much would like to have this on my blog. It
happened already a few times that I made updates to my blog posts in
the form of changes summarized at the bottom under an 'Update'
section. Although this update section contains the date of the
changes, this date was not visible in the header area in my blog.
Researching this question, I found that (of course) it needs separate
machinery in the templates and the zen
family did not have this
machinery.
Fixing this results in the commit zen family: Add handling for 'updated' pages taken from base.
The effect of this change can be seen for example in the jq, xq and
yq - Handy tools for the command line post on my blog. Just beneath
the header there is now the text "2 years ago (updated 11 months ago)"
which is a very nice summary of the facts provided by the date
and
updated
metadata for the post. You can check yourself by looking
at the source of the post.
Self-hosting, finally!
With all this in place, downloading the files for Google Fonts and for Fontawesome and including their usage in the templates was not a big deal anymore:
Self-hosting zen-forkawesome
remains to be done, as I wanted to
gather feedback on the other changes before doing this.
Background from XScreenSaver
And finally as I learned that the background is of course also replaceable an idea began to form in my mind. Only recently Jamie Zawinski announced the 30th anniversary of XScreenSaver and this reminded me of my time at TU Karlsruhe when I was staring a lot at the graphical gems contained in the XScreenSaver collection. By default, XScreenSaver seems not to be installed in a standard Debian Bookworm installation anymore, but a quick
dzu@krikkit:~$ sudo apt install xscreensaver \
xscreensaver-data-extra xscreensaver-gl-extra
fixes the situation for me. This allows me to start the GUI for browsing the included screensaver modules:
dzu@krikkit:~$ xscreensaver-demo
The list is so long that I only looked at a subset from the beginning of the list. But many entries brought back memories from some 25 years ago.
After a while I decided a snapshot from flipflop would be nice to
have. Playing around with the parameters I found that the installation
in Debian puts all the binaries into /usr/libexec/xscreensaver
, so
they are not on the regular PATH
for any user. XScreenSaver also
has a tool called xscreensaver-getimage
which is called by some
graphical demos to retrieve a screenshot, a random picture or even a
snapshot from the webcam. For flipflop
the option --texture
will
make it call out to that program to get a random picture and put it
inside a specific window. But calling flipflop --texture
from the
command line does not have a correct PATH
setup and so
xscreensaver-getimage
cannot be found. This results in a
black/dark-grey test pattern to end up being used in the demo hack.
Even though it's the result of a user error, I found the irony somehow
fitting with my blog as I do write about problems quite a lot.
So after taking a screenshot of flipflip
and cropping it to the
same x dimension of the original background of the zen
theme I had a
.png file I wanted to use. Copying it into the assets/images
directory in the themes/zen
folder in the blog itself will copy it
to the output/
directory when building the blog. Searching for the
exact place where the old background is specified, I found the file
name referenced in assets/css/theme.css
. Changing it there is easy,
but remember what we found in the beginning of the post, namely that
theme.css
is a file generated by a less compiler. So the proper
place to change the variables is in the source file
less/variables.less
:
// Resources
//----------
@sidebarBackground: url('../images/blue-mocha-grunge.jpg') @darkBlue;
@primaryBackground: url('../images/cream-dust.png') @white;
lessc and lessc
Having changed this variable definition, one has to compile the less
code into theme.css
. Googling around for less compilers I happened
to come across the Ruby version first. Installation went about right
after updating my Ruby installation on my Debian machine. Quickly
there was a /usr/local/bin/lessc
, so I could run make
in the
zen
theme directory to compile the less code. Checking the magit
(Emacs git "user interface") diff output, of course I expected only
the file name to change but nothing else. But the diff was rather
large:
diff --git a/v8/zen/assets/css/theme.css b/v8/zen/assets/css/theme.css
index 6bc845f..4378fb2 100644
--- a/v8/zen/assets/css/theme.css
+++ b/v8/zen/assets/css/theme.css
@@ -232,7 +232,7 @@ html {
}
body {
height: 100%;
- background: url('../images/cream-dust.png') #fff;
+ background: url('../images/cream-dust.png') #ffffff;
}
section {
margin: 0;
@@ -241,7 +241,7 @@ section {
.social {
float: left;
min-height: 100%;
- background: url('../images/flipflop-texture.png') #192029;
+ background: url('../images/blue-mocha-grunge.jpg') #192029;
position: absolute;
top: 0;
bottom: 0;
@@ -443,8 +443,8 @@ dd {
padding: 4px 12px;
line-height: 24px;
text-decoration: none;
- background-color: #fff;
- border: 1px solid #ddd;
+ background-color: #ffffff;
+ border: 1px solid #dddddd;
border-left-width: 0;
}
.pagination ul > li > a:hover,
@@ -455,14 +455,14 @@ dd {
}
.pagination ul > .active > a,
.pagination ul > .active > span {
- color: #999;
+ color: #999999;
cursor: default;
}
.pagination ul > .disabled > span,
.pagination ul > .disabled > a,
.pagination ul > .disabled > a:hover,
.pagination ul > .disabled > a:focus {
- color: #999;
+ color: #999999;
background-color: transparent;
cursor: default;
}
@@ -591,7 +591,7 @@ dd {
.pager .disabled > a:hover,
.pager .disabled > a:focus,
.pager .disabled > span {
- color: #999;
+ color: #999999;
background-color: #fff;
cursor: default;
}
@@ -599,10 +599,10 @@ body {
font-size: 16px;
font-size: 15px;
line-height: 24px;
- color: #333;
+ color: #333333;
}
a {
- color: #08c;
+ color: #0088cc;
text-decoration: none;
}
a:hover,
@@ -611,7 +611,7 @@ a:focus {
text-decoration: underline;
}
.social a {
- color: #fff;
+ color: #ffffff;
line-height: 44.4px;
-webkit-transition: text-shadow 0.5s;
-moz-transition: text-shadow 0.5s;
@@ -629,7 +629,7 @@ a:focus {
.social a:hover,
.social a:focus {
text-shadow: 0 0 10px rgba(255, 255, 255, 0.8);
- color: #fff;
+ color: #ffffff;
text-decoration: none;
opacity: 0.9;
filter: alpha(opacity=90);
@@ -697,7 +697,7 @@ a:focus {
.page-content > .content .h6 small {
font-weight: normal;
line-height: 1;
- color: #999;
+ color: #999999;
}
.page-content > .content h1,
.page-content > .content .h1,
@@ -765,7 +765,7 @@ a:focus {
blockquote {
padding: 0 0 0 15px;
margin: 0 0 24px;
- border-left: 5px solid #eee;
+ border-left: 5px solid #eeeeee;
}
blockquote p {
margin-bottom: 0;
@@ -773,7 +773,7 @@ blockquote p {
blockquote small {
display: block;
line-height: 24px;
- color: #999;
+ color: #999999;
}
blockquote small:before {
content: '\2014 \00A0';
@@ -782,7 +782,7 @@ blockquote.pull-right {
float: right;
padding-right: 15px;
padding-left: 0;
- border-right: 5px solid #eee;
+ border-right: 5px solid #eeeeee;
border-left: 0;
}
blockquote.pull-right p,
@@ -865,7 +865,7 @@ address {
}
a.thumbnail:hover,
a.thumbnail:focus {
- border-color: #08c;
+ border-color: #0088cc;
-webkit-box-shadow: 0 1px 4px rgba(0, 105, 214, 0.25);
-moz-box-shadow: 0 1px 4px rgba(0, 105, 214, 0.25);
box-shadow: 0 1px 4px rgba(0, 105, 214, 0.25);
@@ -878,7 +878,7 @@ a.thumbnail:focus {
}
.thumbnail .caption {
padding: 9px;
- color: #555;
+ color: #555555;
}
code,
tt,
@@ -886,7 +886,7 @@ pre {
padding: 0 3px 2px;
font-family: Monaco, Menlo, Consolas, "Courier New", monospace;
font-size: 13px;
- color: #333;
+ color: #333333;
-webkit-border-radius: 3px;
-moz-border-radius: 3px;
border-radius: 3px;
@@ -971,7 +971,7 @@ article.post-text .backlink + h1.title {
}
article.post-text .metadata {
*zoom: 1;
- color: #999;
+ color: #999999;
margin-bottom: 12px;
margin-top: -6px;
}
@@ -990,14 +990,14 @@ article.post-text .metadata .commentline {
article.post-text .metadata p {
margin: 0;
display: inline-block;
- color: #999;
+ color: #999999;
}
article.post-text .metadata p a {
- color: #999;
+ color: #999999;
}
article.post-text .metadata p a:hover,
article.post-text .metadata p:hover {
- color: #333;
+ color: #333333;
-webkit-transition: color 1s linear;
-moz-transition: color 1s linear;
-o-transition: color 1s linear;
@@ -1073,7 +1073,7 @@ article.post-text .entry-content .figure > a > img {
}
article.post-text .entry-content .figure .caption {
padding: 9px;
- color: #555;
+ color: #555555;
font-style: italic;
padding-bottom: 0;
}
So somehow it seemed that the less code did not correspond to the
compiled file theme.css
. I found that strange, but it is a possible
consequence of using the compiled sources in a version controlled
repository. Changes might escape attention, and so I streamlined the
pure re-creation of theme.css
through the lessc
compiler into its
own git commit and proceeded to change the variable in a subsequent
commit yielding a proper, short, diff only for the filename. Only
later, while reading up more on the less
language, I found out that
there is yet another compiler for the node
ecosystem and also easily
installable on my Debian machine with
dzu@krikkit:~$ sudo apt install node-less
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
node-less
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 525 kB of archives.
After this operation, 4346 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bookworm/main amd64 node-less all 3.13.0+dfsg-8 [525 kB]
Fetched 525 kB in 0s (3617 kB/s)
Selecting previously unselected package node-less.
(Reading database ... 593267 files and directories currently installed.)
Preparing to unpack .../node-less_3.13.0+dfsg-8_all.deb ...
Unpacking node-less (3.13.0+dfsg-8) ...
Setting up node-less (3.13.0+dfsg-8) ...
Processing triggers for man-db (2.10.2-2) ...
dzu@krikkit:~$
So taking care to uninstall the Ruby less compiler and now using
lessc
from the node-less
package indeed yielded the diff I
expected in the first place:
diff --git a/v8/zen/assets/css/theme.css b/v8/zen/assets/css/theme.css
index 564783a..6bc845f 100644
--- a/v8/zen/assets/css/theme.css
+++ b/v8/zen/assets/css/theme.css
@@ -241,7 +241,7 @@ section {
.social {
float: left;
min-height: 100%;
- background: url('../images/blue-mocha-grunge.jpg') #192029;
+ background: url('../images/flipflop-texture.png') #192029;
position: absolute;
top: 0;
bottom: 0;
diff --git a/v8/zen/assets/images/flipflop-texture.png b/v8/zen/assets/images/flipflop-texture.png
new file mode 100644
index 0000000..7e34b2d
Binary files /dev/null and b/v8/zen/assets/images/flipflop-texture.png differ
diff --git a/v8/zen/less/variables.less b/v8/zen/less/variables.less
index 2b9b634..6f6464c 100644
--- a/v8/zen/less/variables.less
+++ b/v8/zen/less/variables.less
@@ -36,7 +36,7 @@
// Resources
//----------
-@sidebarBackground: url('../images/blue-mocha-grunge.jpg') @darkBlue;
+@sidebarBackground: url('../images/flipflop-texture.png') @darkBlue;
@primaryBackground: url('../images/cream-dust.png') @white;
// Typography
So beware, even though less
seems to be a very primitive language,
the output of different compilers can really be different!
New icon for Codeberg
While reading up on the Fontawesome icons, I decided to also include a link to my Codeberg space into the navigational bar. Unfortunately, the real Codeberg icon is not in the Fontawesome set, there are a lot of icons there with a relation to git. In the end I decided to go for git-alt.
Correct syntax highlighting for shell sessions in Org mode posts
Writing my blog posts in ReST or in Asciidoc the post Syntax Colors
For My Blog already described how pygments
is being used to do the
actual syntax coloring of the content. For various reasons I switched
to the Org mode for Emacs syntax in the meantime for posts and even
though most things worked as expected, I could not yet get syntax
colors for shell-sessions to work at all. I.e. marking sources with
console
or shell-session
did not result in colorful output.
Realizing that the Nikola plugin for Org mode was probably something
that I could now make sense of with my broadened understanding of the
Nikola ecosystem, I dived right in and sure enough found the problem
in less than 5 minutes. The plugin consists mostly of a python
function to be called with two filenames, the source and the
destination. The bulk of the work is done by Emacs itself. From the
command line, it is instructed to evaluate the contents of init.el
and finally the ELisp function nikola-html-export
contained in this
file is called to do the hard work.
This function calls another function org-html-src-block
defined here
to "translate" the babel names for source blocks to the lexer name
that pygments
understands. And sure enough this function uses the
alist org-pygment-language-alist for this mapping. Simply adding
entries for console
and shell-session
also fixes this problem, so
I worked it into a pull request for the Nikola plugins repository.
Enable syntax highlighting for diff output
As this was so easy, adding yet another two entries for diff
and
udiff
was easy, so I also enabled them in my blog. The results are
already used in this post showing the diffs while changing the
background picture.
Lessons learned along the way
Firefox cache
Especially while testing the changes for the self-hosting, I realized that I need to clear the cache of Firefox, because otherwise I will simply not see the Google download in the web console if the copy in the cache is new enough. To be sure that no file is downloaded from the Google Font Server, the cache needs to be cleared before reloading the page.
As a regular user, I thought going through the "Settings" menu, searching for the "Privacy & Security" section, scrolling down to the "Cookies and Site Data" section, clicking on "Clear Data…", deselecting "Cookies and Site Data" (leaving only "Cache" checked) and pressing "Clear" was the easiest way to do this. Note that those are 6(!) actions to perform this step. Somehow I was sure there would be a quicker way and so I started to search the Interwebs again. And lo and behold, in Emacs parlance the key combination "C-S-<delete>" (i.e. Control-Shift-DeleteKey) will pop up the "Clear Recent History" Menu from Firefox:
Once you have only "Cache" selected, pressing Enter will clear only the cache. This is a much more to my taste. When you need to clear the cache even a few times, the keyboard replacement will save you a lot of nerves (and time!).
Summary
The beauty of Free Software is that you are usually able to fix your own problems if you are willing to invest enough to time to learn and understand the context "close" to your specific problem. Even though this appears to be very time-consuming, you will usually learn a lot of things along the way. And you simply have to feel the satisfaction of having fixed a specific problem in a larger code base for yourself. It is worth it!
Comments
Comments powered by Disqus