Skip to content

Code-highlight c_decl blocks using custom jinja2 filter extension

FeRD (Frank Dana) requested to merge ferdnyc/gi-docgen:highlighted-c_decls into main

Like !206 (merged) this change was initially undertaken to correct invalid HTML in the generated docs. It... grew a bit, along the way. (Though it does achieve that goal, and much more.)

c_decl syntax highlighting

The current gdgenerate.py code attempts to output syntax-highlighted versions of the c_decl property for some nodes by defining its value with utils.code_highlight(). There are a few issues with that:

  1. It has to be careful about doing it, because some nodes' .c_decl properties are used to generate other .c_decl properties. (e.g. a func.c_decl will use arg.c_decl for each argument it takes, in generating the declaration string.) So, only wrap leaf nodes.
  2. The resulting syntax-highlighted code is then inserted into templates containing <pre><code>{{ foo.c_decl }}</code></pre>, resulting in invalid HTML. An example from method.Window.set_application.html:
    <div class="docblock c-decl">
        <pre><code><div class="highlight"><pre><span></span><span class="kt">void</span>
    <span class="n">gtk_window_set_application</span><span class="w"> </span><span class="p">(</span>
    <span class="w">  </span><span class="n">GtkWindow</span><span class="o">*</span><span class="w"> </span><span class="n">window</span><span class="p">,</span>
    <span class="w">  </span><span class="n">GtkApplication</span><span class="o">*</span><span class="w"> </span><span class="n">application</span>
    <span class="p">)</span>
    </pre></div>
    </code></pre>
      </div>
    </div>
    <div><pre><code><div><pre>...</pre></div></code></pre></div> will never fly ­-- <code> can't contain block elements.
  3. And, after all that, joke's on us... the code-highlighting doesn't work anyway, because it uses the default "highlight" class and the CSS for pygments uses the markdown extension class of "codehilite".

Item 1 in particular didn't sit right with me — adding highlighting is a presentation function, not a parsing function, so doing it in the generator felt wrong. And anyway, we have this nice templating system for defining presentation.

So, in exactly the opposite approach I used in !206 (merged), this MR removes the calls to utils.code_highlight() when generating the node data, and instead installs a custom extension in the jinja2 environment, gidocgen.jinja_extension.HighlightExtension. That very simple extension creates a new filter for the templating system, |highlight, which applies syntax highlighting. So, the templates' code like this:

<div class="docblock">
  <pre><code>{{ foo.c_decl }}</code></pre>
</div>

instead becomes this:

<div class="docblock">
  {{ foo.c_decl|highlight }}
</div>

...and pygments' highlighted version of the foo.c_decl value is inserted with the correct DOM structure. The filter also sets the pygments cssclass parameter to "codehilite", same as the Markdown extension, so the existing CSS for highlighting is applied as expected.1

Style changes

So, that was all well and good, and now the classes' declarations were being output with syntax highlighting as valid HTML.

And it looked terrible.

The existing syntax-highlighting (for code blocks in markdown, as well as the CSS node trees) were using the pygments Solarized themes, both light and dark variants. Solarized is an extremely low-contrast theme which can be very hard to read with certain code, especially its dark variant. So now I had my declarations syntax-highlighted, and they went from Meh to Ew:

Meh

image

Ew

image

So, that's no good at all.

Long story long, I replaced the solarized theme CSS with pygments' default and github-dark themes for light and dark modes (respectively). Now, the highlighted declarations are not only valid HTML, but they look like this:

image image

(Yay! \o/)

Notes

  1. (Added bonus, with the filter: The pygments lexer applied to the code can be configured both on the jinja environment, by setting gidocgen_highlight_lexer, and on a per-use basis by passing an arg to the filter. So, {{ foo|highlight("java") }} would highlight foo as Java code even if gidocgen_highlight_lexer is set to the default "c". ...In theory. There's no actual use for that ability, here, so beyond ensuring it didn't break anything I haven't really tested that feature. But it's there, and either works or should be easily fixable.)

Merge request reports