rlworkbench

rlworkbench is my project where I try to build a language workbench. What is that? What am I trying to build?

The way I think about it is that rlworkbench should be a text editor that is aware of the language of the text that is being edited. That awareness can give us syntax highlighting for example. But it can also give us more if it knows more about the language. For example, there might be an editor operation to expand the selection. If the cursor is inside a function, expanding the selection might select the whole function. Because the editor is aware of the language, it knows where the function starts and ends. Furthermore, defining the languages that rlworkbench knows about should be easy. And the norm should be to create many small domain specific languages. And when we define a language, it not only gives us editor support, but we can also define how that language should be compiled or translated.

Screenshots

Here is rlworkbench editing the README file. A markup language has been defined and it provides both the syntax highlighting as well as the ability to turn the README file into HTML for display on this webpage.

Blog

When I write about rlworkbench on my blog, I tag the posts #rlworkbench.

Language Definitions

Languages are defined in a meta language called meta. Meta programs are turned into C programs (using the meta compiler) which are then turned into binaries using a C compiler.

+-------------------+       +----------------+       +--------------+
|                   |       |                |       |              |
| asciidiagram.meta | ----> | asciidiagram.c | ----> | asciidiagram |
|                   |       |                |       |              |
+-------------------+       +----------------+       +--------------+

What does a meta program look like? Let's have a look at an example that defines a language for ascii diagrams used to highlight the diagram above:

main = element*;

element =
    | arrow
    | boxBorder
    | .
    ;

arrow[Preprocessor] =
    | '-' '-'* '>'
    ;

boxBorder[Reserved] =
    | '-'
    | '+'
    | '|'
    ;

It defines syntactic elements of an ascii diagram and highlights different parts differently.

The compilation process is as follows. First, the meta program is turned into a C program:

meta <asciidiagram.meta >asciidiagram.c

Then the C program is turned into a binary:

gcc asciidiagram.c -o asciidiagram

Now, what can this binary do? By default it accepts asciidiagram programs on stdin and spits out how they are transformed to stdout:

echo "----> what?" | ./asciidiagram 2>/dev/null

----> what?

The asciidiagram language does not specify any transformations, so the consumed characters are printed as is.

We can also call asciidiagram with the highlight argument which then prints the input program to the terminal with syntax highlighting:

echo "----> what?" | ./asciidiagram highlight

----> what?

Alternatively it can spit out highlighted HTML with the --html flag:

echo "----> what?" | ./asciidiagram highlight --html

<pre class="highlight"><span class="Preprocessor">-</span><span class="Preprocessor">-</span><span class="Preprocessor">-</span><span class="Preprocessor">-</span><span class="Preprocessor">&gt;</span> what?
</pre>

We have to provide CSS for the different classes.

Let's look at another language that does include transformations. This is a simplified markup to HTML processor:

main = element*:xs -> { xs };

element =
    | '*' emInner:x '*' -> { "<em>" x "</em>" }
    | char:x            -> { x }
    ;

emInner[Meta] = nonStar*:xs -> { xs };

nonStar = !'*' char:x -> { x };

char =
    | '<' -> { "&lt;" }
    | '>' -> { "&gt;" }
    | '&' -> { "&amp;" }
    | .
    ;

Here is how it transforms an input string:

echo 'Is 2<1? No, *2>1*!' | ./examplemarkup 2>/dev/null

Is 2&lt;1? No, <em>2&gt;1</em>!

All languages that rlworkbench knows about also automatically get syntax highlighting in the editor.

Changelog

1.8.0

• Extract some languages to markup2website

• Only attempt to load a language that has a name

1.7.0 (2026-05-21)

• Extract languages to plugins

1.6.0 (2026-05-15)

• sh: Add sh language for highlighting of shell script

• python: Improve highlighting (while keyword)

1.5.1 (2026-05-12)

• Bugfix missing commas

1.5.0 (2026-05-12)

• python: Improve highlighting

• c: Improve highlighting

1.4.0 (2026-02-07)

• markup: Fix that a file can start with empty lines and then a block

• markup: Allow line end to be EOF to not require newlines at end of file

1.3.0 (2026-01-10)

• markup: Add syntax to highlight a file

• python: Highlight pass

1.2.0 (2026-01-10)

• markup: Recognizes .md extension as a markup file

1.1.0 (2026-01-09)

• markup: Allow escaping [ and ]

• markup: Support blockquotes >

• markup: Support numbered lists

• python: First version of basic keyword lexing

1.0.0 (2026-01-04)

• First version (used by markup2website)

TODO

Structure/C

• Common error handling for all parse errors

• Examine errors from -fsanitize=undefined

• Generate highlight.c from high level description (X-macro style)

• Generate input C files in out/c and add only that to include path

• Handmade style

• Program in a style where you can choose the language (abstraction level) at choice, and always have to possibility to go low level when needed

  • Code re-use is bad for performance. It does what you want, but not in the optimal way for this problem

    • Allow layers of abstractions to be shaved off if the problem is more easily solved at a lower level

• How do I do c projects?

  RLC?
  No build system. Only C compiler.
  First compile rlmeta2.c
  Then rlc.rlmeta2 -> RLC.c -> RLC
  Then main.rlc -> main.c -> main
  RLC should have high level syntax for low level stuff. Like nothing?
  

• Make it bootstrapable with only a C compiler (any C compiler)

  • Experiment: fork+exec to run process (Which header? Which linker?)

    • https://github.com/tsoding/nob.h/blob/main/nob.h#L1136-L1226

  • https://dependablec.org/

  • https://github.com/tsoding/nob.h (idea for a build system)

  • Notes:

    gcc -o out/meta src/meta/meta.c
    out/meta <src/make/make.build >out/make.c
    gcc -o out/make out/make.c
    out/make
    

    • How to make make.c platform independent?

      • Build Windows platform layer in wine?

        • i686-w64-mingw32-gcc -Wfatal-errors -Werror -o $2 $1

        • It adds .exe extension

Platform layer

• Use SDL_HINT_MAIN_CALLBACK_RATE=waitevent and animate cursor differently

Test framework

• Convert learningc to tests

Editor

• Prevent close if files is not saved

• Safe write (to prevent corruption)

• : should open command selection window

  • :w should save file

• ctrl-t should be mapped to command :edit (and it opens file section window)

• Support for undo/redo

• Fun "character explode" animation when saving

• Show related functions inline so that there is no need to jump

• Show a tree view of your folder/files in one view

  • You switch to another file by just navigating to a different location in this giant document

  • The document can collapse subsections to make the structure clear

• TempleOS style autocomplete

• Hot reload languages (loaded as dynamic libraries)

Highlighting

• More generic names for MetaHighligt enum members

• Base highlight on top-level items and only re-parse the top-level section that changed (that is how 10x Editor works according to If you're serious about programming, listen to Stewart Lynch)

Language integration

• Create "compile <file>" command (convert sh script to native command)

• Spell check language

Rendering

• Make glyph cache support utf-8

  • Got idea for implementation from hash map that Steward talked about in If you're serious about programming, listen to Stewart Lynch

    • Learned more about hash maps from HashMaps & Dictionaries, Explained Simply

Markup language

• Highlight code blocks in editor (not only when rendering html)

• Should we handle html escape in meta or C? Or both as it is now?

  • Define function in meta and use it from C? String -> String?

• tt -> code?

Meta language/compiler

• Should String use unsigned char?

• Add convenience command to compile a standalone language

  rlworkbench_sdl language compile --standalone <language>.meta
  

  Versus

  rlworkbench_sdl run <language>.meta > <language>.c
  cc -DSTANDALONE <language>.c -o <language>
  

• Allow parsing only (no skip overhead of actions when not used)

• Generalize unique to work on any expression (to replace unseen)?

• Pass Buffer as value instead of allocating on heap?

• Flags for more compact data structure for Chunk?

• Detect infinite loop -> OOM ((rule*)*)

• Optimize 'or' based on first character

  keyword = 'foo' | 'bar' | 'baz' | 'return' | other
  ->
  switch (first_char) {
      case 'f':
          keyword = 'foo' | other
      case 'b':
          keyword = 'bar' | 'baz' | other
      case 'r':
          keyword = 'return' | other
  }
  

  • How to make this transform?

• Add support for unicode character classes?

• Bind variable without action causes error in meta rule = foo:x;

• Implicit counters for star operator? Se aoc20254

• Be able to rotate text with dynamic number of string builders? See aoc20255